The balls dont come out those holes unfortunately. You will have to set all the shafts in their center detent position, then remove them one at a time while keeping the other shafts in the center detent position.
Remove the tight plugs on the ends of the plate and push the shaft you want to remove through from the other end.
If you force it while one of the shafts is not in the correct position you WILL break the aluminum plate.
If you are convinced that you need to get the ball out of that hole, then you will need to remove the rolled steel pin holding anything on the shaft you are working on, then rotate the shaft 90 degrees either direction - you will probably have to have the tight plug out and push the shaft in the direction of the hole.
Chris