Many moon ago this happened to me once plowing snow and once jockying the truck around just as you describe. Lots of quick 1/R/1/R/1, etc shifts. What I did the first time is wiggle the stik as you describe. The second time that would not work so I had to remove the shifter and align the 'tabs' (for lack of a better term) inside the transmision. You can see them clearly when you remove the shifter. Only explation I can think of is a lot of wear on one or both parts.
I have broke, replaced and rebuilt a lot of things on FJ40's but never a 4speed transmission. They last a long time.