I do not think that you can access that just by dropping the pan. The pan will only give you access to the filter and bottom of the valve body.
The switch is on the outside of the selector shaft which goes through the case and should link to the other side of the valve body.. this is how it is on other toyotas, at least
im almost certain that you'll have to drop the valve body and disconnect everything from the selector shaft, then install a new one
or
alternatively there's a possibility that you could use one of those universal linkage clamps to make due as a temporary solution
this feels risky though i personally would replace the selector shaft. its not something that requires expensive tools, just make sure you keep careful inventory of your bolts and you look up proper torque specs
i think 35561-60030 is the part number for our model year with the 4 speed trans
edit:
upon further consideration maybe a universal linkage clamp really isnt that risky if there's enough space, just need to find the right one
also im not a professional

others here may have better recommendations. the break is at the top of the threads and isn't horrible