Ok I figured it out. You do have to get the switch out. What it was is the threads were really tight and it marred the threads just backing it out. I'll clean up the threads when I reinstall it.
Can someone help me with this question. The nut that holds the switch to turn on the light when you open the glove box. Can't seem to get that nut off without getting to the back side of it. Am I overcomplicating removing the glove box.