Sounds like a bad switch. If you have the time you can take the switch apart and clean the contacts and check the springs are not broken or out of place. I would just get a new switch. They are cheap. If you don't have a dual master with the brake warning switches then you only need the two green wires. The red wires are part of the brake check circuit on the dual master.
The attached pic shows how this is wired. The pic is confusing as hell but it is accurate.
The numbers are:
40 is the emergency brake switch
41 is the brake warning light on the dash
42 is the brake switch itself
43 is the turn signal switch. (The brake lights go through the turn signal switch.)
44 are the hydraulic pressure switches on the dual master cylinder.
It works like this.
Key OFF: First Fuse A is a switched hot. It's only hot when the key is on. Fuse B is hot all the time. The hydraulic brake pressure switches (44) are closed when the brakes are NOT applied which grounds the red wire on one side of the switch (42). So with the key off and brakes NOT applied the dash (41) light and the brake lights are off. If you press the brakes in this condition you close BOTH switches in the brake switch. This sends power from Fuse B through the brake switch (42) to the turn signal switch (43) and on to the brake lights which will light. Because the key is not on no power is applied to the dash switch so it will not light even if there is a problem.
Key ON: Now Fuse A is hot. If you pull the emergency brake handle, switch 40 will close. Since the hydraulic pressure switches are already closed there is a complete path from ground through the emergency brake switch through the dash light to the fuse and the Brake Warning light lights telling you the Emergency Brake is on.
Now with the Emergency Brake OFF and key ON. If you press the brake in this condition BOTH switches in the brake switch close. Power from Fuse B goes through the brake switch green wires to the turn signal switch and on to the brake lights which will light. If the master cylinder generates enough pressure in the front and rear circuits then the hydraulic pressure switches (44) will OPEN breaking the circuit to ground and the Brake Warning light on the dash will remain OFF. If however the pressure drops in either circuit due to an internal leak in the master cylinder or a leak in the lines pressure will drop and the switch in that circuit will close. This provides a path to ground to the Brake Warning light which lights telling you there has been a drop in pressure and you should check it NOW. It's normal for the light to flicker when you hit the brakes. HTH's!
View attachment 115762