I was in the same boat after a fresh restoration - I found it to be similar, even after adjustment. I ended up putting a rubber strip on the back of the pedal arm; you could also stick a bumper on the firewall.
Based on the OEM paint chipping there, it seemed like the firewall was the...