I ran into this same issue a few months ago, the cause was the misalignment of the brake pedal bracket against the firewall. That bracket is held in place by the same nuts (using the booster studs) as the booster itself.
So when you remove the nuts on the cabin side of the firewall the brake...