The thickness of the booster gasket affects the adjustment of the booster rod and that will cause the brakes to be either hard pedal or locking the brakes.
The gasket is not really there as anything but a dust seal to help protect the master cylinder seals.