I would also think if you went too big the engine could not keep enough pressure in the exhaust to keep water out when you do a water crossing. I've had my exhaust submerged a few times, so thats something to consider.
I would think 2.5' for a B or HJ would be ok. Whats stock? 2 inch or so? So it's a little better going 2.5.
4 inch and bigger on the domestic diesels is just showing off