That has been a bit different than my experiences with my 470. I have a chain-style WDH with sway control, after getting it dialed (which took some trial and error), the GX tows remarkably stable and is not really affected by wind. We have narrow, steep/curvy roads around here, plus higher-speed interstates. We've towed our camper as far as 700 miles in one day (from MI to MO, all interstate/4-lane at 70+ mph), and it's overall a comfortable experience for our family of 4 (outside of foot fatigue from not using cruiser control due to the gearing/power). With the WDH putting tongue weight on the rear and front axles, it actually rides a bit better than stock and feels very planted to the road. Perhaps some of the MPG issues are a 1UR vs. 2UZ thing, but the chassis is similar between both vehicles.
Regardless of engine/platform, towing either without a WDH or a WDH that isn't dialed properly can make for a miserable experience (bad ride and lots of trailer sway).