You might want to do a similar cross-reference to a 4Runner hitch. I ordered a new Toyota hitch for an '07 4Runner for my Lexus and it was a 100% perfect fit. Since the GX and the 4Runner share a frame, they might also share the same 03/04 and 04+ hitch difference. If not, you might be able to find a used one.
I had a Curt hitch on my 470 and really didn't like it. It was pretty ugly (hung down really low), all of the powdercoat flaked off, and it had limited clearance near the exhaust. It's only rated to 5,000+ relative to the 6,500# OEM hitch. It's probably better than a pintle hitch but is a few notches below the OEM 6,500# hitch.
Edit: This MIGHT be the right OEM Class IV hitch - it appears to be restricted to 2003-2004 4Runners and a couple of 2005 4Runners (PT228-89440). It has a different part number than the 2005-2007 as well (which are PT228-89460), but please verify on your own. Might be worth calling your local Toyota dealer and seeing if they can order one for you to look at. I really don't think you'll like the Curt hitch that much based on my experience, unless you don't have any other options.
Genuine Toyota Accessory # PT22889440 (PT228-89440) - Towing Options, Tow Hitch. CLASS, IV, Exterior. Fits 4Runner
parts.toyota.com