I think this is the piece you are missing:
FYI the OEM 6500# hitch is quite a bit beefier than the pintle style hitch, if you plan to do significant towing or use the hitch as a recovery point.
08925-35870 Genuine Lexus Receiver Assembly, Hitch
Genuine 08925-35870 Receiver Assembly, Hitch fits Lexus models. LexusPartsNow offers 0892535870 for $38.47.
www.lexuspartsnow.com