It's the famous driveline binding. I assume it is a 4WD. SO just lube the slipjoints and it will get better after a few K miles. I would recommend you lube it now and then every 5K service till 20K. AFter that you will need to lube it as required. AT least it will get better with time.
I'm sure you know what to do. If not check out
http://www.toyota120.com/html/driveline_service.html
I had the same issue on my 03 4R adn it is pretty much gone after 60K. I lube it every 10-20k miles and haven't had a clunk for ages. Lexus issued a TSB for the GX with the same issue, actually the GX is worse due to the softer suspension. They replaced the rear control arms and driveshaft. CLunk free since then. Toyota however have refused to do anything about it yet, but the 4R seems to be far less of a problem.