I have been having similar problem where it pulls from right to left, after driving on highway for some time. If I stop and wait a couple of minutes, it goes away for a while. I think I have heard the occasional small clang on passenger side before it happens i.e. side opposite to where it pulls.Have you checked your steering rack bushings? Mine had the same symptoms when I had a bad passenger side bushing
When it first started I had the steering rack and pinion replaced with a refurbished one, that seemed to help quite a bit, able to travel on highway for quite a distance with no issues. Also, it seems to have started after I previously had the power steering pump replaced.
Doing a bit of research online with the symptoms I've been experiencing - starts with gentle pulls & sways to left until it gets progressively worse - could it be the upper ball joint?
Again - thank you so much Forum members for all your comments & recommendations! I'm so glad I discovered this resource for helping with issues with my beloved 2001 Land Cruiser.