Matt,
FWIW caster is for steering angles and castor is for oil

. I just did a bunch of learning again...which means I just spent a bunch of money

. This time it was on all things caster related.
New front upper control arms allow me to adjust for as much caster and camber as I want/need. Previously, with my ~2.0"-2.5" lifted front end (t-bars

), Mike at Sunshine* could only get me close to factory caster #'s. Toyota recommends 2.67 degrees of caster as a target for the 100-Series.
Ok...so with the OEM front UCA he got me to around 2 degrees caster with the raised front end. Steering was "twitchy or nervous" on the highway. Didn't like it but couldn't do anything about it.
Mod #1: When Carl (Carl Montoya JT Differentials) intro'd his custom front UCA for the 100's he had a little caster correction built into them. Mike was able to get me another .5-.75 degrees of caster; 2.69 degrees to be exact. The steering definitely felt better...less twitchy/nervous at speed but still not ideal/perfect IMO.
Mod #2 (current as of a few weeks ago): New custom one-off front UCA with complete adjustability. I had Mike adjust caster to 3.79 degrees (3.23 driver's) as I wanted to test the difference with more caster. Results: Night and day difference. Now the front end is rock steady at highways speeds. Steering effort has risen slightly (to be expected even with power steering) but not objectionable.
So based upon what I've learned: #1 caster is highly subjective to a given driver and the individual rig set-up. But stock non-lifted caster target is insufficient with lifted rigs...at least based upon my findings.
So I would suggest you try getting another degree or two of caster and give it a try!
*Mike at Sunshine Automotive (directly east and across Rock Blvd. from Baldinis) is flat out the best alignment guy, especially for anything lifted/modified suspension, I've ever met: Period. But he's just taken a position with the NHP and thus no longer available for us. Sucks (for us...

for Mike).

We do know someone at NHP...maybe he can help
But he is, apparently, going to work some if not all Saturdays at Sunshine to take care of us!