Is it possible that your panhard bar geometry tends to roll the top of the axle forward when you turn one way and backward when you turn the other way. The result being an increase in caster when turning direction A and a decrease in caster when turning direction B.
Ref: "The Caster angle is the ability of the front suspension system to self centre under cornering loads. Too much caster and the front of the car will understeer more (positive caster), too little and you will get oversteer handling characteristics (negative caster)."
Ref: "The Caster angle is the ability of the front suspension system to self centre under cornering loads. Too much caster and the front of the car will understeer more (positive caster), too little and you will get oversteer handling characteristics (negative caster)."