My dad has a 2003 and I have a 2007, and I have to say that during normal driving there isn't much of a different between non vvti and vvti. I've driven both in the mountains and there is definitely more power going uphills with vvti, and towing with vvti is much better. That's just my experience.
Good point about driving in the mountains - I live at sea level and only drove the vvti local. Didn't notice much, if any, difference. When I was in CO at 10K ft above sea level, my 4.7 felt like an 80 series at sea level