I'd say sour cream is good if you're doing the "garbage" thing, with broccoli, cheese, bacon pieces, etc.
Usually though all I want is a little salt, quite a lot of pepper, and medium amount of butter. I want to taste the potato. Mmmm, potatoes...
Actually, I'd say you're right. I started doing the "garbage" thing back in college because the potatoes themselves were dried out and tasted like s***. Give me a decent potato, properly cooked, and I'm with you.
