If you can't handle the 'adjustments' on a $50 12" bike intended to be operated by a 4-year old...
As much as it's contrary to my nature, I'm with swank on this one. How long does it really NEED to last? A couple summers at most before they're growing into something bigger? It's not like resale value is a consideration here. I say go for the cheapo bike and let them thrash it. If it's still working when it's time to upgrade, pass it on to a kid that doesn't have one.
I'm all for buying quality when you need something to last. I only buy myself quality shoes now that will last and remain comfortable. I insist on the same for my wife, despite her spendthrift inclinations. My kids get decent shoes. I won't foce them to wear shoes that hurt, but I'm also not buying them $300 hiking boots that will be outgrown in a year purely for the self-righteous pleasure of "buying nothing but the best for my child".
I don't buy crap...ever. As far as bike adjustments go, crap bikes will never hold adjustments correctly. Setting up a bike is easy, but keeping it there only works with quality goods.
As far as adjustment goes, you'd be surprised how many people can't figure out pedal height, handlebar angle, stem height, and the rest. I'm used to seeing kids ride around with forks installed backwards because their dad is a systems analyst and never got outside to ride bikes because Space Invaders was more important. As such, the bike never gets set up right, much less adjusted for fit. I used to race bikes, so I consider proper adjustment an artform which is lost on most.
Having 3 boys allows me to hold on to the bikes for a long time, and I'll give them to someone else when they have totally grown out of them. My 11 year old got his 16" Specialized when he was 6 and he still likes to moto around on it since his 9 and 6 year old brothers moto their bikes around too. So I have to say that your growing out of bikes idea is not completely applicable in this situation.
The 11 year old rides his 24" Cruiser, my 26" Redline Cruiser with the seat all the way down, and his 16" Specialized with the seat all the way up. He fits best on a 20" bike, but his choice to get a Cruiser allows him to grow into a larger frame, although he fits better on the 20". Having given him the lecture about how the 20" is the right fit, and allowing him to make his own choice, I was still able to find a 24" that fits him pretty well. He tears it up on that thing, but he has to be more careful when pulling jumps since the mass of the 24" is a lot greater.
Earlier this afternoon, I cleared the backyard for a BMX track that I'm doing the grade work on tomorrow. I am pulling in a bunch of fill with my dump, and building some sweet high berms, banks, and jumps, for the maniacs to tear up, instead of driving Mom nuts by asking her permission to ride in the street.
Since I own a CAT 247B, a 14' dump trailer, and having an unlimited supply of clean fill, I am going to put together a sweet little track. I may even get my Redline MonoCog cruiser out for some fat berm action of my own.

My wife hates the idea of a track in the backyard, but she will appreciate having them out of her hair when it's done....
Besides, show me a kid that likes grass more than a BMX track in the backyard, and I'll show you a kid who like Barbie's and listens to showtunes...
By the way, watching three boys riding in a row and spinning circles on 16" bikes, is like watching the Shriner's go crazy in my own back yard....