For river use, look for something in the 12 to 14 foot length. Any shorter than 12' and the bow will 'bob' side to side with each stroke (a reaction to the force of your torso). My shortest kayak is 10' and I hate it. The bow movement is very inefficient. I fell like I have to work at near maximum capacity to maintain what would be a cruising speed in my 14 footer. Any longer than 14' and you'll have some difficulty with tighter river turns. 16' is about as long as you would want to go for river use. Even then you'll have some frustrations. I love my 14 footer for rivers.
A rudder is nice but only if you expect any type of a cross wind. My 14' has double duty as river and lake kayak...I only use the rudder on the lake and rarely use it on the rivers. It can actually be a hindrance in the shallows and debris of a river. In those cases, I just keep the rudder 'raised' and out of the water. If you get a rudder, you can always not use it. I will admit, I love the rudder for the lake.
I second the 'buy used' recommendation. Mine was a demo for a local kayak shop. Other than the word 'demo' melted into the bow, I never would have known. I actually bought it used from the guy who bought it used from the kayak shop.
I would highly recommend a sit-in type of cockpit. I have friends who have sit-on-top and they always prefer my second sit-in when we go out. If you plan on doing anything more than a few easy miles or if you plan on actually getting your heart rate up or covering some distance...I would recommend going sit-in. Plus, getting a spray skirt really helps keep in the heat for the cold weather paddling.
As far as kayak materials, mine are all plastic (Wilderness Systems 'duralite'). Some call them 'tupperware.' They are definitely heavier than a fiberglass kayak but I can bounce off about anything, scrape bottom, drag it over fallen trees, I don't have to worry about scratching the paint off and I don't have to worry about cracking anything and springing a leak. I do have to be careful how I store it as I do not want to warp the plastic (specially in the heat of the summer) or get any sun fading (red turns to pink...). If you get into racing, the tupperware kayaks are too slow to be competitive. But, you can get pretty serious within the 'recreational realm' with some of the better plastic kayaks.
Getting a good paddle is key. A bigger blade is not necessarily better. Big blades are for sprinting (think white water). Thinner blades are better for distance traveling. In fact, the Inuit eskimos were known from paddling hundreds of miles in open ocean with a blade not much bigger than the palm of your hand. As for blade length, stand in your bare feet and raise one hand as high as you can (stay flat footed). Measure the length from the floor to the tip of your tallest finger for your theoretically ideal paddle length. From there, it is personal preference as to whether or not you want to go slightly shorter or longer. It is nice to get a paddle that can be 'feathered' as it will make a word of difference in moderate to high winds.
Another not often considered piece is a well fitting PFD (life vest; personal flotation device). Do not use a skiing life vest (you'll overheat) or a canoing-type life vest (not enough freedom of movement). Kayak vests allow better shoulder and torso movement. They also 'ride high' meaning they end about at the bottom of your rib cage. This lets you sit in the kayak's seat without the seat back pushing the PFD up over your chin. It is nice to have a few front pockets for food and water bottles. Mine didn't come that way (I didn't think of it) and I ended up sewing on some webbing on the front and back so that I could use the military's MOLLE pouches (I put a MOLLE Camelbak bladder on the back webbing and a mesh-bottomed pouch on front for food).
If you plan on doing any overnight paddles, look for bow and stern storage with water proof 'lids'.
Sorry this is so long. A little bit of info goes a long way. I have seen others just jump into a kayak and hate it because they didn't pick what fit them. Don't be afraid to ask the kayak shop to take it out on the water for a demo. You have to feel it in the water to know where the stability is and how your frame fits in the cockpit. Someone from the kayak shop will likely watch you in the water to make sure you don't paddle off with their boat...so you could even ask them how your form looks and try to coax some pointers.
I would consider myself a recreational paddler leaning toward the serious side. My usual route is about 5 miles with the occasional 10 to 15 mile Saturday paddle. I try to stay on the water until it freezes and get back in once it thaws for the Spring. My max is about 28 miles straight (took forever and one of the hardest parts was reminding my legs to walk when I was finished). I have never overnighted.
Hope this helps. Good luck!
