How did you get that circular pipe out so easily? I tried to take mine out earlier today and it did NOT want to budge. Literally had 2 pairs on pliers on it, standing INSIDE my engine bay, pulling hard as hell and got nothing. I am really considering trying to dissolve the o-rings but pouring some acetone on them andletting it eat them away. I have a new pipe and o-rings ready to go.
Since you have a New Bypass Pipe you are not concerned with marring the old one BUT you must still be careful not to damage the T-Stat housing.
My cooling system has always been kept clean...so I was able to just wiggle mine back and forth while pulling up until it came out. However, some folks have experienced 'corrosion' between the pipe and the housing. IF that is the case...it isn't the O-Ring that is holding it so tightly in there, but rather a bonding of the parts due to corrosive build up (not unlike a chemical weld).
Depending on what these deposits are made of you might be able to find something (CLR/other) that would help dissolve it enough to let you remove the pipe. But in any case it's going to require some mechanical force (rocking back and forth) to get it to come out. By now....I suspect your old one is a bit worse for the wear and it's not an option to leave it in there (provided it wasn't leaking before)? If all else fails you might be able to drill a hole through it (laterally) insert a screwdriver, then use something to pry up (from underneath it). Just be CAREFUL not to apply too much force. Walk away from it...for awhile if you need to.