Chances are, this is bad. We saw this on two 1HDTs and one had internal cracks in the block that were not fixable, the other an crack in the head. Another occurrence on a 1HZ, where only the head gasket was bad.
The oil cooler is a possibility, but not likely. The turbo is definitely a possibility.
I no case it will be cheap and quick to fix.
The do-it-yourself tests designed to find exhaust gas in the coolant work really poorly on a diesel, as the fluorocarbons that are put in are fewer than in a gasser.
As Steve said, have a professional test of both, oil and coolant done (like Blackstone), that will tell you both ways (oil--->coolant and coolant--->oil).