IMO, if you are thinking "money" as the primary motivation with this (and trying to make a rational cost-benefit analysis decision about whether to buy a SC or do an engine swap), then you are probably NOT the customer for the SC or the engine swap.
I'd recommend a used Corolla as a better option.
Well for gravel roads .. I completely agree - the Corolla is tough to beat!! Although an early 1990's Civic with full wishbones is also very tough and has all the durability needed for its weight on third world roads. For the highway ... find a low mileage LS 400. I am not sure anything you do to a series 80 will beat that. And for real offroad the stock 1FZE is sufficient - although I dont get up into the "thin air" altitudes some of you may.