I don't think I explained what I was drawing up very well. I'm not planning on doing anything internally to the 24 volt starter. The easiest route is probably going to be to use two batteries (as most diesel pickups do anyway) and run them in series to the starter with a series/parallel switch so it has it's own 24 volt circuit when starting while the rest of the Cherokee keeps its 12 volt system once the starter is disengaged. This may require a larger alternator, but I doubt it since most NPRs utilize two batteries, anyway. I'm not positive, but I imagine the stock alternator is around 130 amps. Delco Remy made a parallel switch for GM (I assume for CUCVs), and there are probably many others out there for marine applications. A lot of older marine diesels used 24 volt starting systems while the rest of the boat was 12 volt. The Delco part is D2-6032N.
Another option for the 24 volt starter will be to use a transformer. Anything over 1000 watts can get expensive (and 1000w isn't nearly large enough for the starter), so the ideal solution in this case would be to use a 24 volt alternator and starter, and then run the rest of the system through a transformer (24v to 12v). I imagine this is a pretty common topic in forums dedicated to CUCVs.
Honestly, I think I'm going to re-work the front housing so I can clock the original starter down as far as possible and go from there. A lot of different solutions will work, but this one will require the fewest changes from the stock wiring. If anyone wants to play with some parts without buying a whole starter the front housing from a Wilson starter is 54-29-2048. Of course, it may be just as cheap to buy a used starter off ebay.