I want to chime in: I just started driving my fj40 for 1-week now. Just to be quick, I installed a new pilot bearing. If you recall, I was having the same exact noise as you are/were experiencing. The "new" bearing that I got is not 100% sealed. So, I dunked it gear oil for a day before installing so it could soak into the bearing grooves as best as it could. During my installation process, the bearing went in crooked, twice, and I had to remove it. The first time I took it out using wet news-paper packed into the hole the second time I used the bread-method. For reference, the bread method is the best. Each time after removing, I soaked it again in gear oil hoping that the next time I would install it correctly......
Once I started to drive the fj40, it was really quiet for a few miles. Then it started making a noise worse than before. Very loud just on idle. I imagined I somehow damaged the pilot bearing after removing it twice. After about 3 days of the "squealing" and deciding I would learn to live with noise for 40+ years, the noise just stopped. No more pilot bearing noise

I don't know if dunking it in gear oil helped at all or if was just being lucky. But the noise is gone. it's as quiet as it use to run before I starting this 4-speed conversion. So, there is hope on your end. Also, I looked over the pilot bearing that I removed that was causing the noise and I did not note anything of concern. It looks great and feels smooth. I really don't know what caused it to make the noise.
If you do decide to replace it, I hope you cut the cross member and moved it back some. I haven't done that to mine yet, so obviously I had to tilt the engine. Also requires loosening radiator, exhaust, motor mounts, etc. Kind of a pain in the ass and time consuming.
I am now in the process of doing the Transmission tunnel and skid plate like you. Still moving slow, but it'll get there. And unfortunately, At this time, i am unable to do the 4-on-the-tree conversion. I have everything, but the top plate that I have from an h42 will not fit on my early h41 transmission. Things on it are a bit different. I think I know how to make it work but That wont be for a very long time on my end.