I was indeed the pinion angle. As soon as I dropped it off I told them about the growl. He looked underneath and immediately said it was the pinion angle. It rides really smooth now.
I had all the parts necessary to fix, I just installed incorrectly. I have a rear DC driveshaft and extended lower rear control arms from Land Tank, and Dobinsons' adjustable upper control arms. The DC driveshaft and lower arms were on correctly, but the upper arms needed to be extended more.
My upcoming next build will only be a 2.5" lift, but have a similar undercarriage as this one (in case I find a sweet deal for a higher lift before I do the 2.5"). I don't want to chase this problem again