I'm not sure how to interpret your post: do you have a yakima gutter mount tower system with yakima crossbars? Or do you have a yakima system attached to the factory Toyota roof rack?
If you are piggybacking on the Toyota roofrack, you will need to be careful of weight because the Toyota rack will be putting downward pressure on the toyota roof rack 'feet', essentially the roof itself, and the factory racks are less than ideally designed.
If you have a Yakima rack independent of the Toyota rack, you are putting the pressure on the rain gutters. IMO, this can take a lot more than 100 lbs, although that is all Yak will rate it for for liability reasons. It takes a lot of effort to tear off a rain gutter via a yakima rack clamp. If you regularly use it for more weight, simply add crossbars. I've set up rooflines with as many as 6 yakima crossbars to build decking systems or basket designs to carry excessive weight. Keep in mind the terrain makes a huge difference - rolling down the highway with a heavy roof load is much less abusive than driving a washboard dirt road, and driving on trails with weight up top is another issue all together.
But if you are relying on the factory Toyota rack, in any way, be careful with overloading it.
m.