There was a time when I would have agreed with you but that time has passed. I also started using them 30 or so years ago, off road raced with them, etc, but don't run them anymore. It true that they can flow more per square inch than most other filters, but they also allow much more dirt in, especially when freshly cleaned.
Read this:
K&N air filter or OEM, why OEM is better
Some manufactures don't do a good job in designing their air intake systems, so aftermarket systems can make a significantly detectable difference on those rigs. IMHO Toyota does a very good job of air filter design, especially on their off road rigs. The stock filters are not old school wood pulp paper, they are waterproof, synthetic fiber mat, much more efficient. It's slightly more restrictive per square inch than gauze, but Toyota designs the filters with plenty of surface area to provide needed flow.
Toyota's cyclonic filters are especially well designed. Most of my testing/experience is with the 80 series, but the later FJC has a similar cyclonic design. They are very good at rejecting dust and stay clean for a very long time.
I run a filter minder restriction gauge on mine. Cleaned the filter before Cruise Moab last year, put ~12Kmi on it, lots of dusty trails and it never moved from baseline reading. For this years Cruise Moab, I put in a new filter and it showed the same reading. I wanted a "baseline", but now wish I have left the other filter in and done a long life test to see when it produced enough restriction to cause efficiency reduction. From what I have seen, would bet it would go 50K mi in my conditions or much longer in a less dusty environment.
Getting to or exiting many of our trail require long dirt roads across the desert. Some 50+ miles where you can hardly see the rig that your following. In the year that filter was run, there were at least 6 runs were we ran 12+ rig caravan for 50+mi, tons of shorter runs and often I was tail gunning, so taking huge dirt loads.
It's easy to say it works, I have never had a problem, I would expect that and the aftermarket manufactures count on it. Additional abrasive particles in the oil aren't going to cause immediate motor failure, even in the case of huge amount the motor will likely still run. The wear is slow, long term loss of efficiency, reduced engine life. The proof is in testing, locally several of us have done oil analysis on rigs that run about the same trails and it's easy to pick out the rigs with aftermarket air filters, the silicon/insolubles (abrasive particles) are much higher, sometimes alarmingly so.
For me it is a choice; do I want the last .01% horsepower now or protect the motor more for long term efficiency. I intent to keep the rig long term, want to maintain it's efficiency over it's life, so the stock from the dealer filter is my choice.