It's that the rich have already paid the experts for the management and preparation of their taxes, not the threat of hiring a someone to defend themselves.

It's not that it's too expensive, it's that rich people's taxes are done by professionals and questions of tax liability are going to be complex and not necessarily in the IRS's favor. On the other hand, someone that runs their own business, nets in the $10,000s to pushing into 6 figures, and does their own taxes or provides potentially erroneous information to a tax preparer is a fairly easy target. The discrepancies that the IRS's computer systems find are likely going to be in the favor of the IRS.

Because they're losing literally billions of dollars.