I disagree. The IRS would downscale close to 95%, as a result of obtaining revenue exclusively from retailers.
At the current levels  the IRS has a budget of about $11.4B. Of that about $3.5 is taxpayer services and a bit less than $8Billion is Enforcement.
While some collection issues would die, Enforcement of business taxes would increase dramatically as the amounts of cash flow they process would increase.
Also, once you tag a 25-30% sales tax on everything the underground economy is going to skyrocket which will make the "enforcement" element go up. While many people won't worry about that additional 30% on their cloths or car parts or toys, once we start looking at computers and big screen TVs and appliances and other big ticket items where 30% moves the bottom line appreciably there will be a black market that will blossom across the nation. Part of that Entrepreneurial American Spirit at work.