You are right; also I would imagine the tax authority does not even care too much about where the money came from, they just want to make sure you paid enough taxes.
Criminal investigation agencies of course do care.
The IRS has a criminal enforcement department, and they absolutely investigate criminals and press charges that have nothing to do with tax evasion. Often they work with other branches, but they are fully fledged FBI agents and can prosecute any crime they want.
Usually they do that when they stumble upon them during tax/laundering investigations, but they'll prosecute anything.
