In many cases outliers are extremely important. One that comes to mind is high spenders in mobile games.

Trust me, if analysis was as simple as getting rid of outliers, treating everything as Gaussian, and retrieving simple summary statistics, then good data scientists wouldn't be paid $150k+ :)