Sometimes it can be.

Sometimes there are tradeoffs.

If it was as simple as "don't do the thing you hate", nobody would ever get cold in the first place, regardless of it makes them sick.

So it matters whether the sickness effect is negligible or not.

Nope. I guarantee people would like skiing more if you were possible at ~70 degrees. Snowmobiles can be fun, but they'd be more fun without the twenty extra layers of clothing you need to put on. Warming yourself by a fire can be nice, but it's the warming that's enjoyable. Being cold sucks.

Again, being cold leads to hypothermia. Just wear a coat, it's not that big of a deal.