If I have to wear that sweater inside my own home, where I get to control the thermostat, I would say that this is a more miserable existence. I'm not sure what there is that's not understandable about that.
It's better to change clothes when it's -10..-25 C (14..-10F) outside. You can't stay indoors in heavy clothes and heavy shoes (and snow and sand and salt will melt indoors).
I'm sure people here are having so much disagreement because they live in very different climates.
I stayed some days in Italy in winter, and their indoor heating was just minimal, it was like +17 C (62 F). I did find it miserable. You can't take a shower quietly, because it's cold in the bathroom too, and you must dress up very very quickly. How do people in such places go to the toilet when they wake up at night? Do they sleep dressed? Or dress up, or just endure the cold for couple of minutes?
I'll find it ok to put on a sweater for the most cold week or two in a year, but not for months.
Not necessarily. Underwear I'll grant, it'd be uncomfortable not to, but otherwise I don't see why one has to wear clothes.
Also, do blankets count as clothes? You (or I, at least :-)) can easily be comfortable "wearing" just a blanket and underwear at a temp of around 10
°C.
Have you considered that maybe your sweater could be comfortable?