how does it matter? are we supposed to keep downsizing/downgrading from the high school cars just to make up for all these trends that we didn't choose?

A lot of a modern car’s extra weight when compared to older models are mandated safety improvements.

Oh there's no doubt my modern car is objectively better in numerous ways. That said, we could be doing a lot more to optimize for fuel economy but the market doesn't seem to want that.

There are both fuel efficiency optimized cars and not optimized cars. You bought latter.