How should I get started with writing embedded firmware period? I am a SWE but only have professional/personal experience with web development (including backend services). As a start, I have an old Chinese mp3 player that would be really neat to be able to hack and make some changes to. How would I start with that, if that's at all a good starting point.
As suggested, one path is to port Rockbox firmware for your MP3 player. However, since you do not have any prior experience in embedded domain, it will not be a easy path.
I would suggest that you try using the Arduino or RaspberryPI hardware. Both of these platforms are widely used, so you would be able to easily get help if you get stuck. Also, these are 'simple' platforms that are widely available, so the learning materials available is unmatched.
Building a simple project like a weather station or clock will keep you focused on a "actual" target/goal.
If your focus is to just revive/upgrade it with more features, rockbox[0] might be a choice for you. It is open source so if that works for your model, then you may be able to get to hacking quicker than doing it all by yourself (though the latter might be a lot more satisfying).