Your post shows the big difference in mentality that's partly responsible for why we see so many broken sites --- I haven't ever used those tags before and would just go for plain old DOM style manipulation with JS (which has been around for a very long time), whereas you started with something much newer and took backwards-compatibility as an afterthought to be added on. Your solution requires less initial effort (providing you knew about the new tags) but then additional effort to work on older browsers --- that might not even be done --- whereas my solution may take a little more effort initially, but then naturally doesn't need any further considerations for backwards-compatibility.
My mentality is to make the website usable without JS for most people (with the reasonable effort of installing FF or Chrome at most), then use as simple and compatible JS as possible if needed. Thus, old-style DOM fiddling isn't the first choice - and if we're talking very old browsers, it was also tricky to get right due to all the incompatibilities.
Why you think something would be broken when a polyfill is used as fallback, I don't know.
Why would anyone use an older browser? (Other than IE in old firms - but this is probably becoming rarer and rarer) I guess everyone should upgrade their browser regularly, at least to get security upgrades
