I literally write a briefing paper advising my university to clamp down on the nascent web and continue with gopher which was much more systematic and organized.
You're not wrong - the web is very disorganized. This makes it more valuable to shareholders, as they can design sites to present us with whatever extracts optimal value at the present moment. That's how Google gets to interpose itself between us and what we're looking for, and redirect us to SEO spam pages.