The Swede behind search.marginalia.nu has had a working search engine running at a single desktop class computer in a living room, all programmed and maintained on his spare time, that was so good that in its niches (history, programming, open source comes to mind) it would often outshine Google.
Back before I found Kagi I used to use it everytime Google failed me.
So, yes, given he is the only one I know who manages this it isn't trivial.
But it clearly isn't impossible or that expensive either to run an index of the most useful and interesting parts of internet.
I think the problem with search is that while it's relatively doable to build something that is competitive in one or a few niches, Google's real sticking power is how broad their offering is.
Google search has seamless integration with maps, with commercial directories, with translation, with their browser, with youtube, etc.
Even though there's more than a few queries they leave something to desire, the breadth of queries they can answer is very difficult to approach.
This is one of those things that I think is interesting about how "normal" people use the Internet. I am guessing they just always start with google.
But for me, if I want to look up local restaurants, I go straight to Maps/Yelp/FourSquare(RIP). If I want to look up releases of a band, I go straight to musicbrainz. Info about Metal Band, straight to the Encyclopeadia Metallum. History/Facts, straight to Wikipedia. Recipes, straight to yummly. And so on. I rarely start my search with a general search engine.
And now with GPT, I doubt I even perform a single search on a general search engine (google, bind, DDG) even once a day.
"Normal" people don't start with Google, not any more. They start with Facebook, Instagram, X, Reddit, Discord, Substack etc. That's exactly the problem, the world-wide web has devolved back into a collection of walled gardens like things were in the BBS era, except now the boards are all run by a handful of Silicon Valley billionaires instead of random nerds in your hometown.
You have just described how "innovation" is often just a power shift, often one that does not benefit the user. Old is new again, but in different hands; the right hands, of course.
Back before I found Kagi I used to use it everytime Google failed me.
So, yes, given he is the only one I know who manages this it isn't trivial.
But it clearly isn't impossible or that expensive either to run an index of the most useful and interesting parts of internet.