> but I don't see how this avoids that problem other than being small enough that nobody is optimizing for your search engine.
Google doesn't have any countermeasures against a lot of what we consider spam and generally user-hostile.
Google could at least try to detect spammy recipes by for example detecting certain keywords (likelihood of it being a recipe) and then downranking it based on length, with the idea that if we're confident it's a recipe it should be short and to the point, and anyone telling their life story on it can go to hell.
Google could detect listicles similarly.
Yes, this can all be gamed, and I will expect there to be an arms race, but it will at least raise the bar and make spam content costlier to produce and require constant maintenance as Google's algorithms get better. Yet, right now, Google isn't even trying any of this, and why would they? Spam typically has ads and/or analytics on it, both which can be Google's and thus benefit them. Why would they ever expend extra engineering effort (thus money) to ultimately earn less money if they're successful?
In addition, the ultimate counter-attack to profit-motivated spam would be to just detect & downrank what gets them paid - ads, affiliate links or analytics, as I explained in previous comments such as https://news.ycombinator.com/item?id=32434317. All of those often require things that would make them trivially detectable, whether it's ads that must be a third-party script so the ad network can detect fraud, affiliate links which ultimately must lead to a large affiliate domain such as Amazon and sometimes mandate legal disclosures that can be detected, or analytics that likewise rely on a third-party script.
Google may be a bit limited due to their monopolistic position. Many of the more devastating things they could do would probably skirt pretty close to anti-competitive behavior.
There's Bing, etc. I could see issues where Google is unfairly prioritising their own properties, but if they're merely going after user experience (including downranking ads which hurts their bottom-line), I'm not sure where the antitrust argument is?
They can always offer it as an option that the user must explicitly opt-in, that way nothing is forced onto the users (yet everyone will enable it pretty quickly if the results end up better).
Well if you have a 97%+ market share, I'd to say you're a defacto monopoly.
Might fall within refusal-to-deal (which is fairly nebulous concept), especially given they are also in the advertisement business. Anything that could be interpreted as blocking search results with competing ads to their own might come under scrutiny.
Google doesn't have any countermeasures against a lot of what we consider spam and generally user-hostile.
Google could at least try to detect spammy recipes by for example detecting certain keywords (likelihood of it being a recipe) and then downranking it based on length, with the idea that if we're confident it's a recipe it should be short and to the point, and anyone telling their life story on it can go to hell.
Google could detect listicles similarly.
Yes, this can all be gamed, and I will expect there to be an arms race, but it will at least raise the bar and make spam content costlier to produce and require constant maintenance as Google's algorithms get better. Yet, right now, Google isn't even trying any of this, and why would they? Spam typically has ads and/or analytics on it, both which can be Google's and thus benefit them. Why would they ever expend extra engineering effort (thus money) to ultimately earn less money if they're successful?
In addition, the ultimate counter-attack to profit-motivated spam would be to just detect & downrank what gets them paid - ads, affiliate links or analytics, as I explained in previous comments such as https://news.ycombinator.com/item?id=32434317. All of those often require things that would make them trivially detectable, whether it's ads that must be a third-party script so the ad network can detect fraud, affiliate links which ultimately must lead to a large affiliate domain such as Amazon and sometimes mandate legal disclosures that can be detected, or analytics that likewise rely on a third-party script.