Hacker News new | past | comments | ask | show | jobs | submit login

If I'm not mistaken, it's still an option out basis. Google crawl, index and search everything by default and allow content owners some controls, either via site manager or via flags in your html.



Yeah but I didn't think they'd show news there unless you actually agreed to the news terms.

https://support.google.com/news/publisher-center/answer/7526...


Haven't they heard of robots.txt ?




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: