there is a special spec for google by that https://developers.google.com/search/... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		franze on Dec 23, 2023 \| parent \| context \| favorite \| on: Google is apparently struggling to contain an ongo... there is a special spec for google by that https://developers.google.com/search/docs/appearance/structu... basically cloaking + json-ld markup

codetrotter on Dec 23, 2023 [–]

I wonder if Google trains its AI on paywalled data, that other scrapers don’t have access to but which those paywalled sites give full access for the Google bot to.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact