Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There are plenty sites that do that.

And if you're determined to scrape [a website], sometimes it needs proxies, rotating user agents and some rate limiting.

Using a full blown browser sometimes helps prevent you hitting those rate limits, but they're still there.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: