Hacker News new | past | comments | ask | show | jobs | submit login

Main purpose is to render the page, crawl the dom for the data, and then load the next page as fast as possible... so it's equivalent of having a thousand bookmarks and opening them all at the same time, and as soon each loads running the scraper script and reloading them with the new link. Try it for yourself, it's easy to script, and check the load...



Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: