Questions tagged [scraper]
Post a PromptPrompts that build scrapers and data extractors.
1 prompts
28votes
0answers
4.2kviews
One prompt for a polite, resumable web scraper that won't get me IP-banned
I needed to pull a few thousand product pages for a price-history side project. My first scraper hammered the site with 50 concurrent requests, got a 429, then kept retrying instantly and made it wors