Proxies for web scraping
Collecting public pages at scale from one IP address gets that address rate-limited or blocked. Spreading requests over many addresses, in the right countries, keeps a crawl running.
Pick the pool by how strict the site is
Start with the cheapest pool the target accepts. Open data and APIs usually work over datacenter IPs. Sites with bot filtering usually need residential addresses. The few that challenge even those tend to accept mobile carrier addresses.
Rotate for listings, hold for flows
For independent pages, such as search results or product listings, leave the session out of the username and every request gets a new IP. For anything that spans several requests, such as paging through results or a cart, add a session ID so the IP stays the same.
Mind the bill
Traffic is what you pay for. Block images, fonts and media in your scraper, request compressed responses, and cache what does not change.
Stay inside the lines
Collect public data only, respect the law where you and the site are, and do not overload small sites. Our acceptable use policy applies to every plan.