PiratProxies

Web scraping: proxy guides

Every scraping library takes a proxy a little differently, and most of the trouble is in the details: where the password goes, whether SOCKS5 works, how to keep or change the IP. Each guide has an example you can copy and run.

Proxy types these guides use

Rotating or sticky?

When your tool is fetching lots of unrelated pages, leave the session out of the username and each request gets a fresh IP address. When requests belong together, such as a login, a basket or paginated results that set cookies, put a session in the username so they all share one address.

Words used in this guide: web scraping, rotating proxy, sticky session, rate limiting, headless browser.

All categories