PiratProxies

Glossary

What are concurrent connections on a proxy?

Concurrent connections are the requests a program has open through a proxy at the same moment. More of them finish a job sooner.

A scraper that waits for each page before asking for the next is slow. One that keeps fifty requests open at once is fifty times faster, as long as the target and the proxy both keep up. Some proxy services cap this number, sometimes calling it threads.

With a rotating proxy, each of those connections gets its own address. If the parts of the job must stay separate visitors for longer, give each worker its own session ID. Raise the number slowly: too many at once against one site is what triggers rate limiting.

Read next: Code examples, including one session per worker