PiratProxies

Glossary

What is rate limiting?

Rate limiting is a cap on how many requests a website accepts from one visitor in a given time. Going over it brings errors or a block.

A site counts requests per address, per account or per session. Past the limit it answers with an error, most often 429 Too Many Requests, or quietly serves a captcha. The limit protects the site from being overloaded.

There are two honest ways to stay under it: slow down, or spread the same work across more addresses with a rotating proxy. Slowing down costs time; rotating costs traffic. For small sites, slowing down is also the considerate choice.

Read next: Proxies for web scraping