How to Use Proxies with curl_cffi

Which proxy to use

For most scraping, Residential Lite is the cheapest way to get home IP addresses in the right country. Use Residential when you need a particular state or city, and Unlimited Datacenter for targets in the United States that do not block data centre addresses.

What to enter

A PiratProxies plan gives you four things: a host, a port, a username and a password. These are the values for Residential; the username and password are on your plan page.

Host	`resi.piratproxies.com`
HTTP port	`8080`
SOCKS5 port	`1080`
Username	`USERNAME`
Password	from your plan page

As one address:

http://USERNAME:PASSWORD@resi.piratproxies.com:8080

The country and the session are chosen in the username, not in a separate setting. The generator on your plan page writes the username for you, and Targeting and sessions explains each part.

Steps

Pass the proxy for both http and https.
Choose a browser to impersonate, so the connection looks like that browser as well as coming from a home IP address.

Example

Python

from curl_cffi import requests

proxy = "http://USERNAME:PASSWORD@resi.piratproxies.com:8080"
r = requests.get("https://api.ipify.org", impersonate="chrome", proxies={"http": proxy, "https": proxy})
print(r.text)

Replace USERNAME and PASSWORD with the login from your plan page. The example asks an IP-check address, so the output is the address your request left from.

Rotating or sticky

For crawling many pages with curl_cffi, leave the username without a session so every request leaves from a new IP address. For anything with steps that belong together, such as a login, a cart or a paginated search that sets cookies, add a session to the username so those requests share one address.

A sticky username looks like this:

USERNAME-country-us-time-600-session-a1b2c3d4

If it does not work

A browser-like fingerprint on a data centre IP address is still a data centre IP address. Sites that block by address need a residential or mobile proxy.
A 407 error means the username or password was not accepted. Copy both again from your plan page, and check that nothing added a space or cut the username short.
If the IP address is in the wrong country, look at the username: the country code is part of it. More fixes.

Questions

Does curl_cffi work with SOCKS5 proxies?

Yes. Use port 1080 and the same username and password.

Which PiratProxies proxy should I use with curl_cffi?

curl_cffi is a product of its own makers. PiratProxies is not affiliated with it; the name is used here only to say what this guide is about. Use it within the rules of the sites you work with and our acceptable use policy.