Skip to main content
Proxies are how Reader scrapes sites that would otherwise block or rate-limit a single IP. Reader supports two tiers with very different cost and capability profiles.

The two tiers

Standard (datacenter)

  • Fast (~50-100ms overhead per request)
  • Cheap (pennies per GB)
  • Easily detected by sophisticated anti-bot systems - the IP range is a known datacenter
  • Great for: APIs, blogs, docs, news sites, anything without aggressive bot protection

Premium (residential)

  • Slow (300-800ms overhead per request)
  • Expensive (dollars per GB)
  • Indistinguishable from a real user - the IP is a real home ISP
  • Great for: Amazon, LinkedIn, ticketing sites, anything that aggressively blocks datacenters
You’ll use standard for the vast majority of requests and premium only when necessary.

Configuring both tiers

Both pools can have any number of proxies. Rotation within a tier is round-robin by default (or random via proxyRotation).

Tier selection per request

Two modes:

Explicit standard

Always pulls from the standard (datacenter) pool. Cheapest option - use when you know the target doesn’t need residential.

Explicit premium

Always pulls from the premium (residential) pool. Use when you know the target needs residential - don’t waste resources trying standard first.

Sticky sessions for premium

Premium (residential) proxies are typically billed per-request AND per-IP - cycling IPs on every request is wasteful and also tends to trigger anti-bot systems (real users don’t jump IPs mid-session). Reader handles this with sticky sessions: for premium proxies, Reader generates a unique session ID and passes it to the proxy provider in the URL:
The session-reader_... parameter tells the provider “keep this IP for this session.” All requests in the same crawl session use the same IP, mimicking a real user.

Flat proxy list (legacy)

If you only have one tier of proxies, use the flat proxies option instead of proxyPools:
No tier selection - every request rotates through the flat pool. Simpler but less flexible than multi-tier pools.

Per-crawl stickiness

During a crawl() session, Reader picks one proxy at the start and uses it for every request in that crawl. Rotating mid-crawl would trigger anti-bot systems on sites that track session continuity. If you want different crawls to use different proxies, just call crawl() multiple times - each invocation picks a fresh proxy from the pool.

Where to go next

Proxy Configuration guide

Practical setup for single proxies, pools, and tier selection.

Scraping Engine

How the Playwright engine and proxy tiers work together.