> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reader.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Cloudflare and bot walls

> When Reader gets through automatically, when to use premium, and when a site is out of reach.

The #1 reason scrapes come back with thin or wrong content is the target site sitting behind a bot-detection layer: Cloudflare, Akamai, PerimeterX, Datadome, or similar. The site serves real content to browsers and a "checking your browser" page to everything else. Reader handles most of these cases with `premium` mode.

## How Reader handles it

`standard` mode uses datacenter proxies, which many anti-bot systems detect and block. `premium` mode uses residential proxy IPs that are indistinguishable from real users, bypassing the common bot detection vendors.

For sites that are known to be hostile (Amazon, LinkedIn, booking sites, ticket resellers), use `premium` mode directly:

```ts theme={null}
await reader.read({
  url: "https://www.amazon.com/dp/B08N5WRWNW",
  proxyMode: "premium",
});
```

## Detecting a block

Some blocks are subtle: the HTTP status code is 200 but the content isn't what you'd see in a browser. Symptoms:

* `metadata.statusCode` is 200 but the markdown is very short
* The markdown contains phrases like "please enable JavaScript", "checking your browser", or "access denied"
* The same URL in a browser shows totally different content

Reader thinks the scrape succeeded, but what you got isn't the real page. Switch to `premium` when you see this.

## Force premium explicitly

If standard mode is returning thin or challenge content, switch to premium:

```ts theme={null}
await reader.read({
  url: "https://hostile-site.example.com/page",
  proxyMode: "premium",
});
```

This uses residential proxies and goes straight to the bypass strategy. 3 credits per page.

## When premium doesn't work either

If premium-mode scrapes also come back thin, the site is beyond Reader's reach. Common offenders:

* Sites with CAPTCHAs that require human interaction
* Sites using very new or custom bot detection that our premium mode hasn't learned
* Sites that require a logged-in session

For these, your options are:

1. **Use the site's official API** if they have one
2. **Scrape with your own session** (cookies, auth, manual CAPTCHA solving) and only use Reader for the public parts
3. **File an issue.** Sometimes we can tune premium for a specific site

## The `waitForSelector` trick

If a page loads a shell and then hydrates content client-side, even premium won't give you the real markdown unless the page has finished hydrating. Combine premium with `waitForSelector`:

```ts theme={null}
await reader.read({
  url: "https://shop.example.com/item/42",
  proxyMode: "premium",
  waitForSelector: ".product-price",
});
```

Reader waits for `.product-price` to appear before capturing, ensuring you see the fully-rendered post-hydration DOM.

## Detecting thin results in code

Set a minimum content length and flag anything below it:

```ts theme={null}
const result = await reader.read({ url });
if (result.kind === "scrape") {
  const markdown = result.data.markdown ?? "";
  if (markdown.length < 500) {
    console.warn("Suspiciously short result, possible block:", url);
    // Retry with premium, or alert
  }
}
```

The exact threshold depends on the type of content you're scraping. A product detail page under 500 chars is almost certainly a block; a news headline page might legitimately be shorter.

## Next

* [Auth walls](/home/guides/advanced/auth-walls)
* [Choosing a proxy mode](/home/guides/advanced/choosing-a-proxy-mode)
* [Proxy modes (concept)](/home/concepts/proxy-modes)