Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.reader.dev/llms.txt

Use this file to discover all available pages before exploring further.

Reader is the web infrastructure platform for AI. Scrape, crawl, and automate the web from a single API. Get back clean markdown, HTML, and metadata, or spin up cloud browser sessions with full Playwright/Puppeteer control. One API key. Three primitives. You point it at a URL, we handle the rest:
  • Rendering dynamic, JavaScript-heavy sites so you get the page the user sees
  • Bypassing bot walls automatically. Pick standard, stealth, or let auto decide
  • Extracting main content and stripping navigation, ads, and boilerplate
  • Converting to clean markdown optimized for language models
  • Automating with cloud browser sessions for login flows, multi-step navigation, and interactive automation
  • Caching successful scrapes for 24 hours, so repeat requests are free

​
What you can build

RAG pipelines

Scrape docs, blog posts, and knowledge bases into your vector store.

AI agents

Give your agent the ability to read the web as a tool call.

Research tools

Batch scrape hundreds of URLs in parallel and stream results live.

Monitoring

Crawl competitor sites and extract structured data on a schedule.

​
Get started in 60 seconds

1

Sign up free

Create an account at app.reader.dev. No credit card required - you get 1,000 credits every month on the free tier.
2

Grab your API key

Open the API Keys page in the dashboard and click Create API Key. Copy it - you’ll only see it once.
3

Make your first request

Follow the Quickstart to scrape your first page in under a minute.

Open Dashboard →

Sign up free and create your API key in 30 seconds.

​
How Reader works

Three primitives, each with its own endpoint:
EndpointPrimitiveWhat it does
POST /v1/read with single urlReadScrape a page. Returns results synchronously
POST /v1/read with urls arrayBatchScrape many pages. Returns a jobId to poll or stream
POST /v1/read with maxDepth / maxPagesCrawlDiscover links and scrape each page asynchronously
POST /v1/sessionsBrowseSpin up a cloud browser. Connect Playwright/Puppeteer via CDP
See The read primitive for the scrape/batch/crawl mental model, or Browser sessions for interactive automation.

​
Why developers pick Reader

  • Three primitives, one platform. Read, crawl, and browse through one API key, one credit pool. No separate billing tracks or hidden multipliers.
  • Browser automation included. Full Playwright and Puppeteer compatibility via CDP. Connect your existing scripts to cloud browsers with anti-bot stealth active.
  • Handles the modern web. Dynamic pages render correctly, bot walls get bypassed automatically in auto mode, and you never touch the underlying infrastructure.
  • Built for LLMs. Output is clean markdown by default, with optional HTML and metadata. Main content extraction removes boilerplate so your tokens aren’t wasted.
  • Free tier is actually free. 1,000 credits a month, no card required. Enough to build and ship a real prototype.
  • Self-hostable. The underlying engine is open source and Apache 2.0 licensed. Host it yourself if you need to. Learn more →

​
Where to go next

Quickstart

Make your first scrape in 60 seconds.

The read primitive

Understand the mental model behind Reader’s single endpoint.

Guides

Practical recipes for RAG, agents, production patterns, and more.

API Reference

Full API spec for every endpoint.