@vakra-dev/reader-js is the official TypeScript/JavaScript SDK for the Reader API. It wraps the HTTP contract, parses the response envelope into a discriminated result type, polls async jobs to completion, throws typed errors, and retries transient failures with exponential backoff.
Installation
npm install @vakra-dev/reader-js
Current version: 0.2.0. Works in Node 18+, Deno, Bun, Cloudflare Workers, and any modern browser.
Quick start
import { ReaderClient } from "@vakra-dev/reader-js";
const client = new ReaderClient({ apiKey: process.env.READER_KEY! });
const result = await client.read({ url: "https://example.com" });
if (result.kind === "scrape") {
console.log(result.data.markdown);
console.log("scraped in", result.data.metadata.duration, "ms");
}
Do not ship your API key in browser code. The SDK works in the browser, but exposing apiKey client-side means anyone can read and reuse it. Proxy requests through your own backend that holds the key server-side.
Configuration
const client = new ReaderClient({
apiKey: "rdr_your_key", // required
baseUrl: "https://api.reader.dev", // optional, override for self-hosted
timeout: 60_000, // per-request timeout in ms (default 60s)
maxRetries: 2, // retries on transient failures (default 2)
});
Scraping
Single URL, synchronous
Single-URL requests return immediately with { kind: "scrape", data: ScrapeResult }.
const result = await client.read({
url: "https://example.com",
formats: ["markdown"],
onlyMainContent: true,
});
if (result.kind === "scrape") {
console.log(result.data.url); // canonical URL after redirects
console.log(result.data.markdown); // clean markdown
console.log(result.data.metadata.title); // page title
console.log(result.data.metadata.statusCode); // 200
console.log(result.data.metadata.duration); // ms
console.log(result.data.metadata.cached); // true if served from cache
console.log(result.data.metadata.proxyMode); // "standard" | "stealth"
console.log(result.data.metadata.proxyEscalated); // true only if auto escalated
}
Multiple URLs (batch)
Passing urls creates an async job. The SDK auto-polls until the job terminates and returns { kind: "job", data: Job } with all results collected across pagination.
const result = await client.read({
urls: [
"https://example.com/page-1",
"https://example.com/page-2",
"https://example.com/page-3",
],
});
if (result.kind === "job") {
console.log(`completed ${result.data.completed} / ${result.data.total}`);
for (const page of result.data.results) {
if (page.error) {
console.error(`${page.url}: ${page.error}`);
} else {
console.log(page.url, page.markdown?.length, "chars");
}
}
}
Crawl
Same shape as batch, but with maxDepth or maxPages:
const result = await client.read({
url: "https://docs.example.com",
maxDepth: 3,
maxPages: 100,
});
Proxy mode
Control how aggressively Reader bypasses bot walls with proxyMode:
// Default: auto. Starts standard, escalates to stealth on block
await client.read({ url });
// Explicit: force the cheaper tier (error if blocked)
await client.read({ url, proxyMode: "standard" });
// Explicit: force the bypass tier (3x credits but works on hostile sites)
await client.read({ url, proxyMode: "stealth" });
See Proxy modes for the full picture.
Job management
The SDK’s read() method auto-polls batches and crawls, so most callers never need to touch job APIs directly. When you do, these methods are available:
// Fetch a single page of a job's results
const { job, hasMore, next } = await client.getJob(jobId, { skip: 0, limit: 20 });
// Collect every page automatically
const allPages = await client.getAllJobResults(jobId);
// Poll a job by ID until it terminates (collects all results on completion)
const job = await client.waitForJob(jobId, {
pollInterval: 2000,
timeout: 300_000,
});
// Cancel a queued or processing job
await client.cancelJob(jobId);
// Retry failed URLs in a completed job
const retryInfo = await client.retryJob(jobId);
console.log(`retrying ${retryInfo.retrying} failed URLs`);
Streaming
For real-time progress updates on a job, use client.stream(jobId), an async generator that yields events as the job makes progress.
for await (const event of client.stream(jobId)) {
switch (event.type) {
case "progress":
console.log(`${event.completed} / ${event.total}`);
break;
case "page":
console.log("page done:", event.data.url);
break;
case "error":
console.error("page failed:", event.url, event.error);
break;
case "done":
console.log("job finished:", event.status);
return;
}
}
The generator closes automatically when the job terminates.
Credits
const credits = await client.getCredits();
console.log(`${credits.balance} / ${credits.limit}, tier: ${credits.tier}`);
console.log(`resets at: ${credits.resetAt}`);
if (credits.balance < 100) {
// Warn, pause workers, upgrade tier, etc.
}
Error handling
Every error response from the API is parsed into a specific ReaderApiError subclass. Branch on instanceof rather than HTTP status codes.
import {
ReaderApiError,
InvalidRequestError,
UnauthenticatedError,
InsufficientCreditsError,
UrlBlockedError,
NotFoundError,
ConflictError,
RateLimitedError,
ConcurrencyLimitedError,
InternalServerError,
UpstreamUnavailableError,
ScrapeTimeoutError,
} from "@vakra-dev/reader-js";
try {
const result = await client.read({ url });
// ...
} catch (err) {
if (err instanceof InsufficientCreditsError) {
console.error(`Need ${err.required} credits, have ${err.available}`);
console.error(`Resets at ${err.resetAt}`);
} else if (err instanceof RateLimitedError) {
console.error(`Rate limited. Retry after ${err.retryAfterSeconds}s`);
} else if (err instanceof UrlBlockedError) {
console.error(`URL blocked: ${err.reason}`);
} else if (err instanceof ScrapeTimeoutError) {
console.error(`Scrape exceeded ${err.timeoutMs}ms`);
} else if (err instanceof ReaderApiError) {
// Catch-all for known API errors
console.error(`[${err.code}] ${err.message} (see ${err.docsUrl})`);
console.error(`Request ID: ${err.requestId}`);
} else {
throw err;
}
}
Every error has:
code: one of 11 stable codes (e.g. "insufficient_credits", "rate_limited")
message: human-readable description
httpStatus: the HTTP status code
details: typed payload specific to the error (e.g. required/available for insufficient_credits)
docsUrl: deep link to the error’s documentation
requestId: the x-request-id header from the response, for support tickets
The full catalog is at Errors.
Automatic retries
The SDK retries these codes automatically with exponential backoff before throwing: rate_limited (honors Retry-After), concurrency_limited, internal_error, upstream_unavailable, scrape_timeout. All other codes throw immediately.
Webhooks per request
Every read() call can include an inline webhook config that fires on job lifecycle events, useful for fire-and-forget batches.
await client.read({
urls: manyUrls,
webhook: {
url: "https://your-app.example.com/hooks/reader",
events: ["job.completed", "job.failed"],
secret: process.env.READER_WEBHOOK_SECRET,
},
});
See Webhooks for the full delivery contract and signature verification.
Types
All public types are re-exported from the package root:
import type {
ReaderClientConfig,
ReadParams,
ReadResult, // { kind: "scrape", data: ScrapeResult } | { kind: "job", data: Job }
ScrapeResult,
ScrapeMetadata,
Job,
JobStatus,
JobMode,
Page,
ProxyMode, // "standard" | "stealth" | "auto"
Pagination,
Credits,
UsageEntry,
StreamEvent,
} from "@vakra-dev/reader-js";
Next