Loading repository data…
ScrapeGraphAI / repository
Official JavaScript/TypeScript SDK for the ScrapeGraph AI API. Smart scraping, search, crawling, markdownify, agentic browser automation, scheduled jobs, and structured data extraction — all powered by AI.
A transparent discovery signal based on current public GitHub metadata.
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
Official TypeScript SDK for the ScrapeGraphAI AI API.
npm i scrapegraph-js
# or
bun add scrapegraph-js
Log in to the ScrapeGraphAI dashboard to create an API key. The dashboard also shows your request history, usage, credits, and crawl/monitor activity.
Set it in your environment:
export SGAI_API_KEY=...
import { ScrapeGraphAI } from "scrapegraph-js";
// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
const sgai = ScrapeGraphAI();
const result = await sgai.scrape({
url: "https://example.com",
formats: [{ type: "markdown" }],
});
if (result.status === "success") {
console.log(result.data?.results.markdown?.data);
} else {
console.error(result.error);
}
Every function returns ApiResult<T> — no exceptions to catch:
type ApiResult<T> = {
status: "success" | "error";
data: T | null;
error?: string;
elapsedMs: number;
};
This SDK is a client for the managed cloud API. ScrapeGraphAI also ships an open-source library you can run yourself. This table explains the difference so you can pick the right one.
Open Source (scrapegraphai) | Managed API (this SDK) | |
|---|---|---|
| What it is | A Python library you run yourself | A hosted cloud service you call via SDK |
| Where it runs | Your own infrastructure (self-hosted) | ScrapeGraphAI cloud |
| LLM | Bring your own (OpenAI, Groq, Gemini, Azure, local via Ollama) | Managed for you |
Selected from shared topics, language and repository description—not editorial ratings.
MacPaw /
Official TypeScript SDK for AI Gateway - universal client for browser and Node.js (Chat, Embeddings, Images, Audio, NestJS module)
67/100 healthDeskwoot /
Official JavaScript/TypeScript client for the Deskwoot customer support platform. AI chatbot, multi-channel inbox, Shopify & WooCommerce integration.
67/100 health| Browser / JS rendering | You configure it (Playwright) | Managed (stealth, auto/fast/js modes) |
| Proxies & anti-bot | Your responsibility | Included |
| Scaling & maintenance | Your responsibility | Fully managed |
| Cost model | LLM tokens + your own infra | Pay-as-you-go credits |
| Auth | Your own LLM keys | SGAI_API_KEY |
| Capabilities | Graph pipelines (SmartScraper, Search, Speech, ScriptCreator…) | Scrape, Extract, Search, Crawl, Monitor, History |
| Setup effort | More configuration | Minimal — API key + one call |
| License | MIT | SDK is MIT; the API service is paid |
Choose the open-source library if you want full control, on-prem/self-hosted data, local LLMs (Ollama), or fine-grained cost tuning — and you're happy to manage browsers, proxies and scaling yourself.
Choose the managed API (this SDK) if you want zero infrastructure, managed JS rendering & anti-bot, built-in Crawl and scheduled Monitor jobs, and the fastest path to production — billed per credit.
Scrape a webpage in multiple formats (markdown, html, screenshot, json, etc).
const res = await sgai.scrape({
url: "https://example.com",
formats: [
{ type: "markdown", mode: "reader" },
{ type: "screenshot", fullPage: true, width: 1440, height: 900 },
{ type: "json", prompt: "Extract product info" },
],
contentType: "text/html", // optional, auto-detected
fetchConfig: { // optional
mode: "js", // "auto" | "fast" | "js"
stealth: true,
timeout: 30000,
wait: 2000,
scrolls: 3,
headers: { "Accept-Language": "en" },
cookies: { session: "abc" },
country: "us",
},
});
Formats:
markdown — Clean markdown (modes: normal, reader, prune)html — Raw HTML (modes: normal, reader, prune)links — All links on the pageimages — All image URLssummary — AI-generated summaryjson — Structured extraction with prompt/schemabranding — Brand colors, typography, logosscreenshot — Page screenshot (fullPage, width, height, quality)Extract structured data from a URL, HTML, or markdown using AI.
const res = await sgai.extract({
url: "https://example.com",
prompt: "Extract product names and prices",
schema: { /* JSON schema */ }, // optional
mode: "reader", // optional
fetchConfig: { /* ... */ }, // optional
});
// Or pass html/markdown directly instead of url
Search the web and optionally extract structured data.
const res = await sgai.search({
query: "best programming languages 2024",
numResults: 5, // 1-20, default 3
format: "markdown", // "markdown" | "html"
prompt: "Extract key points", // optional, for AI extraction
schema: { /* ... */ }, // optional
timeRange: "past_week", // optional
locationGeoCode: "us", // optional
fetchConfig: { /* ... */ }, // optional
});
Crawl a website and its linked pages.
// Start a crawl
const start = await sgai.crawl.start({
url: "https://example.com",
formats: [{ type: "markdown" }],
maxPages: 50,
maxDepth: 2,
maxLinksPerPage: 10,
includePatterns: ["/blog/*"],
excludePatterns: ["/admin/*"],
fetchConfig: { /* ... */ },
});
// Check status
const status = await sgai.crawl.get(start.data?.id!);
// Fetch paginated pages with resolved scrape results
const pages = await sgai.crawl.pages(start.data?.id!, {
cursor: 0,
limit: 50,
});
// Control
await sgai.crawl.stop(id);
await sgai.crawl.resume(id);
await sgai.crawl.delete(id);
Monitor a webpage for changes on a schedule.
// Create a monitor
const mon = await sgai.monitor.create({
url: "https://example.com",
name: "Price Monitor",
interval: "0 * * * *", // cron expression
formats: [{ type: "markdown" }],
webhookUrl: "https://...", // optional
fetchConfig: { /* ... */ },
});
// Manage monitors
await sgai.monitor.list();
await sgai.monitor.get(cronId);
await sgai.monitor.update(cronId, { interval: "0 */6 * * *" });
await sgai.monitor.pause(cronId);
await sgai.monitor.resume(cronId);
await sgai.monitor.delete(cronId);
Fetch request history.
const list = await sgai.history.list({
service: "scrape", // optional filter
page: 1,
limit: 20,
});
const entry = await sgai.history.get("request-id");
const credits = await sgai.credits();
// { remaining: 1000, used: 500, plan: "pro", jobs: { crawl: {...}, monitor: {...} } }
const health = await sgai.healthy();
// { status: "ok", uptime: 12345 }
| Service | Example | Description |
|---|---|---|
| scrape | scrape_basic.ts | Basic markdown scraping |
| scrape | scrape_multi_format.ts | Multiple formats (markdown, links, images, screenshot, summary) |
| scrape | scrape_json_extraction.ts | Structured JSON extraction with schema |
| scrape | scrape_pdf.ts | PDF document parsing with OCR metadata |
| scrape | scrape_with_fetchconfig.ts | JS rendering, stealth mode, scrolling |
| extract | extract_basic.ts | AI data extraction from URL |
| extract | extract_with_schema.ts | Extraction with JSON schema |
| search | search_basic.ts | Web search with results |
| search | search_with_extraction.ts | Search + AI extraction |
| crawl | crawl_basic.ts | Start and monitor a crawl |
| crawl | crawl_with_formats.ts | Crawl with screenshots and patterns |
| monitor | monitor_basic.ts | Create a page monitor |
| monitor | monitor_with_webhook.ts | Monitor with webhook notifications |
| utilities | credits.ts | Check account credits and limits |
| utilities | health.ts | API health check |
| utilities | history.ts | Request history |
| Variable | Description | Default |
|---|---|---|
SGAI_API_KEY | Your ScrapeGraphAI API key | — |
SGAI_API_URL | Override API base URL | https://v2-api.scrapegraphai.com/api |
SGAI_DEBUG | Enable debug logging ("1") | off |
SGAI_TIMEOUT | Request timeout in seconds | 120 |
bun install
bun run test # unit tests
bun run test:integration # live API tests (requires SGAI_API_KEY)
bun run build # tsup → dist/
bun run check # tsc --noEmit + biome
MIT - ScrapeGraphAI AI
amigo-ai /
Official TypeScript SDK for the Classic Amigo API at api.amigo.ai
wauldo /
Official TypeScript SDK for Wauldo — verified AI answers with zero hallucinations. npm install wauldo
66/100 healthAsiern /
open-cfmoto is an AI-assisted, privacy-conscious reverse-engineering project that rebuilds CFMoto 450MT connectivity: a TypeScript BLE protocol stack, cloud API client, and mobile app to provide an open alternative to the official CFMoto Ride experience, with documented protocol evidence, hardware-validation checklists, and Docusaurus docs.
47/100 healthCardScan-ai /
Official API clients for the Cardscan.ai API, including Python, Typescript, Swift and more.
22/100 health