crw
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scra
What is crw?
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
Overview of crw
What it is
crw is an open-source web scraping, crawling, and search tool written in Rust. It can be run locally as a single binary or used via a managed cloud API. It is designed to provide a faster alternative to Firecrawl and Tavily, offering a drop-in Firecrawl-compatible API for endpoints including /scrape, /crawl, and /search. The project also includes an MCP server for AI agents.
Key features
- Converts URLs to clean markdown, HTML, links, screenshots, or structured JSON via scrape, crawl, map, search, and extract operations.
- Claims to be 2.3x faster than Tavily and 1.5x faster than Firecrawl based on a 1,000-URL benchmark dataset from Firecrawl.
- Idles at approximately 14 MB RAM and runs as a single binary.
- Provides a CLI, Python SDK, Node.js SDK, and REST API.
- Supports self-hosted local deployment or a managed cloud service with 1,000 free credits (no credit card required).
- Includes an MCP server that automatically integrates with AI coding tools such as Claude Code, Cursor, Codex, Gemini CLI, OpenCode, and Windsurf.
Use cases
- Converting web pages into markdown or structured data for AI agent consumption.
- Performing web searches and extracting results programmatically.
- Crawling websites to collect all pages within a bounded scope.
- Discovering URLs on a site without scraping every page.
- Extracting structured fields from one or many URLs for data pipelines.
Getting started
To install the CLI locally, run:bashcurl -fsSL https://fastcrw.com/install | shAfter installation, you can scrape a URL immediately with crw https://example.com. To use the cloud API, paste your API key during installation:bashcurl -fsSL https://fastcrw.com/install | CRW_API_KEY=crw_live_... shFor Python SDK: pip install crw and use the CrwClient. For Node.js: npm install crw-sdk and use the CrwClient. For MCP server for AI agents: npx -y crw-mcp@latest install.
License and pricing
The engine and MCP server are licensed under AGPL-3.0. The Python and TypeScript SDKs are licensed under MIT. The metadata states the license is AGPL-3.0 and the pricing type is open_source. The managed cloud service offers 1,000 free credits with no credit card required; further pricing details are not stated.
Based on the project's public README and repository metadata.