crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scra

What is crw?

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

Overview of crw

What it is

crw is an open-source web scraping, crawling, and search tool written in Rust. It can be run locally as a single binary or used via a managed cloud API. It is designed to provide a faster alternative to Firecrawl and Tavily, offering a drop-in Firecrawl-compatible API for endpoints including /scrape, /crawl, and /search. The project also includes an MCP server for AI agents.

Key features

  • Converts URLs to clean markdown, HTML, links, screenshots, or structured JSON via scrape, crawl, map, search, and extract operations.
  • Claims to be 2.3x faster than Tavily and 1.5x faster than Firecrawl based on a 1,000-URL benchmark dataset from Firecrawl.
  • Idles at approximately 14 MB RAM and runs as a single binary.
  • Provides a CLI, Python SDK, Node.js SDK, and REST API.
  • Supports self-hosted local deployment or a managed cloud service with 1,000 free credits (no credit card required).
  • Includes an MCP server that automatically integrates with AI coding tools such as Claude Code, Cursor, Codex, Gemini CLI, OpenCode, and Windsurf.

Use cases

  • Converting web pages into markdown or structured data for AI agent consumption.
  • Performing web searches and extracting results programmatically.
  • Crawling websites to collect all pages within a bounded scope.
  • Discovering URLs on a site without scraping every page.
  • Extracting structured fields from one or many URLs for data pipelines.

Getting started

To install the CLI locally, run:bashcurl -fsSL https://fastcrw.com/install | shAfter installation, you can scrape a URL immediately with crw https://example.com. To use the cloud API, paste your API key during installation:bashcurl -fsSL https://fastcrw.com/install | CRW_API_KEY=crw_live_... shFor Python SDK: pip install crw and use the CrwClient. For Node.js: npm install crw-sdk and use the CrwClient. For MCP server for AI agents: npx -y crw-mcp@latest install.

License and pricing

The engine and MCP server are licensed under AGPL-3.0. The Python and TypeScript SDKs are licensed under MIT. The metadata states the license is AGPL-3.0 and the pricing type is open_source. The managed cloud service offers 1,000 free credits with no credit card required; further pricing details are not stated.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/fastcrwcrw.svg)](https://yunodb.com/tools/fastcrwcrw?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/fastcrwcrw?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/fastcrwcrw.svg" alt="Featured on YunoDB" height="20"></a>