---
ref: tool:firecrawl/crawl-site
name: Crawl a website
company: company:firecrawl
workflows: []
access: [mcp, cli, api]
tags: [capability:scrape-web, category:scraper, has:api, has:cli, has:mcp]
docs: https://docs.firecrawl.dev/api-reference/endpoint/crawl-post
updated: 2026-09-26
---

# Crawl a website

Follows links from a starting URL and returns the content of every page it reaches, within the limits you set.

Note: The API starts a job and returns its `id`: read the pages with `GET /crawl/{id}`; the MCP tool waits for the crawl to finish. Set a `limit` and include or exclude paths to keep the crawl bounded.

## Set up

Use the first option your agent supports.

### MCP (official, remote)

Add this server to your agent's MCP settings, then sign in when asked.

```json
{ "mcpServers": { "firecrawl": { "url": "https://mcp.firecrawl.dev/v2/mcp-oauth" } } }
```

Call the MCP tool `firecrawl_crawl`.

Server URL: https://mcp.firecrawl.dev/v2/mcp-oauth

### CLI (official)

Install the command, then confirm it runs.

```sh
npm install -g firecrawl-cli
firecrawl --version
```

Run `firecrawl crawl`.

Set `$FIRECRAWL_API_KEY` in your environment first (get a key: https://www.firecrawl.dev/app/api-keys).

### API (official)

- Base URL: https://api.firecrawl.dev/v2
- Endpoint: `POST /crawl`
- Auth: send the header `Authorization: Bearer $FIRECRAWL_API_KEY`
- Get a key: https://www.firecrawl.dev/app/api-keys
- Docs: https://docs.firecrawl.dev/api-reference/introduction

Before anything else, confirm access with the cheapest read-only call, like a list or a search. Never send or change anything to test access.

## Rules

- Ask the user before anything that sends messages, costs money, or changes data, and say how many records it touches. One approval covers a batch the user has seen.
- Never print API keys.