summary: "Firecrawl search, scrape, and web_fetch fallback"

read_when:

- You want Firecrawl-backed web extraction

- You need a Firecrawl API key

- You want keyless Firecrawl web_fetch

- You need a Firecrawl API key for search or higher limits

- You want Firecrawl as a web_search provider

- You want anti-bot extraction for web_fetch

title: "Firecrawl"

It is a hosted extraction/search service that supports bot circumvention and caching,

which helps with JS-heavy sites or pages that block plain HTTP fetches.

## Get an API key

## Keyless web_fetch and API keys

1. Create a Firecrawl account and generate an API key.

2. Store it in config or set `FIRECRAWL_API_KEY` in the gateway environment.

The explicitly selected hosted Firecrawl `web_fetch` fallback supports starter

access without an API key. Add `FIRECRAWL_API_KEY` in the gateway environment

or configure it when you need higher limits. Firecrawl `web_search` and

`firecrawl_scrape` require an API key.

## Configure Firecrawl search

- `baseUrl` defaults to hosted Firecrawl at `https://api.firecrawl.dev`. Self-hosted overrides are allowed only for private/internal endpoints; HTTP is accepted only for those private targets.

- `FIRECRAWL_BASE_URL` is the shared env fallback for Firecrawl search and scrape base URLs.

## Configure Firecrawl scrape + web_fetch fallback

## Configure Firecrawl web_fetch fallback

```json5

{

tools: {

web: {

fetch: {

provider: "firecrawl", // explicit selection enables keyless fallback

},

},

},

plugins: {

entries: {

firecrawl: {

enabled: true,

config: {

webFetch: {

apiKey: "FIRECRAWL_API_KEY_HERE",

baseUrl: "https://api.firecrawl.dev",

onlyMainContent: true,

maxAgeMs: 172800000,

Notes:

- Firecrawl fallback attempts run only when an API key is available (`plugins.entries.firecrawl.config.webFetch.apiKey` or `FIRECRAWL_API_KEY`).

- The explicitly selected Firecrawl `web_fetch` fallback works without an API key. When configured, OpenClaw sends `plugins.entries.firecrawl.config.webFetch.apiKey` or `FIRECRAWL_API_KEY` for higher limits.

- Choosing Firecrawl during onboarding or `openclaw configure --section web` enables the plugin and selects Firecrawl for `web_fetch` unless another fetch provider is already configured.

- `firecrawl_scrape` requires an API key.

- `maxAgeMs` controls how old cached results can be (ms). Default is 2 days.

- Legacy `tools.web.fetch.firecrawl.*` config is auto-migrated by `openclaw doctor --fix`.

- Firecrawl scrape/base URL overrides follow the same hosted/private rule as search: public hosted traffic uses `https://api.firecrawl.dev`; self-hosted overrides must resolve to private/internal endpoints.

- `firecrawl_scrape` rejects obvious private, loopback, metadata, and non-HTTP(S) target URLs before forwarding them to Firecrawl, matching the `web_fetch` target-safety contract for explicit Firecrawl scrape calls.

`firecrawl_scrape` reuses the same `plugins.entries.firecrawl.config.webFetch.*` settings and env vars.

`firecrawl_scrape` reuses the same `plugins.entries.firecrawl.config.webFetch.*` settings and env vars, including its required API key.

### Self-hosted Firecrawl

`web_fetch` extraction order:

1. Readability (local)

2. Firecrawl (if selected or auto-detected as the active web-fetch fallback)

2. Firecrawl (when selected, or auto-detected from configured credentials)

3. Basic HTML cleanup (last fallback)

The selection knob is `tools.web.fetch.provider`. If you omit it, OpenClaw