@@ -0,0 +1,343 @@
1+---
2+summary: "OpenClaw browser control API, CLI reference, and scripting actions"
3+read_when:
4+ - Scripting or debugging the agent browser via the local control API
5+ - Looking for the `openclaw browser` CLI reference
6+ - Adding custom browser automation with snapshots and refs
7+title: "Browser control API"
8+---
9+10+For setup, configuration, and troubleshooting, see [Browser](/tools/browser).
11+This page is the reference for the local control HTTP API, the `openclaw browser`
12+CLI, and scripting patterns (snapshots, refs, waits, debug flows).
13+14+## Control API (optional)
15+16+For local integrations only, the Gateway exposes a small loopback HTTP API:
17+18+- Status/start/stop: `GET /`, `POST /start`, `POST /stop`
19+- Tabs: `GET /tabs`, `POST /tabs/open`, `POST /tabs/focus`, `DELETE /tabs/:targetId`
20+- Snapshot/screenshot: `GET /snapshot`, `POST /screenshot`
21+- Actions: `POST /navigate`, `POST /act`
22+- Hooks: `POST /hooks/file-chooser`, `POST /hooks/dialog`
23+- Downloads: `POST /download`, `POST /wait/download`
24+- Debugging: `GET /console`, `POST /pdf`
25+- Debugging: `GET /errors`, `GET /requests`, `POST /trace/start`, `POST /trace/stop`, `POST /highlight`
26+- Network: `POST /response/body`
27+- State: `GET /cookies`, `POST /cookies/set`, `POST /cookies/clear`
28+- State: `GET /storage/:kind`, `POST /storage/:kind/set`, `POST /storage/:kind/clear`
29+- Settings: `POST /set/offline`, `POST /set/headers`, `POST /set/credentials`, `POST /set/geolocation`, `POST /set/media`, `POST /set/timezone`, `POST /set/locale`, `POST /set/device`
30+31+All endpoints accept `?profile=<name>`.
32+33+If shared-secret gateway auth is configured, browser HTTP routes require auth too:
34+35+- `Authorization: Bearer <gateway token>`
36+- `x-openclaw-password: <gateway password>` or HTTP Basic auth with that password
37+38+Notes:
39+40+- This standalone loopback browser API does **not** consume trusted-proxy or
41+ Tailscale Serve identity headers.
42+- If `gateway.auth.mode` is `none` or `trusted-proxy`, these loopback browser
43+ routes do not inherit those identity-bearing modes; keep them loopback-only.
44+45+### `/act` error contract
46+47+`POST /act` uses a structured error response for route-level validation and
48+policy failures:
49+50+```json
51+{ "error": "<message>", "code": "ACT_*" }
52+```
53+54+Current `code` values:
55+56+- `ACT_KIND_REQUIRED` (HTTP 400): `kind` is missing or unrecognized.
57+- `ACT_INVALID_REQUEST` (HTTP 400): action payload failed normalization or validation.
58+- `ACT_SELECTOR_UNSUPPORTED` (HTTP 400): `selector` was used with an unsupported action kind.
59+- `ACT_EVALUATE_DISABLED` (HTTP 403): `evaluate` (or `wait --fn`) is disabled by config.
60+- `ACT_TARGET_ID_MISMATCH` (HTTP 403): top-level or batched `targetId` conflicts with request target.
61+- `ACT_EXISTING_SESSION_UNSUPPORTED` (HTTP 501): action is not supported for existing-session profiles.
62+63+Other runtime failures may still return `{ "error": "<message>" }` without a
64+`code` field.
65+66+### Playwright requirement
67+68+Some features (navigate/act/AI snapshot/role snapshot, element screenshots,
69+PDF) require Playwright. If Playwright isn’t installed, those endpoints return
70+a clear 501 error.
71+72+What still works without Playwright:
73+74+- ARIA snapshots
75+- Page screenshots for the managed `openclaw` browser when a per-tab CDP
76+ WebSocket is available
77+- Page screenshots for `existing-session` / Chrome MCP profiles
78+- `existing-session` ref-based screenshots (`--ref`) from snapshot output
79+80+What still needs Playwright:
81+82+- `navigate`
83+- `act`
84+- AI snapshots / role snapshots
85+- CSS-selector element screenshots (`--element`)
86+- full browser PDF export
87+88+Element screenshots also reject `--full-page`; the route returns `fullPage is
89+not supported for element screenshots`.
90+91+If you see `Playwright is not available in this gateway build`, repair the
92+bundled browser plugin runtime dependencies so `playwright-core` is installed,
93+then restart the gateway. For packaged installs, run `openclaw doctor --fix`.
94+For Docker, also install the Chromium browser binaries as shown below.
95+96+#### Docker Playwright install
97+98+If your Gateway runs in Docker, avoid `npx playwright` (npm override conflicts).
99+Use the bundled CLI instead:
100+101+```bash
102+docker compose run --rm openclaw-cli \
103+ node /app/node_modules/playwright-core/cli.js install chromium
104+```
105+106+To persist browser downloads, set `PLAYWRIGHT_BROWSERS_PATH` (for example,
107+`/home/node/.cache/ms-playwright`) and make sure `/home/node` is persisted via
108+`OPENCLAW_HOME_VOLUME` or a bind mount. See [Docker](/install/docker).
109+110+## How it works (internal)
111+112+A small loopback control server accepts HTTP requests and connects to Chromium-based browsers via CDP. Advanced actions (click/type/snapshot/PDF) go through Playwright on top of CDP; when Playwright is missing, only non-Playwright operations are available. The agent sees one stable interface while local/remote browsers and profiles swap freely underneath.
113+114+## CLI quick reference
115+116+All commands accept `--browser-profile <name>` to target a specific profile, and `--json` for machine-readable output.
117+118+<AccordionGroup>
119+120+<Accordion title="Basics: status, tabs, open/focus/close">
121+122+```bash
123+openclaw browser status
124+openclaw browser start
125+openclaw browser stop # also clears emulation on attach-only/remote CDP
126+openclaw browser tabs
127+openclaw browser tab # shortcut for current tab
128+openclaw browser tab new
129+openclaw browser tab select 2
130+openclaw browser tab close 2
131+openclaw browser open https://example.com
132+openclaw browser focus abcd1234
133+openclaw browser close abcd1234
134+```
135+136+</Accordion>
137+138+<Accordion title="Inspection: screenshot, snapshot, console, errors, requests">
139+140+```bash
141+openclaw browser screenshot
142+openclaw browser screenshot --full-page
143+openclaw browser screenshot --ref 12 # or --ref e12
144+openclaw browser snapshot
145+openclaw browser snapshot --format aria --limit 200
146+openclaw browser snapshot --interactive --compact --depth 6
147+openclaw browser snapshot --efficient
148+openclaw browser snapshot --labels
149+openclaw browser snapshot --selector "#main" --interactive
150+openclaw browser snapshot --frame "iframe#main" --interactive
151+openclaw browser console --level error
152+openclaw browser errors --clear
153+openclaw browser requests --filter api --clear
154+openclaw browser pdf
155+openclaw browser responsebody "**/api" --max-chars 5000
156+```
157+158+</Accordion>
159+160+<Accordion title="Actions: navigate, click, type, drag, wait, evaluate">
161+162+```bash
163+openclaw browser navigate https://example.com
164+openclaw browser resize 1280 720
165+openclaw browser click 12 --double # or e12 for role refs
166+openclaw browser type 23 "hello" --submit
167+openclaw browser press Enter
168+openclaw browser hover 44
169+openclaw browser scrollintoview e12
170+openclaw browser drag 10 11
171+openclaw browser select 9 OptionA OptionB
172+openclaw browser download e12 report.pdf
173+openclaw browser waitfordownload report.pdf
174+openclaw browser upload /tmp/openclaw/uploads/file.pdf
175+openclaw browser fill --fields '[{"ref":"1","type":"text","value":"Ada"}]'
176+openclaw browser dialog --accept
177+openclaw browser wait --text "Done"
178+openclaw browser wait "#main" --url "**/dash" --load networkidle --fn "window.ready===true"
179+openclaw browser evaluate --fn '(el) => el.textContent' --ref 7
180+openclaw browser highlight e12
181+openclaw browser trace start
182+openclaw browser trace stop
183+```
184+185+</Accordion>
186+187+<Accordion title="State: cookies, storage, offline, headers, geo, device">
188+189+```bash
190+openclaw browser cookies
191+openclaw browser cookies set session abc123 --url "https://example.com"
192+openclaw browser cookies clear
193+openclaw browser storage local get
194+openclaw browser storage local set theme dark
195+openclaw browser storage session clear
196+openclaw browser set offline on
197+openclaw browser set headers --headers-json '{"X-Debug":"1"}'
198+openclaw browser set credentials user pass # --clear to remove
199+openclaw browser set geo 37.7749 -122.4194 --origin "https://example.com"
200+openclaw browser set media dark
201+openclaw browser set timezone America/New_York
202+openclaw browser set locale en-US
203+openclaw browser set device "iPhone 14"
204+```
205+206+</Accordion>
207+208+</AccordionGroup>
209+210+Notes:
211+212+- `upload` and `dialog` are **arming** calls; run them before the click/press that triggers the chooser/dialog.
213+- `click`/`type`/etc require a `ref` from `snapshot` (numeric `12` or role ref `e12`). CSS selectors are intentionally not supported for actions.
214+- Download, trace, and upload paths are constrained to OpenClaw temp roots: `/tmp/openclaw{,/downloads,/uploads}` (fallback: `${os.tmpdir()}/openclaw/...`).
215+- `upload` can also set file inputs directly via `--input-ref` or `--element`.
216+217+Snapshot flags at a glance:
218+219+- `--format ai` (default with Playwright): AI snapshot with numeric refs (`aria-ref="<n>"`).
220+- `--format aria`: accessibility tree, no refs; inspection only.
221+- `--efficient` (or `--mode efficient`): compact role snapshot preset. Set `browser.snapshotDefaults.mode: "efficient"` to make this the default (see [Gateway configuration](/gateway/configuration-reference#browser)).
222+- `--interactive`, `--compact`, `--depth`, `--selector` force a role snapshot with `ref=e12` refs. `--frame "<iframe>"` scopes role snapshots to an iframe.
223+- `--labels` adds a viewport-only screenshot with overlayed ref labels (prints `MEDIA:<path>`).
224+225+## Snapshots and refs
226+227+OpenClaw supports two “snapshot” styles:
228+229+- **AI snapshot (numeric refs)**: `openclaw browser snapshot` (default; `--format ai`)
230+ - Output: a text snapshot that includes numeric refs.
231+ - Actions: `openclaw browser click 12`, `openclaw browser type 23 "hello"`.
232+ - Internally, the ref is resolved via Playwright’s `aria-ref`.
233+234+- **Role snapshot (role refs like `e12`)**: `openclaw browser snapshot --interactive` (or `--compact`, `--depth`, `--selector`, `--frame`)
235+ - Output: a role-based list/tree with `[ref=e12]` (and optional `[nth=1]`).
236+ - Actions: `openclaw browser click e12`, `openclaw browser highlight e12`.
237+ - Internally, the ref is resolved via `getByRole(...)` (plus `nth()` for duplicates).
238+ - Add `--labels` to include a viewport screenshot with overlayed `e12` labels.
239+240+Ref behavior:
241+242+- Refs are **not stable across navigations**; if something fails, re-run `snapshot` and use a fresh ref.
243+- If the role snapshot was taken with `--frame`, role refs are scoped to that iframe until the next role snapshot.
244+245+## Wait power-ups
246+247+You can wait on more than just time/text:
248+249+- Wait for URL (globs supported by Playwright):
250+ - `openclaw browser wait --url "**/dash"`
251+- Wait for load state:
252+ - `openclaw browser wait --load networkidle`
253+- Wait for a JS predicate:
254+ - `openclaw browser wait --fn "window.ready===true"`
255+- Wait for a selector to become visible:
256+ - `openclaw browser wait "#main"`
257+258+These can be combined:
259+260+```bash
261+openclaw browser wait "#main" \
262+ --url "**/dash" \
263+ --load networkidle \
264+ --fn "window.ready===true" \
265+ --timeout-ms 15000
266+```
267+268+## Debug workflows
269+270+When an action fails (e.g. “not visible”, “strict mode violation”, “covered”):
271+272+1. `openclaw browser snapshot --interactive`
273+2. Use `click <ref>` / `type <ref>` (prefer role refs in interactive mode)
274+3. If it still fails: `openclaw browser highlight <ref>` to see what Playwright is targeting
275+4. If the page behaves oddly:
276+ - `openclaw browser errors --clear`
277+ - `openclaw browser requests --filter api --clear`
278+5. For deep debugging: record a trace:
279+ - `openclaw browser trace start`
280+ - reproduce the issue
281+ - `openclaw browser trace stop` (prints `TRACE:<path>`)
282+283+## JSON output
284+285+`--json` is for scripting and structured tooling.
286+287+Examples:
288+289+```bash
290+openclaw browser status --json
291+openclaw browser snapshot --interactive --json
292+openclaw browser requests --filter api --json
293+openclaw browser cookies --json
294+```
295+296+Role snapshots in JSON include `refs` plus a small `stats` block (lines/chars/refs/interactive) so tools can reason about payload size and density.
297+298+## State and environment knobs
299+300+These are useful for “make the site behave like X” workflows:
301+302+- Cookies: `cookies`, `cookies set`, `cookies clear`
303+- Storage: `storage local|session get|set|clear`
304+- Offline: `set offline on|off`
305+- Headers: `set headers --headers-json '{"X-Debug":"1"}'` (legacy `set headers --json '{"X-Debug":"1"}'` remains supported)
306+- HTTP basic auth: `set credentials user pass` (or `--clear`)
307+- Geolocation: `set geo <lat> <lon> --origin "https://example.com"` (or `--clear`)
308+- Media: `set media dark|light|no-preference|none`
309+- Timezone / locale: `set timezone ...`, `set locale ...`
310+- Device / viewport:
311+ - `set device "iPhone 14"` (Playwright device presets)
312+ - `set viewport 1280 720`
313+314+## Security and privacy
315+316+- The openclaw browser profile may contain logged-in sessions; treat it as sensitive.
317+- `browser act kind=evaluate` / `openclaw browser evaluate` and `wait --fn`
318+ execute arbitrary JavaScript in the page context. Prompt injection can steer
319+ this. Disable it with `browser.evaluateEnabled=false` if you do not need it.
320+- For logins and anti-bot notes (X/Twitter, etc.), see [Browser login + X/Twitter posting](/tools/browser-login).
321+- Keep the Gateway/node host private (loopback or tailnet-only).
322+- Remote CDP endpoints are powerful; tunnel and protect them.
323+324+Strict-mode example (block private/internal destinations by default):
325+326+```json5
327+{
328+ browser: {
329+ ssrfPolicy: {
330+ dangerouslyAllowPrivateNetwork: false,
331+ hostnameAllowlist: ["*.example.com", "example.com"],
332+ allowedHostnames: ["localhost"], // optional exact allow
333+ },
334+ },
335+}
336+```
337+338+## Related
339+340+- [Browser](/tools/browser) — overview, configuration, profiles, security
341+- [Browser login](/tools/browser-login) — signing in to sites
342+- [Browser Linux troubleshooting](/tools/browser-linux-troubleshooting)
343+- [Browser WSL2 troubleshooting](/tools/browser-wsl2-windows-remote-cdp-troubleshooting)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。