惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

M
MIT News - Artificial intelligence
雷峰网
雷峰网
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Last Week in AI
Last Week in AI
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
阮一峰的网络日志
阮一峰的网络日志
月光博客
月光博客
博客园 - Franky
腾讯CDC
T
Tailwind CSS Blog
Recent Announcements
Recent Announcements
V
V2EX
N
Netflix TechBlog - Medium
量子位
Jina AI
Jina AI
Y
Y Combinator Blog
The GitHub Blog
The GitHub Blog
G
Google Developers Blog
爱范儿
爱范儿
博客园 - 叶小钗
D
Docker
MongoDB | Blog
MongoDB | Blog
D
DataBreaches.Net
T
The Blog of Author Tim Ferriss

Recent Commits to openclaw:main

test: merge chat side-result checks · openclaw/openclaw@ddd2c2a test: merge cron history checks · openclaw/openclaw@f7eb746 test: merge responsive navigation shell checks · openclaw/openclaw@c2e4b47 docs(changelog): add codex oauth fixes · openclaw/openclaw@628e6cd test: merge navigation routing cases · openclaw/openclaw@5d8cecb Tests: mock channel registry bundled fallback · openclaw/openclaw@2b08233 Secrets: avoid broad web search discovery for single plugin config · openclaw/openclaw@a464f59 test: merge config view browser checks · openclaw/openclaw@20cf511 fix(status): align oauth health with runtime · openclaw/openclaw@eed7116 feat: add macOS screen snapshots for monitor preview (#67954) thanks … · openclaw/openclaw@f377db1 fix: report shared auth scopes in hello-ok (#67810) thanks @BunsDev · openclaw/openclaw@0b6c39b Auto-reply: avoid eager bundled route fallback · openclaw/openclaw@3ea1bf4 Tests: narrow session binding contract setup · openclaw/openclaw@54e4e16 fix(macOS): enable undo/redo in webchat composer text input (#34962) · openclaw/openclaw@00951dc Tests: speed up channel setup promotion · openclaw/openclaw@82b529a Docs: refresh agent instructions · openclaw/openclaw@5775fe2 fix(auth): serialize OAuth refresh across agents to fix #26322 (#67876) · openclaw/openclaw@8e79080 test: allow ollama public surface boundary test · openclaw/openclaw@7d4f1a6 Docs: add test performance guardrails · openclaw/openclaw@89706d3 Tests: restore context-engine usage proof · openclaw/openclaw@e4c4f95 Tests: slim context engine runtime coverage · openclaw/openclaw@74c198f ci: retry failed custom checkouts · openclaw/openclaw@0ee5baf test: trim duplicate provider auth onboarding cases · openclaw/openclaw@1ffc02e matrix: fix sessions_spawn --thread subagent session spawning (#67643) · openclaw/openclaw@1ce2596 test: reduce auth choice fixture churn · openclaw/openclaw@857b9cd test: mock health status config boundaries · openclaw/openclaw@9d5ab4a test: mock onboard config io boundary · openclaw/openclaw@299694d test: mock legacy state plugin boundaries · openclaw/openclaw@2713089 test: mock channel install boundaries · openclaw/openclaw@b945248 test: mock doctor preview channel boundaries · openclaw/openclaw@b1a3ad4
feat(browser): surface observed dialogs (#83099) · opencl...
steipete · 2026-05-18 · via Recent Commits to openclaw:main
Original file line numberDiff line numberDiff line change

@@ -9,6 +9,7 @@ Docs: https://docs.openclaw.ai

99

- Mac app: redesign Settings pages with consistent card layouts, cached navigation, cleaner permissions/voice/skills/cron/exec/debug panes, and steadier spacing around the native sidebar.

1010

- Skills: rename the repo-local Codex closeout review skill and helper to `autoreview` while preserving the Codex-first fallback behavior.

1111

- Skills: add a meme-maker skill for curated template search, local SVG/PNG rendering, Imgflip hosted rendering, and Know Your Meme provenance links.

12+

- Browser: surface pending and recently handled modal dialogs in snapshots, return `blockedByDialog` when an action opens a modal, and allow `browser dialog --dialog-id` to answer pending dialogs.

1213

- Agents/tools: shorten built-in tool descriptions and schema hints across media, messaging, sessions, cron, Gateway, web, image/PDF, TTS, nodes, and plan tools while preserving routing guardrails.

1314

- Skills: add node inspector debugging, fused diagram generation, and throwaway spike workflow skills.

1415

- CLI/plugins: add `defineToolPlugin` plus `openclaw plugins build`, `validate`, and `init` for typed simple tool plugins with generated manifest metadata, optional tool declarations, and context factories.

Original file line numberDiff line numberDiff line change

@@ -204,12 +204,17 @@ openclaw browser upload /tmp/openclaw/uploads/file.pdf --ref <ref>

204204

openclaw browser waitfordownload

205205

openclaw browser download <ref> report.pdf

206206

openclaw browser dialog --accept

207+

openclaw browser dialog --dismiss --dialog-id d1

207208

```

208209
209210

Managed Chrome profiles save ordinary click-triggered downloads into the OpenClaw

210211

downloads directory (`/tmp/openclaw/downloads` by default, or the configured temp

211212

root). Use `waitfordownload` or `download` when the agent needs to wait for a

212213

specific file and return its path; those explicit waiters own the next download.

214+

When an action opens a modal dialog, the action response returns

215+

`blockedByDialog` with `browserState.dialogs.pending`; pass `--dialog-id` to

216+

answer it directly. Dialogs handled outside OpenClaw appear under

217+

`browserState.dialogs.recent`.

213218
214219

## State and storage

215220
Original file line numberDiff line numberDiff line change

@@ -193,6 +193,7 @@ openclaw browser waitfordownload report.pdf

193193

openclaw browser upload /tmp/openclaw/uploads/file.pdf

194194

openclaw browser fill --fields '[{"ref":"1","type":"text","value":"Ada"}]'

195195

openclaw browser dialog --accept

196+

openclaw browser dialog --dismiss --dialog-id d1

196197

openclaw browser wait --text "Done"

197198

openclaw browser wait "#main" --url "**/dash" --load networkidle --fn "window.ready===true"

198199

openclaw browser evaluate --fn '(el) => el.textContent' --ref 7

@@ -228,7 +229,7 @@ openclaw browser set device "iPhone 14"

228229
229230

Notes:

230231
231-

- `upload` and `dialog` are **arming** calls; run them before the click/press that triggers the chooser/dialog.

232+

- `upload` and `dialog` are **arming** calls; run them before the click/press that triggers the chooser/dialog. If an action opens a modal, the action response includes `blockedByDialog` and `browserState.dialogs.pending`; pass that `dialogId` to respond directly. Dialogs handled outside OpenClaw appear under `browserState.dialogs.recent`.

232233

- `click`/`type`/etc require a `ref` from `snapshot` (numeric `12`, role ref `e12`, or actionable ARIA ref `ax12`). CSS selectors are intentionally not supported for actions. Use `click-coords` when the visible viewport position is the only reliable target.

233234

- Download, trace, and upload paths are constrained to OpenClaw temp roots: `/tmp/openclaw{,/downloads,/uploads}` (fallback: `${os.tmpdir()}/openclaw/...`).

234235

- `upload` can also set file inputs directly via `--input-ref` or `--element`.

Original file line numberDiff line numberDiff line change

@@ -646,7 +646,8 @@ Compared to the managed `openclaw` profile, existing-session drivers are more co

646646
647647

- **Screenshots** - page captures and `--ref` element captures work; CSS `--element` selectors do not. `--full-page` cannot combine with `--ref` or `--element`. Playwright is not required for page or ref-based element screenshots.

648648

- **Actions** - `click`, `type`, `hover`, `scrollIntoView`, `drag`, and `select` require snapshot refs (no CSS selectors). `click-coords` clicks visible viewport coordinates and does not require a snapshot ref. `click` is left-button only. `type` does not support `slowly=true`; use `fill` or `press`. `press` does not support `delayMs`. `type`, `hover`, `scrollIntoView`, `drag`, `select`, `fill`, and `evaluate` do not support per-call timeouts. `select` accepts a single value.

649-

- **Wait / upload / dialog** - `wait --url` supports exact, substring, and glob patterns; `wait --load networkidle` is not supported. Upload hooks require `ref` or `inputRef`, one file at a time, no CSS `element`. Dialog hooks do not support timeout overrides.

649+

- **Wait / upload / dialog** - `wait --url` supports exact, substring, and glob patterns; `wait --load networkidle` is not supported. Upload hooks require `ref` or `inputRef`, one file at a time, no CSS `element`. Dialog hooks do not support timeout overrides or `dialogId`.

650+

- **Dialog visibility** - Managed browser action responses include `blockedByDialog` and `browserState.dialogs.pending` when an action opens a modal dialog; snapshots also include pending dialog state. Respond with `browser dialog --accept/--dismiss --dialog-id <id>` while a dialog is pending. Dialogs handled outside OpenClaw appear under `browserState.dialogs.recent`.

650651

- **Managed-only features** - batch actions, PDF export, download interception, and `responsebody` still require the managed browser path.

651652
652653

</Accordion>

Original file line numberDiff line numberDiff line change

@@ -404,6 +404,31 @@ export async function executeSnapshotAction(params: {

404404

}

405405

params.onTabActivity?.(readStringValue(snapshot.targetId) ?? targetId);

406406

if (snapshot.format === "ai") {

407+

const dialogStateFields = {

408+

...(snapshot.blockedByDialog ? { blockedByDialog: true } : {}),

409+

...(snapshot.browserState !== undefined ? { browserState: snapshot.browserState } : {}),

410+

};

411+

if (snapshot.blockedByDialog) {

412+

const wrapped = wrapBrowserExternalJson({

413+

kind: "snapshot",

414+

payload: {

415+

format: snapshot.format,

416+

targetId: snapshot.targetId,

417+

url: snapshot.url,

418+

...dialogStateFields,

419+

},

420+

});

421+

return {

422+

content: [{ type: "text" as const, text: wrapped.wrappedText }],

423+

details: {

424+

...wrapped.safeDetails,

425+

format: snapshot.format,

426+

targetId: snapshot.targetId,

427+

url: snapshot.url,

428+

...dialogStateFields,

429+

},

430+

};

431+

}

407432

const extractedText = snapshot.snapshot ?? "";

408433

const wrappedSnapshot = wrapExternalContent(extractedText, {

409434

source: "browser",

@@ -423,6 +448,7 @@ export async function executeSnapshotAction(params: {

423448

imagePath: snapshot.imagePath,

424449

imageType: snapshot.imageType,

425450

refsFallback,

451+

...dialogStateFields,

426452

externalContent: {

427453

untrusted: true,

428454

source: "browser",

@@ -457,6 +483,8 @@ export async function executeSnapshotAction(params: {

457483

targetId: snapshot.targetId,

458484

url: snapshot.url,

459485

nodeCount: snapshot.nodes.length,

486+

...(snapshot.blockedByDialog ? { blockedByDialog: true } : {}),

487+

...(snapshot.browserState !== undefined ? { browserState: snapshot.browserState } : {}),

460488

externalContent: {

461489

untrusted: true,

462490

source: "browser",

Original file line numberDiff line numberDiff line change

@@ -118,6 +118,7 @@ export const BrowserToolSchema = Type.Object({

118118

paths: Type.Optional(Type.Array(Type.String())),

119119

inputRef: Type.Optional(Type.String()),

120120

timeoutMs: Type.Optional(Type.Number()),

121+

dialogId: Type.Optional(Type.String()),

121122

accept: Type.Optional(Type.Boolean()),

122123

promptText: Type.Optional(Type.String()),

123124

// Legacy flattened act params (preferred: request={...})

Original file line numberDiff line numberDiff line change

@@ -1230,6 +1230,35 @@ describe("browser tool external content wrapping", () => {

12301230

expect(details.nodeCount).toBe(1);

12311231

});

12321232
1233+

it("preserves pending dialog state in ai snapshot results", async () => {

1234+

browserClientMocks.browserSnapshot.mockResolvedValueOnce({

1235+

ok: true,

1236+

format: "ai",

1237+

targetId: "t1",

1238+

url: "https://example.com",

1239+

snapshot: "",

1240+

blockedByDialog: true,

1241+

browserState: {

1242+

dialogs: {

1243+

pending: [{ id: "d1", type: "confirm", message: "Continue?" }],

1244+

recent: [],

1245+

},

1246+

},

1247+

});

1248+
1249+

const tool = createBrowserTool();

1250+

const result = await tool.execute?.("call-1", { action: "snapshot", snapshotFormat: "ai" });

1251+

const text = firstResultText(result);

1252+

expect(text).toContain('"blockedByDialog": true');

1253+

expect(text).toContain('"id": "d1"');

1254+

const details = externalContentDetails(result, "snapshot") as {

1255+

blockedByDialog?: unknown;

1256+

browserState?: { dialogs?: { pending?: Array<{ id?: string }> } };

1257+

};

1258+

expect(details.blockedByDialog).toBe(true);

1259+

expect(details.browserState?.dialogs?.pending?.[0]?.id).toBe("d1");

1260+

});

1261+
12331262

it("wraps tabs output as external content", async () => {

12341263

browserClientMocks.browserTabs.mockResolvedValueOnce([

12351264

{

Original file line numberDiff line numberDiff line change

@@ -867,6 +867,7 @@ export function createBrowserTool(opts?: {

867867

case "dialog": {

868868

const accept = Boolean(params.accept);

869869

const promptText = readStringValue(params.promptText);

870+

const dialogId = readStringValue(params.dialogId);

870871

const { targetId, timeoutMs } = readOptionalTargetAndTimeout(params);

871872

if (proxyRequest) {

872873

const result = await proxyRequest({

@@ -876,6 +877,7 @@ export function createBrowserTool(opts?: {

876877

body: {

877878

accept,

878879

promptText,

880+

dialogId,

879881

targetId,

880882

timeoutMs,

881883

},

@@ -885,6 +887,7 @@ export function createBrowserTool(opts?: {

885887

const result = await browserToolDeps.browserArmDialog(baseUrl, {

886888

accept,

887889

promptText,

890+

dialogId,

888891

targetId,

889892

timeoutMs,

890893

profile,

Original file line numberDiff line numberDiff line change

@@ -19,6 +19,8 @@ type BrowserActResponse = {

1919

url?: string;

2020

result?: unknown;

2121

results?: Array<{ ok: boolean; error?: string }>;

22+

blockedByDialog?: boolean;

23+

browserState?: unknown;

2224

};

2325
2426

const BROWSER_ACT_REQUEST_TIMEOUT_SLACK_MS = 5_000;

@@ -66,6 +68,7 @@ export async function browserArmDialog(

6668

opts: {

6769

accept: boolean;

6870

promptText?: string;

71+

dialogId?: string;

6972

targetId?: string;

7073

timeoutMs?: number;

7174

profile?: string;

@@ -78,6 +81,7 @@ export async function browserArmDialog(

7881

body: JSON.stringify({

7982

accept: opts.accept,

8083

promptText: opts.promptText,

84+

dialogId: opts.dialogId,

8185

targetId: opts.targetId,

8286

timeoutMs: opts.timeoutMs,

8387

}),

Original file line numberDiff line numberDiff line change

@@ -44,6 +44,8 @@ export type SnapshotResult =

4444

targetId: string;

4545

url: string;

4646

nodes: SnapshotAriaNode[];

47+

blockedByDialog?: boolean;

48+

browserState?: unknown;

4749

}

4850

| {

4951

ok: true;

@@ -64,6 +66,8 @@ export type SnapshotResult =

6466

labelsSkipped?: number;

6567

imagePath?: string;

6668

imageType?: "png" | "jpeg";

69+

blockedByDialog?: boolean;

70+

browserState?: unknown;

6771

};

6872
6973

export async function browserStatus(