惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

V
Visual Studio Blog
Stack Overflow Blog
Stack Overflow Blog
G
Google Developers Blog
Microsoft Azure Blog
Microsoft Azure Blog
Engineering at Meta
Engineering at Meta
L
LangChain Blog
T
The Blog of Author Tim Ferriss
J
Java Code Geeks
Y
Y Combinator Blog
月光博客
月光博客
雷峰网
雷峰网
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
F
Fortinet All Blogs
A
About on SuperTechFans
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
P
Proofpoint News Feed
小众软件
小众软件
H
Help Net Security
Last Week in AI
Last Week in AI
B
Blog RSS Feed
宝玉的分享
宝玉的分享
N
Netflix TechBlog - Medium
博客园 - 叶小钗
The GitHub Blog
The GitHub Blog

Recent Commits to openclaw:main

test: merge chat side-result checks · openclaw/openclaw@ddd2c2a test: merge cron history checks · openclaw/openclaw@f7eb746 test: merge responsive navigation shell checks · openclaw/openclaw@c2e4b47 docs(changelog): add codex oauth fixes · openclaw/openclaw@628e6cd test: merge navigation routing cases · openclaw/openclaw@5d8cecb Tests: mock channel registry bundled fallback · openclaw/openclaw@2b08233 Secrets: avoid broad web search discovery for single plugin config · openclaw/openclaw@a464f59 test: merge config view browser checks · openclaw/openclaw@20cf511 fix(status): align oauth health with runtime · openclaw/openclaw@eed7116 feat: add macOS screen snapshots for monitor preview (#67954) thanks … · openclaw/openclaw@f377db1 fix: report shared auth scopes in hello-ok (#67810) thanks @BunsDev · openclaw/openclaw@0b6c39b Auto-reply: avoid eager bundled route fallback · openclaw/openclaw@3ea1bf4 Tests: narrow session binding contract setup · openclaw/openclaw@54e4e16 fix(macOS): enable undo/redo in webchat composer text input (#34962) · openclaw/openclaw@00951dc Tests: speed up channel setup promotion · openclaw/openclaw@82b529a Docs: refresh agent instructions · openclaw/openclaw@5775fe2 fix(auth): serialize OAuth refresh across agents to fix #26322 (#67876) · openclaw/openclaw@8e79080 test: allow ollama public surface boundary test · openclaw/openclaw@7d4f1a6 Docs: add test performance guardrails · openclaw/openclaw@89706d3 Tests: restore context-engine usage proof · openclaw/openclaw@e4c4f95 Tests: slim context engine runtime coverage · openclaw/openclaw@74c198f ci: retry failed custom checkouts · openclaw/openclaw@0ee5baf test: trim duplicate provider auth onboarding cases · openclaw/openclaw@1ffc02e matrix: fix sessions_spawn --thread subagent session spawning (#67643) · openclaw/openclaw@1ce2596 test: reduce auth choice fixture churn · openclaw/openclaw@857b9cd test: mock health status config boundaries · openclaw/openclaw@9d5ab4a test: mock onboard config io boundary · openclaw/openclaw@299694d test: mock legacy state plugin boundaries · openclaw/openclaw@2713089 test: mock channel install boundaries · openclaw/openclaw@b945248 test: mock doctor preview channel boundaries · openclaw/openclaw@b1a3ad4
fix(nodes): return screen snapshots as media (#93499) · o...
zenglingbiao · 2026-06-16 · via Recent Commits to openclaw:main

@@ -38,6 +38,15 @@ const screenMocks = vi.hoisted(() => ({

3838

})),

3939

screenRecordTempPath: vi.fn(() => "/tmp/screen-record.mp4"),

4040

writeScreenRecordToFile: vi.fn(async () => ({ path: "/tmp/screen-record.mp4" })),

41+

parseScreenSnapshotPayload: vi.fn(() => ({

42+

base64: "ZmFrZQ==",

43+

format: "png",

44+

screenIndex: 0,

45+

width: 1920,

46+

height: 1080,

47+

})),

48+

screenSnapshotTempPath: vi.fn(() => "/tmp/screen-snapshot.png"),

49+

writeScreenSnapshotToFile: vi.fn(async () => ({ path: "/tmp/screen-snapshot.png" })),

4150

}));

42514352

vi.mock("./gateway.js", () => ({

@@ -62,6 +71,9 @@ vi.mock("../../cli/nodes-screen.js", () => ({

6271

parseScreenRecordPayload: screenMocks.parseScreenRecordPayload,

6372

screenRecordTempPath: screenMocks.screenRecordTempPath,

6473

writeScreenRecordToFile: screenMocks.writeScreenRecordToFile,

74+

parseScreenSnapshotPayload: screenMocks.parseScreenSnapshotPayload,

75+

screenSnapshotTempPath: screenMocks.screenSnapshotTempPath,

76+

writeScreenSnapshotToFile: screenMocks.writeScreenSnapshotToFile,

6577

}));

66786779

let createNodesTool: typeof import("./nodes-tool.js").createNodesTool;

@@ -123,6 +135,9 @@ describe("createNodesTool screen_record duration guardrails", () => {

123135

nodeUtilsMocks.resolveNode.mockClear();

124136

screenMocks.parseScreenRecordPayload.mockClear();

125137

screenMocks.writeScreenRecordToFile.mockClear();

138+

screenMocks.parseScreenSnapshotPayload.mockClear();

139+

screenMocks.screenSnapshotTempPath.mockClear();

140+

screenMocks.writeScreenSnapshotToFile.mockClear();

126141

nodesCameraMocks.cameraTempPath.mockClear();

127142

nodesCameraMocks.parseCameraSnapPayload.mockClear();

128143

nodesCameraMocks.writeCameraPayloadToFile.mockClear();

@@ -258,6 +273,69 @@ describe("createNodesTool screen_record duration guardrails", () => {

258273

expect(gatewayMocks.callGatewayTool).not.toHaveBeenCalled();

259274

});

260275276+

it("invokes screen.snapshot with validated params and returns file details", async () => {

277+

gatewayMocks.callGatewayTool.mockResolvedValue({ payload: { ok: true } });

278+

const tool = createNodesTool();

279+280+

const result = await tool.execute("call-snapshot", {

281+

action: "screen_snapshot",

282+

node: "macbook",

283+

screenIndex: 1,

284+

maxWidth: "1200",

285+

});

286+287+

expect(gatewayMocks.callGatewayTool).toHaveBeenCalledTimes(1);

288+

const call = gatewayMocks.callGatewayTool.mock.calls[0] as

289+

| [

290+

string,

291+

unknown,

292+

{ command?: string; params?: { screenIndex?: unknown; maxWidth?: unknown } },

293+

]

294+

| undefined;

295+

expect(call?.[0]).toBe("node.invoke");

296+

expect(call?.[2].command).toBe("screen.snapshot");

297+

expect(call?.[2].params).toEqual({ screenIndex: 1, maxWidth: 1200 });

298+

expect(screenMocks.parseScreenSnapshotPayload).toHaveBeenCalledWith({ ok: true });

299+

expect(screenMocks.screenSnapshotTempPath).toHaveBeenCalledWith({ ext: "png" });

300+

expect(screenMocks.writeScreenSnapshotToFile).toHaveBeenCalledWith(

301+

"/tmp/screen-snapshot.png",

302+

"ZmFrZQ==",

303+

);

304+

expect(result).toEqual({

305+

content: [{ type: "text", text: "FILE:/tmp/screen-snapshot.png" }],

306+

details: {

307+

path: "/tmp/screen-snapshot.png",

308+

format: "png",

309+

screenIndex: 0,

310+

width: 1920,

311+

height: 1080,

312+

media: {

313+

mediaUrl: "/tmp/screen-snapshot.png",

314+

},

315+

},

316+

});

317+

});

318+319+

it("rejects unsupported screen.snapshot response formats before writing", async () => {

320+

gatewayMocks.callGatewayTool.mockResolvedValue({ payload: { ok: true } });

321+

screenMocks.parseScreenSnapshotPayload.mockReturnValueOnce({

322+

base64: "ZmFrZQ==",

323+

format: "webp",

324+

screenIndex: 0,

325+

width: 1920,

326+

height: 1080,

327+

});

328+

const tool = createNodesTool();

329+330+

await expect(

331+

tool.execute("call-snapshot", {

332+

action: "screen_snapshot",

333+

node: "macbook",

334+

}),

335+

).rejects.toThrow("unsupported screen.snapshot format: webp");

336+

expect(screenMocks.writeScreenSnapshotToFile).not.toHaveBeenCalled();

337+

});

338+261339

it("rejects the removed run action", async () => {

262340

const tool = createNodesTool();

263341

@@ -387,6 +465,8 @@ describe("createNodesTool screen_record duration guardrails", () => {

387465

["photos_latest", { quality: -0.1 }, "quality must be between 0 and 1"],

388466

["screen_record", { fps: 0 }, "fps must be greater than 0"],

389467

["screen_record", { screenIndex: 1.5 }, "screenIndex must be a non-negative integer"],

468+

["screen_snapshot", { maxWidth: 0 }, "maxWidth must be a positive integer"],

469+

["screen_snapshot", { screenIndex: -1 }, "screenIndex must be a non-negative integer"],

390470

])("rejects invalid %s numeric params %s", async (action, params, message) => {

391471

const tool = createNodesTool();

392472

@@ -561,6 +641,38 @@ describe("createNodesTool screen_record duration guardrails", () => {

561641

);

562642

});

563643644+

it("blocks raw screen.snapshot invoke to prevent base64 context bloat", async () => {

645+

const tool = createNodesTool();

646+647+

await expect(

648+

tool.execute("call-1", {

649+

action: "invoke",

650+

node: "macbook",

651+

invokeCommand: "screen.snapshot",

652+

}),

653+

).rejects.toThrow('use action="screen_snapshot"');

654+

expect(gatewayMocks.callGatewayTool).not.toHaveBeenCalled();

655+

});

656+657+

it("preserves explicitly enabled raw screen.snapshot invoke", async () => {

658+

gatewayMocks.callGatewayTool.mockResolvedValue({

659+

payload: { format: "png", base64: "ZmFrZQ==" },

660+

});

661+

const tool = createNodesTool({ allowMediaInvokeCommands: true });

662+663+

await tool.execute("call-1", {

664+

action: "invoke",

665+

node: "macbook",

666+

invokeCommand: "screen.snapshot",

667+

});

668+669+

expect(gatewayMocks.callGatewayTool).toHaveBeenCalledWith(

670+

"node.invoke",

671+

{},

672+

expect.objectContaining({ command: "screen.snapshot" }),

673+

);

674+

});

675+564676

it("keeps invoke pairing guidance for scope upgrade rejections", async () => {

565677

gatewayMocks.callGatewayTool.mockRejectedValueOnce(

566678

new Error("scope upgrade pending approval (requestId: req-123)"),