惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

WordPress大学
WordPress大学
T
The Blog of Author Tim Ferriss
F
Fortinet All Blogs
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
阮一峰的网络日志
阮一峰的网络日志
The GitHub Blog
The GitHub Blog
Y
Y Combinator Blog
MyScale Blog
MyScale Blog
雷峰网
雷峰网
博客园 - 叶小钗
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
GbyAI
GbyAI
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
博客园 - 三生石上(FineUI控件)
云风的 BLOG
云风的 BLOG
V
V2EX
宝玉的分享
宝玉的分享
酷 壳 – CoolShell
酷 壳 – CoolShell
N
Netflix TechBlog - Medium
Vercel News
Vercel News
美团技术团队
人人都是产品经理
人人都是产品经理
The Cloudflare Blog

Recent Commits to openclaw:main

test: merge chat side-result checks · openclaw/openclaw@ddd2c2a test: merge cron history checks · openclaw/openclaw@f7eb746 test: merge responsive navigation shell checks · openclaw/openclaw@c2e4b47 docs(changelog): add codex oauth fixes · openclaw/openclaw@628e6cd test: merge navigation routing cases · openclaw/openclaw@5d8cecb Tests: mock channel registry bundled fallback · openclaw/openclaw@2b08233 Secrets: avoid broad web search discovery for single plugin config · openclaw/openclaw@a464f59 test: merge config view browser checks · openclaw/openclaw@20cf511 fix(status): align oauth health with runtime · openclaw/openclaw@eed7116 feat: add macOS screen snapshots for monitor preview (#67954) thanks … · openclaw/openclaw@f377db1 fix: report shared auth scopes in hello-ok (#67810) thanks @BunsDev · openclaw/openclaw@0b6c39b Auto-reply: avoid eager bundled route fallback · openclaw/openclaw@3ea1bf4 Tests: narrow session binding contract setup · openclaw/openclaw@54e4e16 fix(macOS): enable undo/redo in webchat composer text input (#34962) · openclaw/openclaw@00951dc Tests: speed up channel setup promotion · openclaw/openclaw@82b529a Docs: refresh agent instructions · openclaw/openclaw@5775fe2 fix(auth): serialize OAuth refresh across agents to fix #26322 (#67876) · openclaw/openclaw@8e79080 test: allow ollama public surface boundary test · openclaw/openclaw@7d4f1a6 Docs: add test performance guardrails · openclaw/openclaw@89706d3 Tests: restore context-engine usage proof · openclaw/openclaw@e4c4f95 Tests: slim context engine runtime coverage · openclaw/openclaw@74c198f ci: retry failed custom checkouts · openclaw/openclaw@0ee5baf test: trim duplicate provider auth onboarding cases · openclaw/openclaw@1ffc02e matrix: fix sessions_spawn --thread subagent session spawning (#67643) · openclaw/openclaw@1ce2596 test: reduce auth choice fixture churn · openclaw/openclaw@857b9cd test: mock health status config boundaries · openclaw/openclaw@9d5ab4a test: mock onboard config io boundary · openclaw/openclaw@299694d test: mock legacy state plugin boundaries · openclaw/openclaw@2713089 test: mock channel install boundaries · openclaw/openclaw@b945248 test: mock doctor preview channel boundaries · openclaw/openclaw@b1a3ad4
docs: detail talk refactor plan · openclaw/openclaw@7431cb8
steipete · 2026-05-06 · via Recent Commits to openclaw:main

@@ -87,20 +87,19 @@ event history, and stale-turn rejection. Provider plugins should keep owning

8787

vendor-specific realtime sessions; surface plugins should keep owning capture,

8888

playback, telephony, and meeting quirks.

898990-

This migration is intentionally adapter-first:

91-92-

1. Add shared controller/runtime primitives to `plugin-sdk/realtime-voice`.

93-

2. Keep existing public Gateway RPCs such as `talk.realtime.session`,

94-

`talk.realtime.relayAudio`, `talk.transcription.session`, and

95-

`talk.handoff.*` as compatibility adapters.

96-

3. Move bundled surfaces onto the shared controller: browser relay, managed-room

97-

handoff, voice-call realtime, voice-call streaming STT, Google Meet realtime,

98-

and VoiceClaw realtime.

99-

4. Advertise all Talk event channels in Gateway `hello-ok.features.events` so

100-

clients can discover `talk.event`, `talk.realtime.relay`, and

101-

`talk.transcription.relay`.

102-

5. Expose the versioned `talk.session.*` API for Gateway-managed Talk sessions

103-

after the adapters are internally backed by the same controller.

90+

This Talk migration is intentionally breaking-clean:

91+92+

1. Keep the shared controller/runtime primitives in

93+

`plugin-sdk/realtime-voice`.

94+

2. Move bundled surfaces onto the shared controller: browser relay,

95+

managed-room handoff, voice-call realtime, voice-call streaming STT, Google

96+

Meet realtime, and native push-to-talk.

97+

3. Replace old Talk RPC families with the final `talk.session.*` and

98+

`talk.client.*` API.

99+

4. Advertise one live Talk event channel in Gateway

100+

`hello-ok.features.events`: `talk.event`.

101+

5. Delete the old realtime HTTP endpoint and any request-time instruction

102+

override path.

104103105104

New code should not call `createTalkEventSequencer(...)` directly unless it is

106105

implementing a low-level adapter or test fixture. Prefer the shared controller

@@ -112,24 +111,33 @@ handoff, and native Talk clients.

112111

The target public API shape is:

113112114113

```typescript

115-

// Versioned Gateway-managed Talk session API.

114+

// Gateway-owned Talk session API.

116115

await gateway.request("talk.session.create", {

117116

mode: "realtime",

118117

transport: "gateway-relay",

119118

brain: "agent-consult",

120119

sessionKey: "main",

121120

});

122-

await gateway.request("talk.session.inputAudio", { sessionId, audioBase64 });

123-

await gateway.request("talk.session.control", { sessionId, type: "turn.cancel" });

124-

await gateway.request("talk.session.toolResult", { sessionId, callId, result });

121+

await gateway.request("talk.session.appendAudio", { sessionId, audioBase64 });

122+

await gateway.request("talk.session.cancelOutput", { sessionId, reason: "barge-in" });

123+

await gateway.request("talk.session.submitToolResult", { sessionId, callId, result });

125124

await gateway.request("talk.session.close", { sessionId });

125+126+

// Client-owned provider session API.

127+

await gateway.request("talk.client.create", {

128+

mode: "realtime",

129+

transport: "webrtc",

130+

brain: "agent-consult",

131+

sessionKey: "main",

132+

});

133+

await gateway.request("talk.client.toolCall", { sessionKey, callId, name, args });

126134

```

127135128-

Browser-owned WebRTC/provider-websocket sessions stay on

129-

`talk.realtime.session`, because the browser owns the provider negotiation and

130-

media transport. `talk.session.*` is the common Gateway-managed surface for

131-

gateway-relay realtime, gateway-relay transcription, and managed-room native

132-

STT/TTS sessions.

136+

Browser-owned WebRTC/provider-websocket sessions use `talk.client.create`,

137+

because the browser owns the provider negotiation and media transport while the

138+

Gateway owns credentials, instructions, and tool policy. `talk.session.*` is the

139+

common Gateway-managed surface for gateway-relay realtime, gateway-relay

140+

transcription, and managed-room native STT/TTS sessions.

133141134142

Legacy configs that placed realtime selectors beside `talk.provider` /

135143

`talk.providers` should be repaired with `openclaw doctor --fix`; runtime Talk

@@ -144,30 +152,43 @@ The supported `talk.session.create` combinations are intentionally small:

144152

| `stt-tts` | `managed-room` | `agent-consult` | Native/client room | Push-to-talk and walkie-talkie style rooms where the client owns capture/playback and the Gateway owns turn state. |

145153

| `stt-tts` | `managed-room` | `direct-tools` | Native/client room | Admin-only room mode for trusted first-party surfaces that execute Gateway tool actions directly. |

146154147-

Everything else should stay on the existing owner-specific adapter until there

148-

is a real Gateway-managed transport for it:

149-150-

| Existing adapter | Keep using it for |

151-

| ----------------------- | ---------------------------------------------------------------------------------------- |

152-

| `talk.realtime.session` | Browser-owned WebRTC and provider-websocket realtime sessions. |

153-

| `talk.realtime.relay*` | Compatibility for existing browser relay clients while they migrate to `talk.session.*`. |

154-

| `talk.transcription.*` | Compatibility for existing streaming STT clients while they migrate to `talk.session.*`. |

155-

| `talk.handoff.*` | Compatibility for room-style native clients; internally this is the managed-room shape. |

155+

Removed method map:

156+157+

| Old | New |

158+

| -------------------------------- | -------------------------------------------------------- |

159+

| `talk.realtime.session` | `talk.client.create` |

160+

| `talk.realtime.toolCall` | `talk.client.toolCall` |

161+

| `talk.realtime.relayAudio` | `talk.session.appendAudio` |

162+

| `talk.realtime.relayCancel` | `talk.session.cancelOutput` or `talk.session.cancelTurn` |

163+

| `talk.realtime.relayToolResult` | `talk.session.submitToolResult` |

164+

| `talk.realtime.relayStop` | `talk.session.close` |

165+

| `talk.transcription.session` | `talk.session.create({ mode: "transcription" })` |

166+

| `talk.transcription.relayAudio` | `talk.session.appendAudio` |

167+

| `talk.transcription.relayCancel` | `talk.session.cancelTurn` |

168+

| `talk.transcription.relayStop` | `talk.session.close` |

169+

| `talk.handoff.create` | `talk.session.create({ transport: "managed-room" })` |

170+

| `talk.handoff.join` | `talk.session.join` |

171+

| `talk.handoff.revoke` | `talk.session.close` |

156172157173

The unified control vocabulary is also deliberately narrow:

158174159-

| Method | Applies to | Contract |

160-

| ------------------------- | ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- |

161-

| `talk.session.inputAudio` | `realtime/gateway-relay`, `transcription/gateway-relay` | Append a base64 PCM audio chunk to the provider session owned by the same Gateway connection. |

162-

| `talk.session.control` | all unified sessions | `turn.cancel` for relay sessions; `turn.start`, `turn.end`, and `turn.cancel` for managed-room sessions. |

163-

| `talk.session.toolResult` | `realtime/gateway-relay` | Complete a provider tool call emitted by the relay. |

164-

| `talk.session.close` | all unified sessions | Stop relay sessions or revoke managed-room handoff state, then forget the unified session id. |

175+

| Method | Applies to | Contract |

176+

| ------------------------------- | ------------------------------------------------------- | --------------------------------------------------------------------------------------------- |

177+

| `talk.session.appendAudio` | `realtime/gateway-relay`, `transcription/gateway-relay` | Append a base64 PCM audio chunk to the provider session owned by the same Gateway connection. |

178+

| `talk.session.startTurn` | `stt-tts/managed-room` | Start a managed-room user turn. |

179+

| `talk.session.endTurn` | `stt-tts/managed-room` | End the active turn after stale-turn validation. |

180+

| `talk.session.cancelTurn` | all Gateway-owned sessions | Cancel active capture/provider/agent/TTS work for a turn. |

181+

| `talk.session.cancelOutput` | `realtime/gateway-relay` | Stop assistant audio output without necessarily ending the user turn. |

182+

| `talk.session.submitToolResult` | `realtime/gateway-relay` | Complete a provider tool call emitted by the relay. |

183+

| `talk.session.close` | all unified sessions | Stop relay sessions or revoke managed-room state, then forget the unified session id. |

165184166185

Do not introduce provider or platform special cases in core to make this work.

167186

Core owns Talk session semantics. Provider plugins own vendor session setup.

168187

Voice-call and Google Meet own telephony/meeting adapters. Browser and native

169188

apps own device capture/playback UX.

170189190+

The detailed implementation plan lives in [Talk refactor plan](/refactor/talk).

191+171192

## Compatibility policy

172193173194

For external plugins, compatibility work follows this order: