docs: document OpenAI realtime voices · openclaw/openclaw@b75e5c5
steipete
·
2026-05-08
·
via Recent Commits to openclaw:main
| Original file line number | Diff line number | Diff line change |
|---|
@@ -1184,7 +1184,10 @@ Auto-join example:
|
1184 | 1184 | reconnectGraceMs: 15000, |
1185 | 1185 | tts: { |
1186 | 1186 | provider: "openai", |
1187 | | - openai: { voice: "onyx" }, |
| 1187 | + openai: { |
| 1188 | + model: "gpt-4o-mini-tts", |
| 1189 | + voice: "cedar", |
| 1190 | + }, |
1188 | 1191 | }, |
1189 | 1192 | }, |
1190 | 1193 | }, |
@@ -1195,8 +1198,9 @@ Auto-join example:
|
1195 | 1198 | Notes: |
1196 | 1199 | |
1197 | 1200 | - `voice.tts` overrides `messages.tts` for voice playback only. |
1198 | | -- `voice.model` overrides the LLM used for Discord voice channel responses only. Leave it unset to inherit the routed agent model. |
| 1201 | +- `voice.model` overrides the LLM used for Discord voice channel responses only. Leave it unset to inherit the routed agent model. Do not set this to `gpt-realtime-2`; Discord voice channels use STT plus TTS playback, not the OpenAI Realtime session transport. |
1199 | 1202 | - STT uses `tools.media.audio`; `voice.model` does not affect transcription. |
| 1203 | +- For an OpenAI voice on Discord playback, set `voice.tts.provider: "openai"` and choose a Text-to-speech voice under `voice.tts.openai.voice` or `voice.tts.providers.openai.voice`. `cedar` is a good masculine-sounding choice on the current OpenAI TTS model. |
1200 | 1204 | - Per-channel Discord `systemPrompt` overrides apply to voice transcript turns for that voice channel. |
1201 | 1205 | - Voice transcript turns derive owner status from Discord `allowFrom` (or `dm.allowFrom`); non-owner speakers cannot access owner-only tools (for example `gateway` and `cron`). |
1202 | 1206 | - Discord voice is opt-in for text-only configs; set `channels.discord.voice.enabled=true` (or keep an existing `channels.discord.voice` block) to enable `/vc` commands, the voice runtime, and the `GuildVoiceStates` gateway intent. |
|
| Original file line number | Diff line number | Diff line change |
|---|
@@ -1389,7 +1389,7 @@ Defaults for Talk mode (macOS/iOS/Android).
|
1389 | 1389 | providers: { |
1390 | 1390 | openai: { |
1391 | 1391 | model: "gpt-realtime-2", |
1392 | | - voice: "alloy", |
| 1392 | + voice: "cedar", |
1393 | 1393 | }, |
1394 | 1394 | }, |
1395 | 1395 | mode: "realtime", |
|
| Original file line number | Diff line number | Diff line change |
|---|
@@ -82,7 +82,7 @@ Supported keys:
|
82 | 82 | openai: { |
83 | 83 | apiKey: "openai_api_key", |
84 | 84 | model: "gpt-realtime-2", |
85 | | - voice: "alloy", |
| 85 | + voice: "cedar", |
86 | 86 | }, |
87 | 87 | }, |
88 | 88 | mode: "realtime", |
@@ -104,6 +104,7 @@ Defaults:
|
104 | 104 | - `providers.elevenlabs.apiKey`: falls back to `ELEVENLABS_API_KEY` (or gateway shell profile if available). |
105 | 105 | - `realtime.provider`: selects the active browser/server realtime voice provider. Use `openai` for WebRTC, `google` for provider WebSocket, or a bridge-only provider through Gateway relay. |
106 | 106 | - `realtime.providers.<provider>` stores provider-owned realtime config. The browser receives only ephemeral or constrained session credentials, never a standard API key. |
| 107 | +- `realtime.providers.openai.voice`: built-in OpenAI Realtime voice id. Current `gpt-realtime-2` voices are `alloy`, `ash`, `ballad`, `coral`, `echo`, `sage`, `shimmer`, `verse`, `marin`, and `cedar`; `marin` and `cedar` are recommended for best quality. |
107 | 108 | - `realtime.brain`: `agent-consult` routes realtime tool calls through Gateway policy; `direct-tools` is owner-only compatibility behavior; `none` is for transcription or external orchestration. |
108 | 109 | - `talk.catalog` exposes each provider's valid modes, transports, brain strategies, realtime audio formats, and capability flags so first-party Talk clients can avoid unsupported combinations. |
109 | 110 | - Streaming transcription providers are discovered through `talk.catalog.transcription`. The current Gateway relay uses the Voice Call streaming provider config until the dedicated Talk transcription config surface is added. |
|
| Original file line number | Diff line number | Diff line change |
|---|
@@ -648,10 +648,23 @@ Legacy `plugins.entries.openai.config.personality` is still read as a compatibil
|
648 | 648 | | Silence duration | `...openai.silenceDurationMs` | `500` | |
649 | 649 | | API key | `...openai.apiKey` | Falls back to `OPENAI_API_KEY` | |
650 | 650 | |
| 651 | +Available built-in Realtime voices for `gpt-realtime-2`: `alloy`, `ash`, |
| 652 | +`ballad`, `coral`, `echo`, `sage`, `shimmer`, `verse`, `marin`, `cedar`. |
| 653 | +OpenAI recommends `marin` and `cedar` for the best Realtime quality. This |
| 654 | +is a separate set from the Text-to-speech voices above; do not assume a TTS |
| 655 | +voice such as `fable`, `nova`, or `onyx` is valid for Realtime sessions. |
| 656 | + |
651 | 657 | <Note> |
652 | 658 | Backend OpenAI realtime bridges use the GA Realtime WebSocket session shape, which does not accept `session.temperature`. Azure OpenAI deployments remain available via `azureEndpoint` and `azureDeployment` and keep the deployment-compatible session shape. Supports bidirectional tool calling and G.711 u-law audio. |
653 | 659 | </Note> |
654 | 660 | |
| 661 | +<Note> |
| 662 | +Realtime voice is selected when the session is created. OpenAI allows most |
| 663 | +session fields to change later, but the voice cannot be changed after the |
| 664 | +model has emitted audio in that session. OpenClaw currently exposes the |
| 665 | +built-in Realtime voice ids as strings. |
| 666 | +</Note> |
| 667 | + |
655 | 668 | <Note> |
656 | 669 | Control UI Talk uses OpenAI browser realtime sessions with a Gateway-minted |
657 | 670 | ephemeral client secret and a direct browser WebRTC SDP exchange against the |
|
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。