@@ -18,32 +18,35 @@ OpenClaw generates images, videos, and music, understands inbound media (images,
1818| Image generation | `image_generate` | ComfyUI, fal, Google, MiniMax, OpenAI, Vydra, xAI | Creates or edits images from text prompts or references |
1919| Video generation | `video_generate` | Alibaba, BytePlus, ComfyUI, fal, Google, MiniMax, OpenAI, Qwen, Runway, Together, Vydra, xAI | Creates videos from text, images, or existing videos |
2020| Music generation | `music_generate` | ComfyUI, Google, MiniMax | Creates music or audio tracks from text prompts |
21-| Text-to-speech (TTS) | `tts` | ElevenLabs, Google, Gradium, Microsoft, MiniMax, OpenAI, Vydra, xAI | Converts outbound replies to spoken audio |
21+| Text-to-speech (TTS) | `tts` | ElevenLabs, Google, Gradium, Local CLI, Microsoft, MiniMax, OpenAI, Vydra, xAI, Xiaomi MiMo | Converts outbound replies to spoken audio |
2222| Media understanding | (automatic) | Any vision/audio-capable model provider, plus CLI fallbacks | Summarizes inbound images, audio, and video |
23232424## Provider capability matrix
25252626This table shows which providers support which media capabilities across the platform.
272728-| Provider | Image | Video | Music | TTS | STT / Transcription | Realtime Voice | Media Understanding |
29-| ---------- | ----- | ----- | ----- | --- | ------------------- | -------------- | ------------------- |
30-| Alibaba | | Yes | | | | | |
31-| BytePlus | | Yes | | | | | |
32-| ComfyUI | Yes | Yes | Yes | | | | |
33-| Deepgram | | | | | Yes | | |
34-| ElevenLabs | | | | Yes | Yes | | |
35-| fal | Yes | Yes | | | | | |
36-| Google | Yes | Yes | Yes | Yes | | Yes | Yes |
37-| Gradium | | | | Yes | | | |
38-| Microsoft | | | | Yes | | | |
39-| MiniMax | Yes | Yes | Yes | Yes | | | |
40-| Mistral | | | | | Yes | | |
41-| OpenAI | Yes | Yes | | Yes | Yes | Yes | Yes |
42-| Qwen | | Yes | | | | | |
43-| Runway | | Yes | | | | | |
44-| Together | | Yes | | | | | |
45-| Vydra | Yes | Yes | | Yes | | | |
46-| xAI | Yes | Yes | | Yes | Yes | | Yes |
28+| Provider | Image | Video | Music | TTS | STT / Transcription | Realtime Voice | Media Understanding |
29+| ----------- | ----- | ----- | ----- | --- | ------------------- | -------------- | ------------------- |
30+| Alibaba | | Yes | | | | | |
31+| BytePlus | | Yes | | | | | |
32+| ComfyUI | Yes | Yes | Yes | | | | |
33+| Deepgram | | | | | Yes | Yes | |
34+| ElevenLabs | | | | Yes | Yes | | |
35+| fal | Yes | Yes | | | | | |
36+| Google | Yes | Yes | Yes | Yes | | Yes | Yes |
37+| Gradium | | | | Yes | | | |
38+| Local CLI | | | | Yes | | | |
39+| Microsoft | | | | Yes | | | |
40+| MiniMax | Yes | Yes | Yes | Yes | | | |
41+| Mistral | | | | | Yes | | |
42+| OpenAI | Yes | Yes | | Yes | Yes | Yes | Yes |
43+| Qwen | | Yes | | | | | |
44+| Runway | | Yes | | | | | |
45+| SenseAudio | | | | | Yes | | |
46+| Together | | Yes | | | | | |
47+| Vydra | Yes | Yes | | Yes | | | |
48+| xAI | Yes | Yes | | Yes | Yes | | Yes |
49+| Xiaomi MiMo | Yes | | | Yes | | | Yes |
47504851<Note>
4952Media understanding uses any vision-capable or audio-capable model registered in your provider config. The table above highlights providers with dedicated media-understanding support; most LLM providers with multimodal models (Anthropic, Google, OpenAI, etc.) can also understand inbound media when configured as the active reply model.
@@ -53,11 +56,11 @@ Media understanding uses any vision-capable or audio-capable model registered in
53565457Video and music generation run as background tasks because provider processing typically takes 30 seconds to several minutes. When the agent calls `video_generate` or `music_generate`, OpenClaw submits the request to the provider, returns a task ID immediately, and tracks the job in the task ledger. The agent continues responding to other messages while the job runs. When the provider finishes, OpenClaw wakes the agent so it can post the finished media back into the original channel. Image generation and TTS are synchronous and complete inline with the reply.
555856-Deepgram, ElevenLabs, Mistral, OpenAI, and xAI can all transcribe inbound
57-audio through the batch `tools.media.audio` path when configured. Deepgram,
58-ElevenLabs, Mistral, OpenAI, and xAI also register Voice Call streaming STT
59-providers, so live phone audio can be forwarded to the selected vendor
60-without waiting for a completed recording.
59+Deepgram, ElevenLabs, Mistral, OpenAI, SenseAudio, and xAI can all transcribe
60+inbound audio through the batch `tools.media.audio` path when configured.
61+Deepgram, ElevenLabs, Mistral, OpenAI, and xAI also register Voice Call
62+streaming STT providers, so live phone audio can be forwarded to the selected
63+vendor without waiting for a completed recording.
61646265Google maps to OpenClaw's image, video, music, batch TTS, backend realtime
6366voice, and media-understanding surfaces. OpenAI maps to OpenClaw's image,
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。