by Alem Tuzlak on Apr 24, 2026.

The AI audio ecosystem is a mess. Gemini's Lyria wants a natural-language prompt and returns raw PCM you have to wrap in a RIFF header yourself. Fal hosts dozens of audio models where one wants music_length_ms in milliseconds, the next wants seconds_total, and most want plain duration. ElevenLabs has its own shape. Whisper has another. Every provider disagrees on whether you get a URL, a base64 blob, or a raw buffer.

If you are shipping an AI product that needs music, sound effects, speech, or transcription, you end up writing the same boring glue code five times.

TanStack AI just removed that glue. The latest release lands a full audio stack: a new generateAudio activity, streaming support, fal and Gemini Lyria adapters, and framework hooks for React, Solid, Vue, and Svelte. One typed API, any provider.

Here is what shipped and why you should care.

The new generateAudio() activity sits alongside generateImage, generateSpeech, generateVideo, and generateTranscription in @tanstack/ai. It takes a text prompt, dispatches to whatever adapter you hand it, and returns a GeneratedAudio object with exactly one of url or b64Json.

Swap geminiAudio for falAudio and the exact same call generates music through MiniMax, DiffRhythm, Stable Audio 2.5, or any of the other models in fal's catalog. The adapter translates per-model details (like fal's music_length_ms vs seconds_total vs duration naming) so your app code never sees them.

Music and SFX generation is slow. Lyria 3 Pro takes several seconds. Stable Audio takes longer. If you are building a UI, blocking the request the whole time is a bad experience.

generateAudio now supports stream: true, returning an AsyncIterable<StreamChunk> you can pipe straight through toServerSentEventsResponse():

The client receives progress events and the final audio over a single SSE connection, the same transport model already used by generateImage and generateVideo. No new infrastructure, no special-case code paths.

Every framework integration gets a new hook matching the existing media-hook shape:

• @tanstack/ai-react: useGenerateAudio
• @tanstack/ai-solid: useGenerateAudio
• @tanstack/ai-vue: useGenerateAudio
• @tanstack/ai-svelte: createGenerateAudio

The API is identical to useGenerateImage and friends:

Both connection (SSE) and fetcher (plain HTTP) transports are supported, so this works with TanStack Start, Next.js, Remix, or any backend you already have.

Gemini gets two new entry points:

• geminiAudio() for Lyria 3 Pro and Lyria 3 Clip music generation. Lyria Pro reads duration from the natural-language prompt; Clip is fixed at 30 seconds and returns MP3.
• A new gemini-3.1-flash-tts-preview TTS model with 70+ languages, 200+ audio tags, and multi-speaker dialogue via multiSpeakerVoiceConfig.

Fal gets three tree-shakeable adapters:

• falSpeech() for TTS via fal-ai/gemini-3.1-flash-tts, fal-ai/minimax/speech-2.6-hd, and the fal-ai/kokoro/* family.
• falTranscription() for STT via fal-ai/whisper, fal-ai/wizper, and fal-ai/speech-to-text/turbo.
• falAudio() for music, SFX, and the wider fal catalog: audio-to-audio, voice conversion and cloning, enhancement, separation, isolation, understanding, and merge.

All four follow the tree-shakeable subpath-import pattern, so your bundle only grows by the adapters you actually import.

The new activity is live in @tanstack/ai and the two provider packages:

Then open the audio generation guide for the full adapter matrix, or pull the ts-react-chat example to see working TTS and transcription tabs plus a /generations/audio route covering Lyria and fal side by side.

Star TanStack AI on GitHub if you want to see where this goes next.