My core is backend engineering Java/Spring, .NET, Python, cloud services. Over the last few months I've been building something well outside that comfort zone: a platform that lets businesses deploy AI-powered voice and WhatsApp assistants, built on LiveKit, retrieval-augmented generation (RAG), and telephony/SIP integrations.

What it does. Businesses can stand up an AI assistant that answers customer calls and WhatsApp messages, pulls accurate answers from their own knowledge base via RAG, and routes or escalates when it needs to. Under the hood it ties together SIP telephony, a real-time media pipeline (LiveKit/WebRTC), speech processing, and an LLM orchestration layer.

The unfamiliar part. Almost none of the real-time stack was in my background. WebRTC, SDP/media negotiation, ICE, codec handling, SIP trunking, AudioHook-style streaming — this is low-level, finicky territory where a single wrong assumption costs you a day. Coming from request/response backend systems, the mental model for continuous, stateful, real-time media was the steepest part.

How AI let me punch above my weight. I didn't ask AI to "build a voice agent." I used it as an on-demand expert on the protocol details while I owned the architecture and business logic. Concretely:

I fed it the actual docs (LiveKit/SIP/Genesys), my real error signatures, and packet/log excerpts, then had it reason through things like the SDP exchange or a one-way-audio failure step by step.
I treated every answer as a hypothesis to verify against a minimal repro or the real logs — not as truth. When it anchored on the wrong layer (e.g. blaming audio encoding for what was actually a connection-state bug), I'd hand it the real message flow and make it drop the bad hypothesis.
I kept orchestration, business logic, and prompts cleanly separated, so the AI-generated pieces stayed easy to reason about and replace.
The outcome. I shipped production-ready systems in domains WebRTC, SIP, speech processing that would otherwise have taken weeks just to get oriented in. AI collapsed the learning curve from "read everything first" to "learn while building, verify as I go."

What I'd tell another backend engineer moving into real-time/AI infra: lead the AI with the layer your evidence points to, not the layer your symptoms suggest. It anchors hard on whatever context you give it, so the skill is curating that context and verifying relentlessly.