What is Claw Learn?

Claw Learn combines the ElevenLabs Speech Engine with an AI scene planner and a custom canvas renderer to turn math questions into live animated explanations with synchronized narration.

The Speech Engine is the core of the experience — it handles both voice input and audio output over WebRTC, so you can speak your question, interrupt mid-explanation, and ask follow-ups without ever touching a keyboard. When the Speech Engine isn't configured, the app falls back to REST TTS and browser-based speech recognition.

No slides. No textbooks. No pre-recorded videos. Every explanation is generated fresh for your exact question.

You:  "Why does the derivative represent slope?"

App:  → ElevenLabs Speech Engine captures your voice over WebRTC
      → AI generates a 10-scene visual teaching plan
      → Canvas renders: axes, parabola, tangent line, slope formula
      → Speech Engine narrates each scene in sync with the animation
      → Interrupt at any time to ask a follow-up — just speak

Demo

Add a GIF or screenshot here

Try these questions:

• "How does matrix multiplication work?"
• "Explain the Fourier transform visually"
• "What is integration and why does it find area?"
• "Show me Euler's formula e^(iπ) + 1 = 0"
• "How does gravity create orbits?"

Tech Stack

Getting Started

Prerequisites

• Node.js 18+
• An OpenAI-compatible API key — Gemini (free at aistudio.google.com), OpenAI, or any compatible provider
• Google OAuth credentials — required for login (console.cloud.google.com)
• Upstash Redis — recommended for rate limiting (console.upstash.com, free tier)
• ElevenLabs — optional, free tier at elevenlabs.io

1. Clone and install

git clone https://github.com/arzumanabbasov/claw-learn.git
cd claw-learn
npm install

2. Configure environment variables

cp .env.local.example .env.local

Open .env.local and fill in your keys:

# ── AI Provider (required) ────────────────────────────────────────────────────
OPENAI_API_KEY=your_api_key_here
OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
OPENAI_MODEL=gemini-2.5-flash

# ── Auth (required) ───────────────────────────────────────────────────────────
# Generate a secret: openssl rand -base64 32
AUTH_SECRET=your_auth_secret_here

# Google OAuth — https://console.cloud.google.com/
GOOGLE_CLIENT_ID=your_google_client_id
GOOGLE_CLIENT_SECRET=your_google_client_secret

# ── Rate limiting — Upstash Redis (recommended) ───────────────────────────────
# Without these, rate limiting falls back to in-memory (resets on server restart)
# Create a free Redis DB at https://console.upstash.com/
UPSTASH_REDIS_REST_URL=https://your-db.upstash.io
UPSTASH_REDIS_REST_TOKEN=your_token_here

# ── ElevenLabs Voice (optional) ───────────────────────────────────────────────
ELEVENLABS_API_KEY=your_elevenlabs_api_key_here
ELEVENLABS_VOICE_ID=pNInz6obpgDQGcFmaJgB

# Speech Engine — full WebRTC voice I/O
# Create an agent at https://elevenlabs.io/app/conversational-ai
ELEVENLABS_SPEECH_ENGINE_ID=agent_xxxxxxxxxxxxxxxxxxxx

# ── Security ──────────────────────────────────────────────────────────────────
ALLOWED_ORIGIN=https://your-domain.com

3. Run

Open http://localhost:3000.

Authentication

Claw Learn uses NextAuth.js v5 with Google OAuth. All routes require a valid session — unauthenticated users are redirected to /login.

Setup:

1. Go to console.cloud.google.com → APIs & Services → Credentials
2. Create an OAuth 2.0 Client ID (Web application)
3. Add your domain to Authorized JavaScript origins and https://your-domain.com/api/auth/callback/google to Authorized redirect URIs
4. Copy the Client ID and Secret into your env vars
5. Generate AUTH_SECRET with openssl rand -base64 32

For local dev, add http://localhost:3000 as an authorized origin and http://localhost:3000/api/auth/callback/google as a redirect URI.

Rate Limiting

Each authenticated user gets 3 questions per day, tracked by their Google user ID and reset at UTC midnight.

Rate limiting uses Upstash Redis in production — an atomic INCR with a TTL set to the end of the current UTC day. This is serverless-safe and works across all Vercel edge instances.

Without Upstash credentials, the app falls back to an in-memory store that resets whenever the server restarts (fine for local dev, not suitable for production).

Setup:

1. Create a free Redis database at console.upstash.com
2. Copy the REST URL and token into UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN

The remaining question count is shown in the top bar as a live badge and resets automatically each day.

Claw Learn uses the OpenAI-compatible API format, so it works with any provider that supports it.

Gemini (default)

OPENAI_API_KEY=your_gemini_api_key
OPENAI_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai
OPENAI_MODEL=gemini-2.5-flash

OpenAI

OPENAI_API_KEY=your_openai_api_key
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-4o

Ollama (local)

OPENAI_API_KEY=ollama
OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_MODEL=llama3.1

Voice Modes

Mode 1 — ElevenLabs Speech Engine (recommended)

The Speech Engine connects via WebRTC to an ElevenLabs Conversational AI agent and is the primary voice interface for Claw Learn. It handles both input and output in a single low-latency connection:

• Voice input — speak your questions naturally, no typing needed
• Streaming TTS — audio streams directly from ElevenLabs as each scene plays
• Interruption — speak mid-explanation to redirect or ask a follow-up
• Lower latency — WebRTC is significantly faster than the REST fallback

Setup:

1. Go to elevenlabs.io/app/conversational-ai
2. Create a new agent
3. Set the system prompt to: "You are a math narration voice. Read exactly what the user sends you as clear, natural narration."
4. Copy the Agent ID and set it as ELEVENLABS_SPEECH_ENGINE_ID in your .env.local

The Voice button in the top bar connects and disconnects the Speech Engine. When connected, a pulsing green indicator shows the session is live.

Mode 2 — REST TTS fallback

When ELEVENLABS_API_KEY is set but no Speech Engine is configured, each scene's narration is sent to the ElevenLabs REST API and played back as audio. No voice input in this mode.

Mode 3 — No voice

The app works fully without any ElevenLabs configuration — text input and silent animations only.

Deployment

Vercel (recommended)

Set these environment variables in the Vercel dashboard under Settings → Environment Variables:

The vercel.json in the repo is pre-configured.

Self-hosted

Requires Node.js 18+ and the environment variables above.

Visual Element Reference

The canvas renderer supports 30+ element types:

Coordinate system: origin at center, x right, y up. Typical visible range: x ∈ [-6, 6], y ∈ [-4, 4].

Project Structure

clawlearn/
├── app/
│   ├── api/
│   │   ├── explain/route.ts          # POST — AI scene plan generation
│   │   ├── narrate/route.ts          # POST — ElevenLabs REST TTS
│   │   └── speech-engine/token/      # GET  — WebRTC conversation token
│   ├── page.tsx                      # Root — landing ↔ tutor router
│   ├── layout.tsx                    # Fonts, metadata, global CSS
│   └── globals.css                   # Design tokens, animations
│
├── components/
│   ├── LandingPage.tsx               # Marketing page
│   ├── TutorApp.tsx                  # App shell
│   ├── AnimationCanvas.tsx           # Canvas + scene sequencer
│   ├── ConversationPanel.tsx         # Chat history
│   ├── QuestionInput.tsx             # Input bar
│   └── NarrationSubtitle.tsx         # Subtitle below canvas
│
├── hooks/
│   ├── useTutor.ts                   # Core orchestration
│   ├── useSpeechEngine.ts            # ElevenLabs Speech Engine (WebRTC)
│   └── useVoice.ts                   # Web Speech API fallback
│
├── lib/
│   ├── openai.ts                     # OpenAI-compatible client + system prompt
│   ├── animationEngine.ts            # Canvas renderer (30+ elements)
│   ├── elevenlabs.ts                 # ElevenLabs REST helpers
│   └── voiceRecognition.ts           # Web Speech API wrapper
│
├── types/
│   └── scene.ts                      # Scene plan TypeScript types
│
├── .env.local.example                # Environment variable template
├── CONTRIBUTING.md                   # Contribution guide
├── LICENSE                           # MIT
└── vercel.json                       # Vercel deployment config

Security

• API keys are server-side only — never exposed to the browser
• Input is length-limited and validated on every API route
• CORS is locked to ALLOWED_ORIGIN in production
• The canvas renderer uses a safe recursive-descent math parser — no eval or new Function
• Security headers (X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy) are set on all responses

See SECURITY.md for the full security policy and how to report vulnerabilities.

Known Limitations

• No persistence — conversation history is in-memory, cleared on page refresh
• Voice input — Web Speech API fallback requires Chrome or Edge; the Speech Engine works in all modern browsers
• JSON truncation — very complex topics may cause the AI to return truncated JSON; the parser attempts recovery by finding the last complete scene
• ElevenLabs free tier — 10,000 characters/month; the app continues silently without narration when quota is exceeded

Contributing

Contributions are welcome. Please read CONTRIBUTING.md first.

# Fork, then:
git checkout -b feat/your-feature
# Make changes
npx tsc --noEmit   # must pass
git commit -m "feat: your feature"
git push origin feat/your-feature
# Open a pull request

Acknowledgments

• Inspired by 3Blue1Brown and the manim animation library
• Built with the ElevenLabs Speech Engine, Next.js, and Framer Motion

License

MIT © 2025 Claw Learn Contributors