Building an Inference OS for prediction markets
Most AI agent stacks default to "throw the prompt at GPT-4o, hope for the best." For prediction markets that's expensive AND wrong — most market questions don't need a paid LLM at all. Here's how we built a 6-hook deterministic-first inference router on top of Kairon Forge.
The 6 hooks (in priority order)
- Market Regime classifier — 5 deterministic regimes (whale_dominant / meme_volatile / macro_anchored / panic_liquidation / dead_liquidity). Confident classification short-circuits the entire router. Zero LLM call.
- Anomaly detector — 3σ price spike + sentiment divergence. Confident anomaly FORCES Tier-2 (paid Claude/Anthropic), bypassing the viability cost cap on rare-and-important markets.
- Time-to-Resolution decay — exponential confidence decay vs event horizon. Low decayed confidence forces Tier-1 (Haiku-only).
- Persona overlay — 5 archetype priors (calibrated_researcher / whale_mimic / panic_seller / momentum_trader / contrarian) adjust baseline confidence.
- Panic mode circuit breaker — 60s rolling burn-rate σ. >2σ from baseline → force Ollama-only.
- Economic Viability Filter — per-tier hard cost cap (Free $0.05 / Pro $0.50 / Elite $5 / Enterprise $100). >cap → 402 quotaExhausted.
Cost-aware Cognition
Before every paid call, EIG / cost ratio gate (shouldEscalate(eig, cost, threshold=0.5)). Information gain ÷ inference cost. Below threshold → collapse to Tier-1 + budget consumption note.
Test coverage
350+ inference tests covering router decision boundaries. Components: budget consumption gate, complexity classifier (trivial / medium / rare_hard), Tier-2 dispatch, recursion-depth + context-bloat guards, reflection-loop + duplicate-prompt detection.
Why this matters
Cursor's silent auto-upgrade on quota exhaustion triggered viral brand backlash + US state class-action allegations. We engineered a structural answer: tier caps, panic mode, no-auto-charge — all enforced at the router layer.
Source: github.com/greymoth-jp · Live: kairon.trade
This is part of the API Kernel work at services/kairon-guardian/ — happy to answer architecture questions.