🌌 The Vision: Reimagining Web Interaction for Everyone
The web was built for everyone, but it wasn't built equally.
For users with visual impairments, motor challenges, or cognitive differences, navigating modern, heavily-nested single-page applications is a series of frustrating obstacles. Screen readers read endless lists of unlabelled divs, modal traps block focus, and complex keyboard navigation paths turn a simple form submission into a test of patience.
What if your browser didn't just render pages, but understood them? What if it could see what you wanted to do, execute actions on your behalf, and self-heal broken web pages on the fly to meet accessibility standards?
Introducing Aura: A next-generation, agentic browser assistant built on Gemma 4 (gemma-4-26b-a4b-it). Aura transforms Google Chrome from a passive renderer into an active accessibility copilot, combining voice-driven browser automation, semantic page memory, and a self-healing WCAG engine.
⚡ What is Aura?
Aura is a Manifest V3 Google Chrome extension that operates via an elegant, glassmorphic sidepanel. It acts as a companion that can see the viewport, listen to natural speech, analyze vocal urgency, and interact with the page just like a human would.
🌟 Core Capabilities
- Agentic Web Automation: Ask Aura to "Find the latest post, fill out the sign-up form, and submit it." Aura compiles the DOM tree, reasons about the goal using Gemma 4, predicts the optimal path, and executes native clicks and keyboard inputs.
- Live WCAG Self-Healing Engine: Aura continuously scans the active tab for web accessibility violations (missing alt tags, poor contrast, broken aria roles, unlabeled inputs). It doesn't just warn you—it reconstructs and injects CSS/ARIA fixes in real-time to patch the website's code on the fly.
- Vocal Urgency & Stress Analysis: Built specifically for accessibility, Aura monitors raw microphone streams to measure amplitude variance, rms amplitude, and pitch fluctuations. If a visually impaired user expresses stress or urgency, Aura detects it and accelerates execution, skipping redundant safety validation prompts.
4. On-Device Vector Memory & Habit Graphs: Aura learns how you use your favorite sites. It builds a local semantic index of page elements and uses a predictive transition graph to predict your next actions, cutting down LLM latency to near-zero.
🛠️ The Architecture Behind Aura
Aura utilizes a sophisticated service worker, content scripts, and a React-based web panel. Because LLM calls are asynchronous and Chrome's Manifest V3 service workers are prone to random termination, Aura employs a highly resilient, enterprise-grade architecture:
- Microphone Stream -> Vocal Urgency Analyzer -> Sidepanel Host UI
- User Input -> Service Worker Keep-Alive Loop -> DOM Compressor & Parser
* Semantic Trees -> Gemma 4 Tool Router -> Tool Selection (runs Audits, searches memory, or triggers native elements inside pages).
🛡️ Overcoming Real-World Challenges
Building extension-based AI agents is incredibly difficult due to the sandboxed nature of Chrome. Here is how we tackled major development bottlenecks:
The Tab Window Dilemma: chrome.tabs.query({ active: true, currentWindow: true }) often fails when the user clicks inside the sidepanel itself (as the sidepanel becomes the current active window). We resolved this by querying with lastFocusedWindow: true, making sure actions are targetable globally.
The Memory Explosion Guard: Continuous DOM scanning could lead to embedding thousands of useless nodes in memory. We introduced an Embedding Storm Guard that caps vector updates to 15 semantic elements per state-change.