This is a submission for the Gemma 4 Challenge: Build with Gemma 4

What I Built

Gemma 4 CAD Orchestrator is a cloud-native, AI-powered parametric CAD application that lets engineers describe mechanical parts in plain English and instantly see them rendered in 2D blueprint view and 3D interactive viewport — powered by Google DeepMind's Gemma 4 26B A4B IT via Vertex AI Model Garden.

Demo

https://gemma4-cad-orchestrator-176775177828.us-central1.run.app/

Code

How I Used Gemma 4

How I Built This

1. The Idea

Mechanical engineers spend hours in legacy CAD suites manually modeling parts that already exist entirely in their heads. I wanted to eliminate this friction by creating a deterministic, zero-install, text-to-geometry pipeline. By simply describing a part in natural language—"a mounting plate 100mm long, 60mm wide, 8mm thick with 10mm holes"—the AI instantly generates a fully interactive 3D model, synchronized 2D engineering blueprints, and an editable parametric feature tree.

2. Choosing the AI Backend

The foundational challenge was identifying an LLM capable of reliably parsing technical engineering intent into rigid, structured JSON schemas without hallucinating geometries. To maximize resilience, I built the system model-agnostic from day one with a pluggable backend interface that dynamically scans and cascades through available endpoints:

1. Ollama (Local workstation fallback via gemma4:31b or gemma4:26b)
2. Vertex AI Gemma 4 MaaS (Production-grade, global managed infrastructure)
3. Hugging Face Inference API (Alternative serverless endpoint)
4. Google Gemini API (High-availability operational fallback)

The Express backend auto-detects operational availability and gracefully handles failsafes down the stack.

3. System Architecture

User's Browser Cloud Run Vertex AI
┌──────────────┐ HTTP ┌──────────────────┐ gRPC ┌──────────────────┐
│ Three.js 3D │◄────────────►│ Express Server │◄──────────►│ Gemma 4 26B │
│ Canvas 2D │ /api/* │ @google/genai │ MaaS │ (global) │
│ Copilot UI │ │ System Prompt │ │ │
└──────────────┘ │ JSON parser │ └──────────────────┘
└──────────────────┘
│
┌────▼────┐
│google.json│

4. Operational Pipeline:

1. Intent Capture: The user defines a component concept inside the terminal-style Copilot UI.
2. Context Contextualization: The middleware intercepts the request and injects a strict system prompt bounding the data structure across 9 geometric primitives, explicit parametric ranges, and multi-shot valid execution pairs.
3. Inference Execution: Gemma 4 uses its high-speed Mixture-of-Experts (MoE) routing matrix to generate a strictly typed JSON payload holding shapeType, params, a descriptive explanation, a sequential featureTree, and raw native coordinates.
4. Client Translation: The frontend parses the schema. Three.js spins up the WebGL interactive viewport using responsive OrbitControls, while an HTML5 Canvas programmatically renders the 2D orthographic blueprints with accurate technical dimensions, centerlines, and hidden layer calculations.

5. Key Technical Decisions

6. Technical Engineering Challenges Overcome

• API Paradigm Shifts: Rebuilding routing structures during mid-hackathon SDK shifts required migrating from traditional model instantiations (getGenerativeModel) to direct top-level unified client calls.
• Undocumented Endpoint Properties: Debugging the global routing requirement for Model Garden MaaS instances required checking runtime variables, as generic regional project variables consistently failed network handshakes.
• JSON Conformance Over Enforcement: Standard models often fail to follow JSON output rules when technical jargon increases. This was solved by modifying the prompt structure to split thinking blocks from payload data blocks using distinctive tag sets.

7. Core Parametric Library

8. What's Next

• Multimodal Blueprint Auditing: Gemma 4 natively supports image and video processing inputs. The immediate next phase is enabling users to snap a photo of a rough white-board sketch, passing it directly into Gemma to reconstruct an exact CAD replica.
• Vector & Asset Export: Implement programmatic transformation modules to export geometries directly into industrial formats like STEP, DXF, and STL.
• Edge Modifications: Introduce dynamic modifier hooks to compute automated Chamfer, Fillet, and Shell algorithms across active objects.

Snip about my ideas: