This is a submission for the Gemma 4 Challenge: Write About Gemma 4

Disclaimer: This article is an independent opinion piece. The author has no affiliation with Google DeepMind or any entity associated with the Gemma project. All benchmark figures cited are sourced from publicly available documentation, the official Gemma 4 model card, peer-reviewed preprints, and community evaluations as of May 2026. Where benchmarks are self-reported by Google, this is noted explicitly. This analysis should not be treated as definitive technical guidance. Readers are encouraged to run their own evaluations against their specific use cases.

A 10-dimension comparative analysis of E2B, E4B, 26B MoE, and 31B Dense — with the hard opinion that architecture choice is being systematically confused with capability choice.

Table of Contents

• Why This Analysis Exists
• The Models: A Technical Primer
• The Framework: 10 Evaluation Dimensions
• 1️⃣ Instruction Following
• 2️⃣ Reasoning Capability
• 3️⃣ Coding Ability
• 4️⃣ Hallucination Resistance
• 5️⃣ Privacy & Safety Compliance
• 6️⃣ Domain Knowledge
• 7️⃣ Long-Context Understanding
• 8️⃣ Creativity & Writing Quality
• 9️⃣ Multilingual Capability
• 🔟 Efficiency & Cost
• The Master Decision Matrix
• The Overlooked Argument: MoE Is Not a Middle Ground
• Deployment Scenarios with Recommendations
• My Verdict
• Key Takeaways

Why This Analysis Exists

When Google DeepMind released Gemma 4 on April 2, 2026, most coverage landed on two narratives: small models now run on phones and 31B beats models 20× its size. Both are accurate. Neither is sufficient for a developer deciding which variant to actually deploy.

The four Gemma 4 models are not a simple size ladder where bigger is always better. They represent three distinct architectural philosophies — Per-Layer Embedding (PLE) dense models for edge, Mixture-of-Experts for efficient serving, and standard dense for maximum quality — deployed across a hardware spectrum from Raspberry Pi to H100 server.

Choosing the wrong variant doesn't just waste compute. It can mean deploying a model that fundamentally cannot perform the task you're asking of it — not because the quality is insufficient, but because the architecture is mismatched.

This analysis provides the comparison I couldn't find when I needed it: systematic, honest, multi-dimensional, and opinionated where the evidence supports an opinion.

The Models: A Technical Primer

Before evaluating, we need to be precise about what we're comparing. The Gemma 4 family shares a 262,144-token vocabulary and a hybrid local/global attention architecture across all variants. That architectural commonality is important — it means the comparison is about deployment target and capacity, not fundamentally different design philosophies.

The Four Variants

┌─────────────────┬──────────────┬──────────────┬──────────────┬──────────────┐
│                 │     E2B      │     E4B      │  26B MoE     │  31B Dense   │
├─────────────────┼──────────────┼──────────────┼──────────────┼──────────────┤
│ Architecture    │ Dense + PLE  │ Dense + PLE  │ MoE (8/128)  │ Dense        │
│ Total Params    │ 5.1B (2.3B   │ 8B (4.5B     │ 25.2B        │ 30.7B        │
│                 │ effective)   │ effective)   │              │              │
│ Active Params   │ 2.3B         │ 4.5B         │ 3.8B         │ 30.7B        │
│ Context Window  │ 128K tokens  │ 128K tokens  │ 256K tokens  │ 256K tokens  │
│ Sliding Window  │ 512 tokens   │ 512 tokens   │ 1024 tokens  │ 1024 tokens  │
│ Modalities      │ Text+Img+Aud │ Text+Img+Aud │ Text+Image   │ Text+Image   │
│ Vision Encoder  │ ~150M        │ ~150M        │ ~550M        │ ~550M        │
│ Layers          │ 35           │ 42           │ 30           │ 60           │
│ RAM (Q4)        │ ~1.5 GB      │ ~5 GB        │ ~14–18 GB    │ ~20 GB       │
│ License         │ Apache 2.0   │ Apache 2.0   │ Apache 2.0   │ Apache 2.0   │
└─────────────────┴──────────────┴──────────────┴──────────────┴──────────────┘

A note on "effective" vs "total" parameters in E2B/E4B:
The "E" stands for "effective." These models use Per-Layer Embeddings (PLE) — each decoder layer maintains its own small embedding table. The embedding tables are large in total parameter count (5.1B for E2B, 8B for E4B) but are lookup operations, not matrix multiplications. The computational parameter count during inference is the effective number (2.3B, 4.5B). This distinction matters for understanding both the RAM requirements and the quality ceiling of these models.

A note on the 26B MoE's "A4B":
The "A4B" suffix means "4 Billion Active." Of the 25.2B total parameters, only 3.8B are activated per token — routing to 8 of 128 available expert networks. This is why the 26B MoE can deliver near-31B quality at near-E4B inference cost, but only when the router makes good decisions. Expert routing stability under adversarial or out-of-distribution prompts remains an open research question.

The Framework: 10 Evaluation Dimensions

Rather than treating this as a flat benchmark race, I evaluate each model across ten dimensions that reflect real deployment decisions. For each dimension, I provide: a qualitative assessment, supporting evidence, and a winner recommendation.

The evidence base draws from: the official Gemma 4 model card (Google DeepMind), the preprint "Gemma 4, Phi-4, and Qwen3: Accuracy-Efficiency Tradeoffs in Dense and MoE Reasoning Language Models" and community benchmarks.

1️⃣ Instruction Following: How Accurately Does It Follow You?

What this measures: Single-step and multi-step instruction compliance, constraint satisfaction, format adherence, and sensitivity to ambiguous or contradictory instructions.

Assessment

Instruction following quality scales roughly linearly with parameter count in this family — but the type of instruction-following failure differs by architecture.

E2B follows simple, unambiguous single-step instructions reliably. Its failure mode is constraint stacking: give it three simultaneous constraints ("write in Tamil, use formal register, limit to 150 words") and it begins dropping constraints silently, typically the most structurally demanding one. This is a documented limitation of the PLE architecture at the 2B effective scale.

E4B handles 2–3 simultaneous constraints well. Community benchmarks show it completing structured format tasks (JSON output, markdown tables, code with specific patterns) at a success rate competitive with models 3–4× its effective parameter count. The 128K context window is sufficient to provide rich few-shot examples for complex instruction patterns.

26B MoE shows qualitatively different behavior: instruction adherence is excellent when the query falls within a well-represented expert's domain, but occasional inconsistencies appear at domain-task junctions — likely because different experts were activated for different parts of a multi-step task. The 256K context window meaningfully helps here by allowing more extensive system prompts that constrain behavior upfront.

31B Dense is the most consistent instruction-follower in the family. Its single-architecture design means there's no expert routing ambiguity. In the arXiv:2604.07035 study, 31B Dense achieved the highest scores on structured output tasks requiring precise format adherence across all tested chain-of-thought prompting strategies.

Winner: 31B Dense
Best value: E4B (excellent compliance at a fraction of the resource cost)

2️⃣ Reasoning Capability: The Benchmark That Shocked the Community

What this measures: Mathematical reasoning, logical inference, multi-step problem decomposition, chain-of-thought quality, and performance on competition-grade reasoning benchmarks.

Assessment

This is where Gemma 4's benchmark numbers became a topic of genuine discussion in the open-source AI community.

Key figures (sourced from official model card and community validation):

AIME 2026 scores for Gemma 3 27B reference: 20.8% — illustrating the generational improvement.

The 31B model's Arena AI ELO of 1452 places it third on the text leaderboard among all models (open and closed), ahead of Qwen 3.5 27B (1403) and DeepSeek-V3.2 (~1425). The 26B MoE reaches sixth place at 1441. Both achieve this at a parameter efficiency that, in the evaluation community's language, "shouldn't be possible."

The arXiv:2604.07035 finding that deserves attention:

Under few-shot chain-of-thought prompting, E4B achieves a weighted accuracy of 0.675 across ARC-Challenge, GSM8K, and Math Level 1–3 — marginally outperforming the 26B MoE's 0.663 on the same benchmark suite, while consuming only 14.9 GB VRAM versus 48.1 GB. This is a striking result. At specific task types, the efficiency of the E4B's architecture may actually produce better outcomes than the MoE router's task routing.

My interpretation: E4B is an underrated reasoning model. Most practitioners immediately jump to the 26B MoE for "serious" reasoning tasks — but for structured mathematical and logical problems, E4B with few-shot CoT prompting is surprisingly competitive, at a fraction of the hardware cost.

Winner: 31B Dense (marginally, ~89.2% vs 88.3% AIME 2026)
Best surprise: E4B (0.675 weighted multi-task under few-shot CoT)

3️⃣ Coding Ability: Where the Reputational Ceiling Shows

What this measures: Code generation from spec, debugging, refactoring, multi-file task completion, and agentic coding benchmarks.

Assessment

Coding is the dimension where Gemma 4's strongest models face their clearest competitive ceiling.

Key figures:

For reference: GLM-5.1 reaches 78% SWE-bench Verified; Claude Opus 4.7 reaches 87.6%. Gemma 4 31B's ~64% puts it firmly in the "strong for individual functions and moderate tasks" category but below the current frontier for complex multi-file agentic coding.

Notable for local deployment: E4B running via Ollama at 57 tokens/second on an M4 Pro MacBook generated working full-stack React applications in community testing. This is a qualitative claim, not a standardized benchmark — but the implication is significant: for mid-complexity application scaffolding, E4B's speed advantage compensates for its quality gap in practice.

Function calling: an important asymmetry. Only 31B Dense supports reliable structured function calling. 26B MoE has limited support. E2B and E4B do not reliably support tool use. For any agentic application requiring tool orchestration, this is not a preference — it is a hard constraint.

Winner: 31B Dense (quality + function calling)
Practical local: E4B (speed + sufficient quality for most tasks)
Do not use for agentic coding: E2B, E4B (no reliable tool calling)

4️⃣ Hallucination Resistance: The Metric Nobody Advertises

What this measures: Factual accuracy, confident confabulation of non-existent information, citation fabrication, and TruthfulQA performance.

Assessment

This is the dimension with the least publicly available controlled data for Gemma 4 specifically — which is itself a signal worth noting. Google's official model card does not publish TruthfulQA scores for the full family.

From the arXiv:2604.07035 preprint, TruthfulQA MC1 results under zero-shot prompting show:

• E2B: 0.423
• E4B: 0.461
• 26B MoE: 0.498
• 31B Dense: 0.512 (inferred)

These are not impressive absolute scores — TruthfulQA is notoriously difficult, and the MC1 metric is particularly strict. But the relative ordering is consistent: more parameters correlates with better factual calibration, and few-shot CoT prompting improves all models substantially on this metric.

A pattern worth noting: The 26B MoE shows occasional inconsistency in factual recall that appears linked to expert routing — a factual claim that activates one expert is answered correctly; a semantically similar question that routes differently produces a confabulation. This is a known failure mode of first-generation MoE models and is under active community investigation.

For high-stakes factual applications (legal research, medical information, compliance queries), none of the Gemma 4 variants should be deployed without retrieval augmentation. The 31B Dense provides the strongest baseline, but RAG is not optional for accuracy-critical domains regardless of model size.

Winner: 31B Dense
Practical guidance: Use RAG for all variants in factual-accuracy-critical applications

5️⃣ Privacy & Safety Compliance: The Local Deployment Advantage

What this measures: Handling of sensitive data in prompts, jailbreak resistance, refusal of harmful requests, and compliance with privacy principles.

Assessment

This dimension is where the Gemma 4 family's on-device deployment capability creates a structurally different conversation compared to cloud models.

Jailbreak resistance improves with scale across the family. The 31B Dense model with its native system prompt support (system role) allows more robust behavioral guardrailing than the smaller variants. In community red-teaming, E2B showed the highest susceptibility to prompt injection and role-play jailbreaks. 31B Dense with a well-engineered system prompt demonstrated substantially stronger constraint adherence.

Privacy compliance — the structural advantage:

For applications governed by data protection frameworks (GDPR, Sri Lanka PDPA No. 9 of 2022, UAE PDPL), the on-device deployment capability of the E2B and E4B models creates a legal architecture that cloud models cannot replicate:

Local deployment data flow:
  User input → local inference → local output
  [No data leaves device — no "processing by a controller in a third country"]

Cloud deployment data flow:
  User input → network transmission → cloud inference → response
  [Triggers cross-border data flow provisions; Section 26 PDPA applies]

For controllers processing special categories of personal data (health, legal, financial), the E2B and E4B models running on-device are not just technically interesting — they may be legally preferable under data minimization and purpose limitation principles (Sections 6–7, PDPA 2022).

The safety evaluation gap: Google has not published comprehensive safety evaluation results for the full Gemma 4 family. This absence is notable. ShieldGemma exists as a separate safety classifier, and integrating it as a pre/post-filter is recommended for any production deployment regardless of model size.

Winner for privacy-critical local deployments: E2B / E4B
Winner for safety constraint adherence: 31B Dense
Best practice: Deploy ShieldGemma as a filter for all variants

6️⃣ Domain Knowledge: Where the Gap Between Variants Is Largest

What this measures: Performance in specialized domains — law, medicine, finance, engineering — requiring both factual depth and domain-specific reasoning patterns.

Assessment

GPQA Diamond (Graduate-Level Google-Proof Q&A) is the most useful proxy for domain knowledge depth. These are questions that require genuine domain expertise, not surface-level pattern matching.

The 31B at 84.3% GPQA Diamond outperforms Llama 4 Scout (109B total, 17B active) at 74.3%. This is the benchmark result that most concretely demonstrates the Gemma 4 efficiency story.

A domain-specific observation from legal AI: For legal research applications requiring statutory interpretation, case law analysis, and multi-jurisdictional comparison, the 26B MoE and 31B Dense operate in a qualitatively different regime than E2B/E4B. The difference is not just accuracy — it is the ability to hold and reason over multiple competing legal frameworks simultaneously, which requires a context coherence that smaller models demonstrably lack.

For medical applications: Google has released MedGemma as a specialized medical variant. For production medical AI, MedGemma should be preferred over the base Gemma 4 variants regardless of size — it represents domain fine-tuning that general-purpose benchmarks cannot replicate.

Winner: 31B Dense
Practical alternative: 26B MoE (2% gap on GPQA Diamond, substantially lower hardware requirements)

7️⃣ Long-Context Understanding: The 256K Story Is Half-Told

What this measures: Coherence maintenance over large documents, needle-in-a-haystack retrieval, multi-document synthesis, and performance degradation as context length increases.

Assessment

The context window specifications are well-documented:

The architectural enabler: Gemma 4 uses a hybrid local/global attention mechanism. Local sliding window attention (512 tokens for small models, 1024 for large) keeps per-token compute linear in sequence length. Periodic global attention layers (unified Keys/Values with Proportional RoPE) handle long-range dependencies without the quadratic cost of full attention. This is why these context windows are practically usable, not just theoretically specified.

The honest caveat the specifications don't tell you: Context window ≠ effective context use. In practice, all language models show attention degradation as the context fills — the "lost in the middle" problem, where information in the middle of a long context is less reliably attended to than information at the beginning and end. Gemma 4 mitigates this better than its predecessors due to the global attention layers, but the problem is not eliminated.

Community evidence: At 24K-token contexts (roughly 100 pages), E4B showed reliable single-document comprehension in structured Q&A tasks. At 80K+ tokens (multi-document synthesis), degradation became noticeable. The 26B and 31B models maintained better coherence at the same context densities — consistent with the larger sliding window and more layers for global integration.

For entire-codebase analysis, legal corpus review, or long-document research: 26B MoE or 31B Dense are required. E2B/E4B's 128K window is sufficient for most individual documents but not for multi-document cross-referencing tasks.

Winner: 31B Dense (marginally better than 26B MoE on coherence at maximum context)
Practical choice for most document tasks: E4B (128K covers the overwhelming majority of single-document workflows)

8️⃣ Creativity & Writing Quality: Underexplored Territory

What this measures: Narrative coherence in creative writing, stylistic control, summarization fidelity, argumentation quality, and human preference in writing evaluation.

Assessment

Human preference evaluation on writing tasks is where Arena AI's ELO ratings are most directly informative — they are derived primarily from human comparative ratings of response quality across conversational and generative tasks.

The Arena AI ELO correlation with subjective writing quality is imperfect but meaningful at the population level. The 31B Dense's ELO of 1452 versus the 26B MoE's 1441 represents a small but consistent human preference advantage in direct comparisons.

Practical observation: At the E4B level, writing quality is competitive with 7B-class models from earlier generations. For blog post drafts, email composition, and general-purpose summarization, E4B produces output that most professionals would find adequate. For long-form analytical writing requiring sustained argument coherence over 2,000+ words, the 26B MoE and 31B Dense models are noticeably superior.

The creativity ceiling for small models: E2B and E4B show a characteristic pattern in creative writing: strong sentence-level quality with weak paragraph-level coherence — individual sentences are grammatically correct and contextually appropriate, but the narrative arc degrades over longer pieces. This reflects the smaller context of the sliding window (512 tokens) limiting how much of the preceding text is actively attended to.

Winner: 31B Dense
Adequate for most professional writing: E4B

9️⃣ Multilingual Capability: Where the Promise Outruns the Delivery

What this measures: Performance across languages with varying resource levels — English and major European languages (high-resource), Arabic and Hindi (medium-resource), Tamil and Sinhala (low-resource).

Assessment

This is the dimension I hold the strongest opinion on, and where my research background in South Asian NLP informs my analysis most directly.

The official claim: Gemma 4 supports 140+ languages. This is technically accurate and practically misleading for low-resource language applications.

What "140+ languages" actually means in practice:

The training data distribution in large multilingual models follows a power law — a small number of high-resource languages dominate, and performance degrades as a function of training data volume for each language. Google has not published per-language training data statistics for Gemma 4.

Observed performance pattern:

Language Tier      | Representative Languages | Effective Quality Level
───────────────────┼─────────────────────────┼─────────────────────────
Tier 1 (High)      | English, French, German, | E4B approaches 31B quality
                   | Spanish, Chinese, Japanese|
───────────────────┼─────────────────────────┼─────────────────────────
Tier 2 (Medium)    | Arabic, Hindi, Turkish,  | Significant gap between
                   | Korean, Portuguese       | E4B and 31B; 31B preferred
───────────────────┼─────────────────────────┼─────────────────────────
Tier 3 (Low)       | Tamil, Sinhala, Swahili, | All models degrade;
                   | Nepali, Bengali          | 31B meaningfully better
                   |                          | but still unreliable for
                   |                          | complex tasks
───────────────────┼─────────────────────────┼─────────────────────────
Tier 4 (Very Low)  | Sinhala (complex script),| Gemma 4 insufficient
                   | minority languages       | without fine-tuning

The Tamil and Sinhala case — a personal note:

The research literature on LLM performance in Sinhala (Jayakody & Dias, 2024, arXiv:2407.21330 documents consistently poor performance of general-purpose multilingual models on complex Sinhala tasks — including text generation, summarization, and translation — even from models nominally supporting the language. Tamil, while better resourced, also shows significant performance degradation on domain-specific tasks (legal, medical) and regional variants.

For practitioners building applications for Sri Lankan users in their native languages: Gemma 4's multilingual support for Tamil and Sinhala is a baseline, not a solution. Fine-tuning on local-language corpora — using the approach demonstrated in arXiv:2604.07035 and the Hypa-Gemma4-E2B work — is necessary for production-quality applications. The E2B and E4B models are the practical fine-tuning targets at this scale; the 26B MoE is notoriously difficult to fine-tune due to router weight sensitivity.

Arabic performance: The medium-resource tier. Standard Arabic performs at near-Tier-1 quality in 31B Dense. Dialectal Arabic variants (Egyptian, Levantine, Gulf) degrade substantially. For legal or financial applications in Arabic, 31B Dense with domain context is recommended; E4B is insufficient.

The multilingual audio advantage (E2B/E4B): The native audio capability of the small models supports multilingual transcription and translation in a single inference call — a genuine capability gap versus the larger models, which lack audio support entirely. For multilingual audio applications (voice assistants, transcription tools), E2B or E4B may be the correct choice regardless of text quality considerations.

Winner for high-resource languages: 31B Dense
Winner for low-resource language fine-tuning target: E4B (manageable size, Apache 2.0)
Winner for multilingual audio: E2B / E4B (only variants with audio support)
Honest assessment: All variants require fine-tuning for serious low-resource language applications

🔟 Efficiency & Cost: The Number That Changes the Decision

What this measures: Inference latency, token throughput, memory consumption, hardware requirements, and total cost of ownership across deployment scenarios.

Assessment

This is the dimension that most directly determines which model is deployable rather than merely capable.

Hardware requirements and throughput:

The 26B MoE problem on consumer hardware:

The ~2 tok/s throughput of the 26B MoE on a 24GB MacBook is not a usable local inference speed. The model technically loads — but at 2 tokens per second, a 500-word response takes over four minutes. This is the machine learning equivalent of technically correct but practically unusable.

The 26B MoE's intended deployment target is GPU servers with HBM memory, where its MoE routing delivers high throughput through expert parallelism. On a single consumer GPU, the MoE architecture's benefits largely disappear — you're paying the routing overhead without the parallelism benefit.

The E4B sweet spot:

At 57 tok/s on consumer hardware, 5 GB RAM, and a 128K context window, E4B occupies the most interesting efficiency point in the family. It runs on an 8GB laptop, streams responses faster than most people can read, and handles the majority of practical tasks at a quality level that would have been considered a large model two years ago.

API access for larger models:

For teams without the hardware for 26B or 31B local deployment, Google provides free API access via the Gemini API with rate limits (26B: 15 RPM, 1500 TPM). OpenRouter hosts Gemma 4 31B as a free tier. For intermittent use, API access eliminates the infrastructure cost entirely — the question becomes whether your use case tolerates rate limiting and the privacy implications of cloud API submission.

Fine-tuning cost comparison:

For organizations wanting to fine-tune on proprietary domain data, E4B provides the best balance of fine-tunability, quality post-fine-tune, and hardware accessibility. The 26B MoE is notoriously difficult to fine-tune correctly — router and expert weights require careful handling, and the community tooling (Unsloth, TRL) was still maturing as of this writing.

Winner: E2B (absolute efficiency)
Best value: E4B (57 tok/s, 5 GB, 128K context, practical fine-tuning)
Avoid for local deployment: 26B MoE (2 tok/s on consumer hardware is not viable)

The Master Decision Matrix

┌──────────────────────────────┬──────┬──────┬──────────┬──────────┐
│ Dimension                    │ E2B  │ E4B  │ 26B MoE  │ 31B Dense│
├──────────────────────────────┼──────┼──────┼──────────┼──────────┤
│ 1. Instruction Following     │  ●●  │ ●●●  │  ●●●●    │  ●●●●●   │
│ 2. Reasoning                 │  ●●  │ ●●●  │  ●●●●●   │  ●●●●●   │
│ 3. Coding Ability            │  ●●  │ ●●●  │  ●●●●    │  ●●●●●   │
│ 4. Hallucination Resistance  │  ●●  │ ●●●  │  ●●●●    │  ●●●●●   │
│ 5. Privacy (local)           │ ●●●●●│●●●●● │  ●●●     │  ●●●     │
│ 5. Safety (jailbreak)        │  ●●  │ ●●●  │  ●●●●    │  ●●●●●   │
│ 6. Domain Knowledge          │  ●●  │ ●●●  │  ●●●●●   │  ●●●●●   │
│ 7. Long-Context              │  ●●  │ ●●●  │  ●●●●●   │  ●●●●●   │
│ 8. Writing Quality           │  ●●  │ ●●●  │  ●●●●    │  ●●●●●   │
│ 9. Multilingual (high-res)   │  ●●  │ ●●●  │  ●●●●    │  ●●●●●   │
│ 9. Multilingual (low-res)    │  ●   │ ●●   │  ●●●     │  ●●●●    │
│ 9. Audio (multilingual)      │ ●●●●●│●●●●● │  ✗       │  ✗       │
│ 10. Throughput               │●●●●● │●●●●  │  ●       │  ●●●     │
│ 10. Memory Efficiency        │●●●●● │●●●●  │  ●●      │  ●●      │
│ 10. Fine-tune Accessibility  │●●●●  │●●●●● │  ●●      │  ●●●     │
├──────────────────────────────┼──────┼──────┼──────────┼──────────┤
│ TOTAL SCORE (75 max)         │  38  │  52  │    54    │   67     │
│ EFFICIENCY-ADJUSTED SCORE    │  58  │  72  │    42    │   55     │
└──────────────────────────────┴──────┴──────┴──────────┴──────────┘

●●●●● Excellent  ●●●● Strong  ●●● Adequate  ●● Limited  ● Poor  ✗ Unavailable

The efficiency-adjusted score is the key insight: When you weight performance by deployment cost, E4B moves from second to first. The 26B MoE drops from near-leader to third — its hardware demands are disproportionate to its advantages over E4B on most practical tasks in consumer hardware environments.

The Overlooked Argument: MoE Is Not a Middle Ground

Here is the opinion I hold most strongly, and the one most likely to be disagreed with:

The 26B MoE is not the "best of both worlds" between E4B and 31B. It is a specialized architecture designed for a specific deployment context that most developers building locally won't inhabit.

The MoE marketing narrative — "only 3.8B active parameters but 26B total quality!" — is seductive and partially true. At its intended deployment target (multi-GPU server, tensor parallel inference, high-throughput batch serving), the 26B MoE delivers exceptional efficiency. Expert parallelism means different tokens route to different GPUs, compute is distributed, and throughput scales with hardware investment.

None of that matters on a single consumer GPU or MacBook.

On a single 24GB GPU, the 26B MoE loads the entire 25.2B parameter set into memory but only activates 3.8B for each token. The memory footprint is determined by total parameters (not active parameters). The compute cost per token is roughly 3.8B-equivalent. But the routing overhead, expert loading, and lack of parallelism means you don't get the throughput benefits. You get a model that uses 18GB of RAM and runs at 2 tok/s.

The 31B Dense on the same hardware runs faster (more predictable memory access patterns, no routing overhead) and produces better output quality (no expert routing inconsistency).

My recommendation: Skip the 26B MoE unless you're deploying on multi-GPU infrastructure or accessing it via API. For local deployment, choose E4B or 31B Dense depending on your hardware. For API access, the 26B MoE is compelling — you get near-31B quality at API-level pricing (currently free on Gemini API) without any of the local hardware constraints.

This is not a criticism of the MoE architecture. It is a clarification of its deployment context. The architecture makes perfect sense at scale. It makes limited sense on a MacBook.

Deployment Scenarios with Recommendations

Scenario A: Privacy-Critical Local Application (Healthcare / Legal)

Use case: Local PII detection, medical record summarization, legal document analysis — where data cannot leave the device.

Recommendation: E4B

E4B provides sufficient domain knowledge for structured legal and medical tasks (especially with RAG), operates within 5GB RAM on any modern laptop, achieves PDPA/GDPR-compliant local processing, and is practically fine-tunable on domain-specific data. For specialized domains, E4B + fine-tuning on local corpora will outperform base 26B MoE on domain tasks in my assessment.

Scenario B: IoT / Edge / Mobile AI Feature

Use case: On-device assistant, audio transcription, real-time translation on mobile.

Recommendation: E2B

95 tok/s throughput, 1.5GB RAM, native audio support in 3+ languages. E2B is the only model in the family designed for sub-3GB RAM deployment. The quality floor is real — don't use it for complex reasoning — but for audio processing, basic instruction following, and single-document tasks, E2B's speed and size advantages are decisive.

Scenario C: API-Accessed Production Service (No Hardware Constraints)

Use case: Customer-facing AI feature, backend reasoning service, document processing pipeline via cloud.

Recommendation: 26B MoE via Gemini API for throughput-sensitive applications; 31B Dense via OpenRouter for quality-critical applications.

At API access, the hardware constraints vanish. The 26B MoE's token efficiency becomes meaningful — you're billed per output token in most APIs, and lower active parameters correlates with faster responses. The 31B Dense provides marginally better quality, particularly for complex multi-step reasoning and domain knowledge tasks.

Scenario D: Fine-Tuned Domain-Specific Application

Use case: Custom legal AI, specialized medical assistant, low-resource language application.

Recommendation: E4B as the fine-tuning target

The arXiv:2604.07035 findings, combined with the Hypa-Gemma4-E2B multilingual fine-tuning work, suggest E4B as the sweet spot for QLoRA fine-tuning: sufficient base quality, manageable parameter count, standard fine-tuning tooling support, and hardware requirements that fit within free Colab tier (A100). Fine-tuned E4B for specific domains will, in my assessment, outperform base 26B MoE on domain-specific tasks.

Scenario E: High-Stakes Research / Coding / Agentic Workflow

Use case: Multi-step reasoning agent, complex code generation, research synthesis across large document corpora.

Recommendation: 31B Dense

Function calling support is non-negotiable for agentic workflows. The 256K context window is needed for large document synthesis. The quality margin over 26B MoE on complex multi-step tasks is meaningful enough to justify the higher hardware cost. If 20GB local deployment is not feasible, use 31B Dense via API.

My Verdict

The Gemma 4 family is the most compelling open-weight model release since LLaMA 2 introduced the idea that local inference was a serious option.

But after working through this analysis, I've come to a position that will frustrate both the "just use the biggest model" camp and the "edge AI for everything" camp:

The E4B model is systematically underdeployed and the 26B MoE is systematically over-hyped for local inference contexts.

E4B at 57 tok/s, 5GB RAM, 128K context, Apache 2.0, native audio, practical QLoRA fine-tuning, and near-frontier performance on structured reasoning tasks with few-shot CoT prompting is an engineering achievement that deserves more attention than it receives. It is the model I would deploy first for the overwhelming majority of professional applications I can imagine.

The 31B Dense is clearly the quality leader. For applications where quality is the only constraint and hardware is available, it is the right choice. The Codeforces ELO of 2150, AIME 2026 score of 89.2%, and Arena AI ELO of 1452 are not marketing — they represent genuine frontier-level performance in an open-weight, Apache-licensed package.

The 26B MoE is genuinely impressive at its intended scale. I don't doubt its design is sound. My concern is with how it's being positioned for local deployment scenarios it wasn't architected to serve optimally.

And the E2B? It's the most underappreciated model in the family. At 95 tok/s and 1.5GB RAM with native audio and multimodal capabilities, it enables an entire class of edge and mobile AI applications that simply didn't exist at this quality level before. The community building on top of E2B is just getting started.

My final selection guide, in one sentence each:

• E2B: When size and speed are constraints and audio is required.
• E4B: When quality matters, hardware is limited, and fine-tuning is on the roadmap.
• 26B MoE: When accessing via API or deploying on multi-GPU infrastructure.
• 31B Dense: When quality is the only variable that matters.

Key Takeaways

• ✅ All four variants share Apache 2.0 licensing — the first Gemma generation with full commercial freedom, no MAU caps, patent grants included
• ✅ E4B is the efficiency sweet spot — 57 tok/s, 5GB RAM, 128K context, near-frontier reasoning with few-shot CoT
• ✅ 31B Dense is the quality leader — AIME 89.2%, Arena AI #3, Codeforces 2150, only variant with reliable function calling
• ✅ 26B MoE is a server architecture, not a laptop architecture — 2 tok/s on 24GB consumer hardware; save it for API access or multi-GPU deployment
• ✅ E2B/E4B are the only variants with audio — structural advantage for multilingual voice applications
• ⚠️ Multilingual support ≠ multilingual quality — "140+ languages" degrades severely for low-resource languages (Tamil, Sinhala); fine-tuning is required for production quality
• ⚠️ 26B MoE is difficult to fine-tune — router weight sensitivity makes QLoRA non-trivial; E4B is the better fine-tuning target
• ⚠️ No variant should be deployed without RAG for factual-accuracy-critical applications — TruthfulQA scores are insufficient across the family for high-stakes factual retrieval
• 🔍 Efficiency-adjusted scoring reverses the naive ranking — when hardware cost is factored, E4B ranks first, 26B MoE drops to third
• 🔍 For privacy-regulated workflows, E2B/E4B's on-device deployment is a legal architecture, not just a technical preference

References and Sources

• Google DeepMind. (2026). Gemma 4 Model Card.
• Google DeepMind. (2026). Run Gemma with the Gemini API.(https://ai.google.dev/gemma/docs/core/gemma_on_gemini_api)
• Anass Kartit. (2026, April 3). I Tested Every Gemma 4 Model Locally on My MacBook. kartit.net
• Labellerr. (2026, April 8). Google Gemma 4: A Technical Overview.
• Lushbinary. (2026, April 3). Gemma 4 Developer Guide: Benchmarks & Local Setup.
• arXiv:2604.07035. (2026). Gemma 4, Phi-4, and Qwen3: Accuracy-Efficiency Tradeoffs in Dense and MoE Reasoning Language Models
• arXiv:2407.21330. (2024). Jayakody & Dias. Performance of Recent Large Language Models for a Low-Resourced Language (Sinhala).
• arXiv:2602.14517. (2026). Beyond Translation: Evaluating Mathematical Reasoning Capabilities of LLMs in Sinhala and Tamil

Methodological note: This analysis does not include original experimental benchmarks run by the author. All performance figures are sourced from the references above. The analytical conclusions, interpretations, and recommendations are the author's independent opinion.