An inference architecture that makes LLMs stateful.

No chat history reprocessing. No RAG. No prompt replay. The model sends only the current message, yet remembers everything.

The Problem

Every time you send a message to an LLM, the entire conversation history is re-transmitted and re-processed through every layer of the model. Turn 1 costs X. Turn 10 costs 10X. Turn 50 costs 50X.

This means:

• Cost scales linearly with conversation length
• Latency grows as context accumulates
• Long-term memory is economically impossible — providers either truncate context, summarize (losing information), or charge per-token for the privilege of the model re-reading what it already processed

The entire industry accepts this as a fundamental constraint of the transformer architecture. It isn't.

What NLS Does

When a message is processed by an LLM, it passes through dozens of attention and recurrent layers, producing intermediate numerical representations (key-value tensors, state-space model states) that encode the model's understanding of that message.

NLS captures those representations, stores them on disk, and re-injects them directly into the model's cache on the next request.

From the model's perspective, the injected states are indistinguishable from having just processed the original text. The model "remembers" without recomputing.

The result:

• Each request sends only the new user message — no history, no prior context
• Prior understanding is loaded from cheap storage (NVMe SSD at $0.10/GB) instead of recomputed on expensive HBM ($30-40/GB)
• Token savings compound with conversation length: 90%+ savings at 50 turns
• The model produces functionally equivalent output to full-context processing

What NLS Is NOT

Architecture

NLS operates as a cycle with five phases:

 User Message
      │
      ▼
 ┌─────────────┐     ┌──────────────┐
 │  Retrieval   │────▶│   Injection   │──── Phantom tokens written
 │  BM25+Embed  │     │  RoPE-correct │     to paged KV cache
 └─────────────┘     └──────┬───────┘
                            │
                            ▼
                    ┌───────────────┐
                    │   Scoring     │──── Model's own attention
                    │   Q @ K       │     ranks memory relevance
                    └──────┬────────┘
                           │
                           ▼
                    ┌───────────────┐
                    │  Monitoring   │──── Hidden-state probes
                    │  Live Swap    │     hot-swap during decode
                    └──────┬────────┘
                           │
                           ▼
                    ┌───────────────┐
                    │   Capture     │──── New states saved
                    │   Quantize    │     to persistent storage
                    └───────────────┘

For the full architecture breakdown, see ARCHITECTURE.md.

Key Technical Contributions

1. Phantom Token Injection The inference engine allocates KV cache positions for "phantom tokens" — positions that receive stored K/V values without any text being processed. The model's attention operates over both injected and real tokens seamlessly.

2. Attention-Based Neural Scoring During prefill, the model's own Q@K attention patterns score each injected memory for relevance. Low-relevance memories have their value vectors zeroed in the cache (V-suppression), preventing them from degrading output quality.

3. Real-Time Memory Hot-Swapping During token generation, the system monitors the model's hidden states and can swap memory contents in reserved "register slots" mid-generation — no restart needed. If the model's response drifts to a topic not covered by the initial memory selection, relevant memories are loaded and injected in real-time.

4. Hybrid Architecture Support NLS handles both standard attention layers (K/V tensors) and recurrent layers (DeltaNet/Mamba state-space models). A two-pass capture method ensures clean attention states while preserving compounded recurrent narrative context.

5. Cross-Session Persistence Memories persist on disk across sessions, server restarts, and model reloads. The filesystem is the single source of truth, with boot-time reconciliation rebuilding indices from stored file manifests.

6. Multi-Signal Quality Filtering Memory quality is assessed using both language-aware signals and model-native signals derived from internal state geometry. The signals are combined for robustness — neither alone can hide a low-quality memory. This filtering operates at capture time (junk never enters the pool) and at retrieval time (low-quality memories never out-rank facts).

7. Agent-Mode Tool Memory Auto-detection of OpenAI-compatible function-calling requests. Tool result messages (from API calls, deployments, file reads) are captured with chain metadata. When an agent returns to a project after context compaction, retrieval surfaces the user's prior memory and chain-walk pulls back the operational details (IPs, configs, deploy commands) — solving the agent context-loss failure mode.

Results

Try It

Conversational demo

Punk Records Demo — A personal AI assistant powered by NLS.

Chat with it, build up a conversation, close the tab, come back. It remembers. Open the API Log drawer at the bottom to see proof: every request sends only the current message.

Agentic demo (validated April 27, 2026)

NLS has been validated end-to-end behind a real coding agent (OpenCode TUI driving a multi-phase scaffold task). After Phase 1 was complete, fully restarting the TUI and asking "what frontend port did we pick?" in a fresh chat — with zero chat history — returned the correct answer (3000) by injecting 18,751 tokens of stored context from a 124-token prompt. 99.3% prompt-token savings on the recall path, with disambiguation between close facts (frontend 3000 vs backend 3001) ruling out lexical-prior hallucination. See BENCHMARKS for full methodology.

Read More

• ARCHITECTURE.md — Technical architecture with diagrams
• JOURNEY.md — The story: from LoRA adapters to MoE routing to KV injection
• RESEARCH_LOG.md — Curated log of 4 months of experimentation
• benchmarks/BENCHMARKS.md — Token savings, text-vs-KV parity, retrieval quality, 67-Q and 18-Q sweep results
• benchmarks/ECONOMICS.md — Unit economics, energy impact, GPU market implications, subscription model viability
• diagrams/ — Architecture and economics diagrams (Mermaid source, render to PNG)

About

NLS was built by Umberto Canessa Cerchi over ~3 months of after-hours research (650+ research log entries). Two months of exploration (LoRA, MoE routing) led to the key insight; the final architecture went from proof-of-concept to production demo in 8 days, then through subsequent refinement leading to the first end-to-end agentic-loop validation on April 27, 2026.

It runs on a single NVIDIA Grace Blackwell desktop GPU.

The core plugin is proprietary. This repository contains the architecture documentation, research narrative, benchmarks, and a live demo.

Patent pending — U.S. Provisional Patent Application No. 64/050,345 filed April 27, 2026.

"The best way to predict the future is to invent it." — Alan Kay