惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

V
Visual Studio Blog
Stack Overflow Blog
Stack Overflow Blog
G
Google Developers Blog
Microsoft Azure Blog
Microsoft Azure Blog
Engineering at Meta
Engineering at Meta
L
LangChain Blog
T
The Blog of Author Tim Ferriss
J
Java Code Geeks
Y
Y Combinator Blog
月光博客
月光博客
雷峰网
雷峰网
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
F
Fortinet All Blogs
A
About on SuperTechFans
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
P
Proofpoint News Feed
小众软件
小众软件
H
Help Net Security
Last Week in AI
Last Week in AI
B
Blog RSS Feed
宝玉的分享
宝玉的分享
N
Netflix TechBlog - Medium
博客园 - 叶小钗
The GitHub Blog
The GitHub Blog

DEV Community

Authentication Security Deep Dive: From Brute Force to Salted Hashing (With Java Examples) Why AI Systems Don’t Fail — They Drift Spilling beans for how i learn for exam😁"Reinforcement Learning Cheat Sheet" I Replaced Chrome with Safari for AI Browser Automation. Here's What Broke (and What Finally Worked) How Python Borrows Other People's Work The $40 Architecture: Processing 1 Billion API Requests with 99.99% Uptime Vibe Coding: A Workflow Guide (From Zero to SaaS) Most webhook security guides protect the wrong side. The scary part is delivery. Headless CMS for TanStack Start: Build a Blog with Cosmic EU Age Verification App "Hacked in 2 Minutes" — What Actually Happened Comfy Cloud’s delete function does not actually remove files Running AI Models on GPU Cloud Servers: A Beginner Guide Event-driven media intelligence with AWS Step Functions and Bedrock I scored 500 AI prompts across 8 quality dimensions — here's what broke How to Call Google Gemini API from Next.js (Free Tier, No Backend Needed) The Portal Protocol: Reclaiming Human Connection in the Age of AI How to Fix Your Team's Scattered Knowledge Problem With a Self-Hosted Forum Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud Designing Multi-Tenant Backends With Both Ownership and Team Access I Built a Neumorphic CSS Library with 77+ Components — Here's What I Learned PostgreSQL Performance Optimization: Why Connection Pooling Is Critical at Scale Cómo construí un SaaS multi-rubro para gestionar expensas en Argentina con FastAPI + Vue 3 🚀 I Built an Ethical Hacking Scanner Tool – Open Source Project I Replaced /usage and /context in Claude Code With a Single Statusline A Pythonic Way to Handle Emails (IMAP/SMTP) with Auto-Discovery and AI-Ready Design I Collected 8.9 Million Polymarket Price Points — Here's What I Found About How Markets Really Move EcoTrack AI — Carbon Footprint Tracker & Dashboard Everyone's Using AI. No One Agrees How. 5 self-hosted ebook managers worth trying in 2026 Building Your First AI Agent with LangChain: From Chatbot to Autonomous Assistant
# How I Found Out 52% of My Knowledge Graph Was Duplicate...
Ernesto Arias · 2026-06-25 · via DEV Community

Ernesto Arias

I've spent the last several months building ANIMUS, an autonomous system in Rust that gives a local LLM persistent memory. The idea is simple: a knowledge graph that grows on its own, cycle after cycle, as the system reads documents, detects gaps in its knowledge, and fills them in.

For months, the metric I watched most closely was the node count of the graph. It kept climbing. I felt good about that.

Until I ran a full audit and found out that 52% of those nodes were undetected duplicates. Of 1,892 reported nodes, only 911 were actually unique.

How did this happen?

ANIMUS's autonomous loop actively looks for "gaps" — holes in its knowledge that the system decides to fill on its own. The problem: an overly aggressive filter was excluding certain categories from the gap pool, which trapped the system in a loop of re-exploring the same ~40 topics for thousands of cycles. Each pass generated content that was similar but not identical to the last — different enough to avoid triggering any exact-duplicate check, but substantially the same information rephrased.

The node count kept climbing. Actual knowledge, not so much.

The Rust engineering side

The fix wasn't magic, it was audit work:

  • Reopening the gap filter that had been closed too aggressively, so the system would explore genuinely new topics instead of repeating itself.
  • Fixing a recency bias in the semantic search (Brain::search): it walked the graph from node 0 with .take(2), which meant it almost always returned stale content from earlier versions of the system. A simple .rev() fixed it.
  • Building an "auto-census" process that runs every 37 cycles and generates real statistics about the graph by category — so the system itself (and I) could see with numbers, not intuition, whether it was growing in a healthy way.

Along the way, I also migrated the inference engine: from a Python wrapper to a llama-server.exe launched directly from Rust, and from the original model to a quantized Gemma 4 E2B, running at ~77 tokens/second on a consumer GPU (RTX 3050, 4GB). None of this required the cloud or paid APIs — everything runs locally.

What I learned

The most valuable part of this whole episode wasn't fixing the bug. It was realizing that a metric that only goes up never warns you that something is wrong. Node count was a proxy for "the system is learning," but optimizing that one proxy, with nothing to balance it, ended up producing the opposite: inflated content, not new knowledge.

ANIMUS now runs on several cross-checked signals (verified uniqueness, recency-weighted relevance, source validation) instead of one vanity metric. If two signals start to diverge, the system stops and re-audits instead of continuing to generate.

If you're curious about the full picture (architecture, benchmarks, comparison against a simple vector RAG baseline), the technical paper is open access with a DOI: 10.5281/zenodo.20674981. Code is on GitHub.


ANIMUS is an independent project, developed in Santo Domingo, Dominican Republic.