By now, you would have definitely come across either Andrej Karpathy’s LLM Wiki design or some version of it. (And there are some versions with a lot of enhancements too!) TL;DR version of this post is: I built a system based on that proposal but made it cheaper and more controlled. Problem with PKMs and Karpathy’s solution Most knowledge workflows break down after collection. Saving documents is easy. Building a system that keeps them organized, connected, and queryable over time is much harder. Search helps, but search alone does not create structure. Traditional RAG systems improve retrieval, but they still force the model to reconstruct meaning every time we ask a question. Moreover when the knowledgebase starts expanding and related concepts sit in multiple files, RAG becomes slow and expensive. Proposed solution: a self-evolving LLM-managed wiki Replace “query-time-only RAG” with a persistent, LLM-maintained markdown wiki that compounds over time. Raw sources stay immutable; the wiki is the synthesized layer that keeps getting updated as new sources arrive. Human curates sources and asks questions; LLM handles maintenance (summaries, cross-links, updates, consistency work). Graph view in Obsidian How does it work? Ingest: process one or more sources, update multiple wiki pages, append logs. Query: answer from wiki first, and optionally file good answers back into wiki. Lint: periodic health checks (contradictions, stale claims, orphan pages, missing concepts/links). Why it matters It front-loads synthesis once and reuses it, instead of rediscovering from raw chunks on every question. It treats knowledge work as a maintained artifact, not disposable chat outputs. This is his instruction file for the LLMs. It gives entire responsibility of building and managing the wiki to the LLM. My take: A hybrid solution My problem with this approach is that it gives too much to LLM to do. (I know it is easy to hand it over to an agent to manage everything and in this day and age, writing code instead of using an agent seems a little too old-school.) What were the problems with a pure LLM approach? Cost: dramatically higher. Each pipeline stage would require API calls. Even with a cheap model, ingesting hundreds of documents through an agent orchestrator would rapidly become expensive. This local LLM approach is nearly free at scale. Non-determinism: agents adapt and reason differently each run. Same input might produce slightly different concept extractions, different merges, different quality decisions. Code-based pipeline produces consistent, reproducible results. Latency: API calls add network round-trips. Local pipeline can process in parallel. An agent would be sequential and slow. Predictable failure: Python code fails in expected ways. Agents fail by hallucinating strategies or getting stuck in loops. Then we would be debugging prompts instead of code. On the other hand, a pure LLM system would be easy to setup. You just dump the instruction file and you are done! My pipeline is a hybrid solution which uses a pipeline composed of python scripts each using LLM calls for different tasks: turning raw documents into a persistent, evolving knowledge layer and making that layer easy to explore visually and interactively. That last part is owed to Obsidian! https://medium.com/media/958197a742d2f8ed2bcd27bc8619e691/href The Core Idea The goal is to transform raw inputs into a structured, linked, wiki-like knowledge graph. Each stage of the pipeline converting the documents and storing them in separate folders. At a high level: raw stores source documents processed stores cleaned and enriched markdown wiki stores concept pages To make compiled knowledge queryable, rather than asking LLM to look at all the documents, it uses an ‘index’ as the routing layer for query-time access. Where Obsidian fits Obsidian is a key part of the workflow. The generated wiki is stored as markdown with Obsidian-style `[[links]]`, so the output is not locked inside a database or a custom UI. It remains transparent, editable, portable, and easy to inspect. It gives a human-friendly interface to the knowledge graph. Instead of seeing isolated outputs, I can open the vault in Obsidian, follow links between concepts, use backlinks, and visually inspect how ideas connect. In practice, this means Obsidian is not just a note-taking tool in this project. It is the front end for our continuously evolving knowledgebase. How the pipeline works The first stage is ingestion. compile.py reads markdown documents from raw , extracts metadata, downloads images when needed, rewrites image links locally, and uses a local LLM to convert the source into structured markdown. The output is saved into processed . Then wiki_generator.py takes over. It reads the processed files, extracts durable concepts, normalizes names, deduplicates overlapping ideas, and merges new findings into existing concept pages. Instead of creating isolated summaries, it builds a concept-centric wiki that evolves over time. After that, auto_linker.py scans the wiki and inserts `[[Concept]]` links across pages, turning individual files into a navigable graph inside Obsidian. resolve_ghost_concepts.py helps fill in missing linked concepts, and knowledge_linter.py acts as a quality layer that flags weak pages, duplicates, and missing concepts. The overall result is a knowledge base that is not just stored, but actively maintained. All these elements of the pipeline are embedded in run_pipeline.py which runs every ten minutes using a simple cron task. This is like an agent monitoring the file base and starting the processing. Only cheaper and more reliably. Using local LLMs A major design choice was keeping the system local-first. The pipeline uses local LLMs for ingestion, concept extraction, merging, and linting. That keeps the workflow private, predictable, and independent of external APIs. It also makes the system practical for internal documentation or sensitive personal knowledge bases. The LLM is not being used as a generic chatbot over files. It is being used as a knowledge compiler. Querying the wiki The query model is one of the most important parts. Instead of asking an LLM to search all raw documents directly, the system relies on an index-driven workflow. The `index.md` helps route the query to the most relevant concept pages. The answering model then reads only those selected pages and synthesizes a response with citations. Below instructions can be put in an file for using either with a CLI based LLM agent (Claude Code or Copilot-cli). # Role You are the Librarian for my Obsidian Vault. # Scope Lock (strict) - You are answering ONLY from this workspace vault. - Allowed sources: wiki/index.md, wiki/, processed/, raw/ - Do not use external/general knowledge unless the user explicitly asks for it. # Term Resolution Rules - Never expand an acronym from prior knowledge. - For any acronym (e.g., UAS), first search index.md and vault pages for its local meaning. - If multiple meanings exist, list them and ask which one. - If no evidence exists in vault, reply exactly: “This term is not defined in the current vault.” # Evidence Requirement - Every factual claim must be supported by selected vault pages. - If evidence is insufficient, say so explicitly; do not guess. # Query Workflow 1. Parse question terms. 2. Search index.md for term/alias match. 3. Read top 3–6 relevant pages only. 4. Answer in a balanced way. 5. Cite exact page names used. # Knowledge Source 1. Use wiki/index.md as the routing catalog for concepts and page links. 2. Do not read the entire vault. Select only relevant pages for each query. # Query Workflow 1. Parse the user question into key terms/entities. 2. Scan wiki/index.md for matching concepts (keyword/alias match first). 3. Select the top 3–6 most relevant pages and list them. 4. Read only those pages and synthesize the answer. 5. If evidence is insufficient, say so explicitly instead of guessing. # Output Format - Answer: - balanced response grounded in selected pages (clear and complete, not overly long) - Citations: - include the exact page names used (and quoted lines when possible) # Missing Knowledge Handling - If no matching concept exists in index.md, state: “This concept is missing from the current wiki/index.” - Optionally suggest which new concept/page should be added. Here is the entire code: https://github.com/Waterfox83/obsidian-wiki Go ahead and give it a try. Let me know how it works out for you and what enhancements would you like in this pipeline. Building a Local-first Knowledge Management System with LLM and Obsidian was originally published in GoPenAI on Medium, where people are continuing the conversation by highlighting and responding to this story.