转自https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f

LLM Wiki

一种利用大语言模型（LLM）构建个人知识库的模式。

这是一个概念文件，旨在供你复制并发送给自己的 LLM Agent（例如 OpenAI Codex、Claude Code、OpenCode / Pi 等）。其目标是传达核心构想，而你的 Agent 将与你协作完成具体的构建工作。

核心构想

大多数人使用 LLM 处理文档的体验类似于 RAG（检索增强生成）：你上传一组文件，LLM 在查询时检索相关的文本块并生成答案。这种方式虽然有效，但 LLM 在处理每个问题时都在“从零开始”重新发现知识，缺乏知识的累积。如果你问一个需要综合五个文档的微妙问题，LLM 每次都必须重新寻找并拼接这些零散的片段。没有任何东西被沉淀下来。NotebookLM、ChatGPT 文件上传以及大多数 RAG 系统都是这种工作模式。

本方案则完全不同。LLM 不仅仅是在查询时从原始文档中检索，而是增量式地构建并维护一个持久化的 Wiki —— 这是一个介于你和原始素材之间的、结构化且相互链接的 Markdown 文件集合。当你添加新素材时，LLM 不只是为了以后的检索而对其进行索引，而是会阅读它，提取关键信息，并将其整合到现有的 Wiki 中：更新实体页面、修订主题摘要、标记新数据与旧主张的冲突之处，从而强化或挑战正在演进的综合认知。知识被“编译”一次后便持续保持最新状态，而不是在每次查询时重新推导。

这就是关键区别：Wiki 是一个持久化的、具有复利效应的产物。交叉引用已经存在，冲突已被标记，综合论述已经反映了你阅读过的所有内容。随着你添加的每一个素材和提出的每一个问题，Wiki 都会变得愈发丰富。

你从不（或极少）亲自编写 Wiki —— LLM 负责所有的编写和维护工作。你负责提供素材、进行探索并提出正确的问题。LLM 负责所有的繁琐工作：摘要、交叉引用、归档和记录，正是这些工作让知识库在长期使用中真正发挥价值。在实践中，我通常在屏幕一侧开启 LLM Agent，另一侧开启 Obsidian。LLM 根据我们的对话进行编辑，我则实时浏览结果 —— 点击链接、查看关系图谱（Graph View）、阅读更新后的页面。Obsidian 是 IDE，LLM 是程序员，而 Wiki 则是代码库。

该模式可应用于多种场景，例如：

• 个人层面：追踪个人目标、健康、心理、自我提升 —— 归档日志、文章、播客笔记，随时间推移构建出关于自我的结构化图景。
• 研究领域：在数周或数月内深入研究某一主题 —— 阅读论文、文章、报告，增量构建一个带有演进论点的全面 Wiki。
• 读书笔记：边读边归档每一章节，构建人物、主题、情节线索及其关联的页面。读完后，你就拥有了一个丰富的配套 Wiki。想象一下像 Tolkien Gateway 这样的粉丝维基 —— 由志愿者社区多年构建的数千个互联页面。你可以在阅读时利用 LLM 负责所有的交叉引用和维护，亲自构建类似的东西。
• 企业/团队：由 LLM 维护的内部 Wiki，输入源包括 Slack 会话、会议纪要、项目文档、客户电话。可以引入人工审核更新。Wiki 能保持最新，是因为 LLM 承担了团队中没人愿意做的维护工作。
• 竞品分析、尽职调查、旅行规划、课程笔记、爱好者深度研究 —— 任何你需要长期积累知识并希望其井然有序而非零散分布的场景。

架构

架构分为三层：

1. 原始素材 (Raw sources) —— 你精心挑选的源文档集合。包括文章、论文、图片、数据文件。这些是不可变的 —— LLM 只读取它们，从不修改。这是你的事实来源（Source of Truth）。
2. Wiki 层 —— 由 LLM 生成的 Markdown 文件目录。包含摘要、实体页面、概念页面、对比分析、综述和综合论证。LLM 完全拥有这一层。它创建页面、在新素材进入时更新页面、维护交叉引用，并保持全局一致性。你阅读，LLM 编写。
3. 模式层 (The schema) —— 一个定义文档（例如 Claude Code 的 <code>CLAUDE.md</code> 或 Codex 的 <code>AGENTS.md</code>），用于告知 LLM Wiki 的结构、约定以及在摄取素材、回答问题或维护 Wiki 时应遵循的工作流。这是关键的配置文件 —— 它使 LLM 成为一名纪律严明的 Wiki 维护者，而非通用的聊天机器人。随着你摸索出适合该领域的方案，你和 LLM 会共同演进这个文件。

操作流程

摄取 (Ingest)：将新素材放入原始集合并要求 LLM 处理。典型流程如下：LLM 阅读素材，与你讨论关键要点，在 Wiki 中编写摘要页，更新索引，更新整个 Wiki 中相关的实体和概念页面，并在日志中追加一条记录。单一素材可能会触达 10-15 个 Wiki 页面。我个人倾向于逐一摄取素材并保持参与 —— 阅读摘要、检查更新，并引导 LLM 强调重点。但你也可以在较少监管的情况下批量摄取多个素材。你可以根据自己的风格开发工作流，并将其记录在模式层中。

查询 (Query)：针对 Wiki 提问。LLM 搜索相关页面，阅读并综合出带有引用出处的答案。答案可以根据问题采取不同形式 —— Markdown 页面、对比表、幻灯片 (Marp)、图表 (Matplotlib) 或画布 (Canvas)。核心洞察是：高质量的答案可以作为新页面归档回 Wiki。你要求的对比、分析、发现的关联 —— 这些都是宝贵的资产，不应消失在对话历史中。这样，你的探索过程就像摄取的素材一样，在知识库中产生复利。

巡检 (Lint)：定期要求 LLM 对 Wiki 进行健康检查。查找：页面间的冲突、已被新素材取代的过时主张、没有入站链接的孤儿页面、提到但缺乏专属页面的重要概念、缺失的交叉引用，以及可以通过网页搜索填补的数据空白。LLM 擅长建议新的研究问题和寻找新素材的方向。这能确保 Wiki 在增长过程中保持健康。

索引与日志

两个特殊文件可帮助 LLM（及你）在 Wiki 增长时进行导航，它们各有用途：

• index.md 是以内容为中心的。它是 Wiki 中所有内容的目录 —— 列出每个页面及其链接、单行摘要，以及可选的元数据（如日期或来源数量）。按类别（实体、概念、素材等）组织。LLM 在每次摄取时更新它。回答查询时，LLM 先阅读索引找到相关页面，再深入挖掘。这种方式在中等规模（约 100 个素材，数百个页面）下表现惊人，且无需基于向量嵌入（Embedding）的 RAG 基础设施。
• log.md 是以时间为序的。它是操作过程的只增记录 —— 包含摄取、查询、巡检。一个小技巧：如果每条条目都以固定的前缀开始（例如 <code>## [2026-04-02] ingest | 文章标题</code>），日志就可以通过简单的 Unix 工具进行解析 —— <code>grep "^## \[" log.md | tail -5</code> 即可获取最后 5 条记录。日志提供了 Wiki 演进的时间线，帮助 LLM 了解近期开展的工作。

可选：命令行工具 (CLI)

在某些阶段，你可能想构建一些小工具来帮助 LLM 更高效地操作 Wiki。最显而易见的是针对 Wiki 页面的搜索引擎 —— 在小规模下，索引文件足够了，但随着 Wiki 增长，你需要真正的搜索。qmd 是一个不错的选择：它是一个针对 Markdown 文件的本地搜索引擎，支持 BM25/向量混合搜索和 LLM 重排序，全部在本地运行。它既有 CLI（方便 LLM 调用 Shell 运行），也有 MCP 服务（方便 LLM 作为原生工具使用）。你也可以自己构建更简单的工具 —— LLM 可以根据需要帮你快速编写一个简单的搜索脚本。

提示与技巧

• Obsidian Web Clipper：一个浏览器扩展，可将网页文章转换为 Markdown。对于快速将素材放入原始集合非常有用。
• 本地下载图片：在 Obsidian 设置 → 文件与链接中，将“附件默认存放路径”设为固定目录（如 <code>raw/assets/</code>）。然后在设置 → 快捷键中搜索“下载”，找到“下载当前文件的附件”并绑定快捷键（如 <code>Ctrl+Shift+D</code>）。剪藏文章后，按下快捷键，所有图片都会下载到本地。这虽然是可选的，但非常有用 —— 它让 LLM 能够直接查看和引用图片，而非依赖可能失效的 URL。注意，LLM 通常无法在一次处理中原生读取带有内嵌图片的 Markdown —— 解决方法是让 LLM 先阅读文本，然后根据需要单独查看部分或全部引用的图片以获取额外上下文。虽然有些繁琐，但效果不错。
• Obsidian 关系图谱 (Graph View)：观察 Wiki 形态的最佳方式 —— 哪些页面是互联的，哪些是核心枢纽，哪些是孤儿页面。
• Marp：一种基于 Markdown 的幻灯片格式。Obsidian 有对应的插件。适合直接根据 Wiki 内容生成演示文稿。
• Dataview：一个通过页面元数据（Frontmatter）运行查询的 Obsidian 插件。如果你的 LLM 为 Wiki 页面添加了 YAML 元数据（标签、日期、素材计数），Dataview 可以生成动态表格和列表。
• 版本控制：Wiki 本质上只是一个存放 Markdown 文件的 Git 仓库。你可以免费获得版本历史、分支管理和协作功能。

为什么这套方案有效

维护知识库最枯燥的部分不是阅读或思考，而是记录与整理（Bookkeeping）。更新交叉引用、保持摘要最新、标注新旧数据冲突、在几十个页面间保持一致性。人类之所以放弃 Wiki，是因为维护成本的增长速度超过了价值产出。LLM 不会感到厌倦，不会忘记更新交叉引用，并且可以在一次操作中修改 15 个文件。Wiki 能够得到维护，是因为维护成本几乎降到了零。

人类的工作是筛选素材、指引分析、提出好问题，并思考这一切的意义。LLM 负责其余的所有工作。

这个想法在精神上与范内瓦·布什（Vannevar Bush）1945 年提出的 Memex 相关 —— 一种个人化的、精心策展的知识存储，文档之间通过联想路径相连。布什的愿景比现代互联网更接近于此：私密的、积极策展的，且文档间的连接与文档本身同样宝贵。他唯一无法解决的问题是谁来负责维护，而 LLM 解决了这一点。

备注

本文档有意保持抽象。它描述的是一种模式，而非特定的实现。具体的目录结构、模式约定、页面格式和工具选择 —— 所有这些都取决于你的领域、偏好以及你选择的 LLM。上文提到的所有内容都是可选且模块化的 —— 选取有用的，忽略无用的。例如：如果你的素材全是文本，则完全不需要处理图片。如果你的 Wiki 规模较小，索引文件就足够了，不需要搜索引擎。你可能不在乎幻灯片演示，只需要 Markdown 页面。你可能需要完全不同的输出格式。使用此文档的正确方式是将其分享给你的 LLM Agent，并共同协作实例化一个符合你需求的版本。本文档唯一的任务就是传达这种模式，剩下的交给 LLM。

---

LLM Wiki

A pattern for building personal knowledge bases using LLMs.

This is an idea file, it is designed to be copy pasted to your own LLM Agent (e.g. OpenAI Codex, Claude Code, OpenCode / Pi, or etc.). Its goal is to communicate the high level idea, but your agent will build out the specifics in collaboration with you.

The core idea

Most people’s experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There’s no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.

The idea here is different. Instead of just retrieving from raw documents at query time, the LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files that sits between you and the raw sources. When you add a new source, the LLM doesn’t just index it for later retrieval. It reads it, extracts the key information, and integrates it into the existing wiki — updating entity pages, revising topic summaries, noting where new data contradicts old claims, strengthening or challenging the evolving synthesis. The knowledge is compiled once and then kept current, not re-derived on every query.

This is the key difference: the wiki is a persistent, compounding artifact. The cross-references are already there. The contradictions have already been flagged. The synthesis already reflects everything you’ve read. The wiki keeps getting richer with every source you add and every question you ask.

You never (or rarely) write the wiki yourself — the LLM writes and maintains all of it. You’re in charge of sourcing, exploration, and asking the right questions. The LLM does all the grunt work — the summarizing, cross-referencing, filing, and bookkeeping that makes a knowledge base actually useful over time. In practice, I have the LLM agent open on one side and Obsidian open on the other. The LLM makes edits based on our conversation, and I browse the results in real time — following links, checking the graph view, reading the updated pages. Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase.

This can apply to a lot of different contexts. A few examples:

• Personal: tracking your own goals, health, psychology, self-improvement — filing journal entries, articles, podcast notes, and building up a structured picture of yourself over time.
• Research: going deep on a topic over weeks or months — reading papers, articles, reports, and incrementally building a comprehensive wiki with an evolving thesis.
• Reading a book: filing each chapter as you go, building out pages for characters, themes, plot threads, and how they connect. By the end you have a rich companion wiki. Think of fan wikis like Tolkien Gateway — thousands of interlinked pages covering characters, places, events, languages, built by a community of volunteers over years. You could build something like that personally as you read, with the LLM doing all the cross-referencing and maintenance.
• Business/team: an internal wiki maintained by LLMs, fed by Slack threads, meeting transcripts, project documents, customer calls. Possibly with humans in the loop reviewing updates. The wiki stays current because the LLM does the maintenance that no one on the team wants to do.
• Competitive analysis, due diligence, trip planning, course notes, hobby deep-dives — anything where you’re accumulating knowledge over time and want it organized rather than scattered.

Architecture

There are three layers:

Raw sources — your curated collection of source documents. Articles, papers, images, data files. These are immutable — the LLM reads from them but never modifies them. This is your source of truth.

The wiki — a directory of LLM-generated markdown files. Summaries, entity pages, concept pages, comparisons, an overview, a synthesis. The LLM owns this layer entirely. It creates pages, updates them when new sources arrive, maintains cross-references, and keeps everything consistent. You read it; the LLM writes it.

The schema — a document (e.g. CLAUDE.md for Claude Code or AGENTS.md for Codex) that tells the LLM how the wiki is structured, what the conventions are, and what workflows to follow when ingesting sources, answering questions, or maintaining the wiki. This is the key configuration file — it’s what makes the LLM a disciplined wiki maintainer rather than a generic chatbot. You and the LLM co-evolve this over time as you figure out what works for your domain.

Operations

Ingest. You drop a new source into the raw collection and tell the LLM to process it. An example flow: the LLM reads the source, discusses key takeaways with you, writes a summary page in the wiki, updates the index, updates relevant entity and concept pages across the wiki, and appends an entry to the log. A single source might touch 10-15 wiki pages. Personally I prefer to ingest sources one at a time and stay involved — I read the summaries, check the updates, and guide the LLM on what to emphasize. But you could also batch-ingest many sources at once with less supervision. It’s up to you to develop the workflow that fits your style and document it in the schema for future sessions.

Query. You ask questions against the wiki. The LLM searches for relevant pages, reads them, and synthesizes an answer with citations. Answers can take different forms depending on the question — a markdown page, a comparison table, a slide deck (Marp), a chart (matplotlib), a canvas. The important insight: good answers can be filed back into the wiki as new pages. A comparison you asked for, an analysis, a connection you discovered — these are valuable and shouldn’t disappear into chat history. This way your explorations compound in the knowledge base just like ingested sources do.

Lint. Periodically, ask the LLM to health-check the wiki. Look for: contradictions between pages, stale claims that newer sources have superseded, orphan pages with no inbound links, important concepts mentioned but lacking their own page, missing cross-references, data gaps that could be filled with a web search. The LLM is good at suggesting new questions to investigate and new sources to look for. This keeps the wiki healthy as it grows.

Indexing and logging

Two special files help the LLM (and you) navigate the wiki as it grows. They serve different purposes:

index.md is content-oriented. It’s a catalog of everything in the wiki — each page listed with a link, a one-line summary, and optionally metadata like date or source count. Organized by category (entities, concepts, sources, etc.). The LLM updates it on every ingest. When answering a query, the LLM reads the index first to find relevant pages, then drills into them. This works surprisingly well at moderate scale (~100 sources, ~hundreds of pages) and avoids the need for embedding-based RAG infrastructure.

log.md is chronological. It’s an append-only record of what happened and when — ingests, queries, lint passes. A useful tip: if each entry starts with a consistent prefix (e.g. <code>## [2026-04-02] ingest | Article Title</code>), the log becomes parseable with simple unix tools — <code>grep "^## \[" log.md | tail -5</code> gives you the last 5 entries. The log gives you a timeline of the wiki’s evolution and helps the LLM understand what’s been done recently.

Optional: CLI tools

At some point you may want to build small tools that help the LLM operate on the wiki more efficiently. A search engine over the wiki pages is the most obvious one — at small scale the index file is enough, but as the wiki grows you want proper search. qmd is a good option: it’s a local search engine for markdown files with hybrid BM25/vector search and LLM re-ranking, all on-device. It has both a CLI (so the LLM can shell out to it) and an MCP server (so the LLM can use it as a native tool). You could also build something simpler yourself — the LLM can help you vibe-code a naive search script as the need arises.

Tips and tricks

• Obsidian Web Clipper is a browser extension that converts web articles to markdown. Very useful for quickly getting sources into your raw collection.
• Download images locally. In Obsidian Settings → Files and links, set "Attachment folder path" to a fixed directory (e.g. <code>raw/assets/</code>). Then in Settings → Hotkeys, search for "Download" to find "Download attachments for current file" and bind it to a hotkey (e.g. Ctrl+Shift+D). After clipping an article, hit the hotkey and all images get downloaded to local disk. This is optional but useful — it lets the LLM view and reference images directly instead of relying on URLs that may break. Note that LLMs can’t natively read markdown with inline images in one pass — the workaround is to have the LLM read the text first, then view some or all of the referenced images separately to gain additional context. It’s a bit clunky but works well enough.
• Obsidian’s graph view is the best way to see the shape of your wiki — what’s connected to what, which pages are hubs, which are orphans.
• Marp is a markdown-based slide deck format. Obsidian has a plugin for it. Useful for generating presentations directly from wiki content.
• Dataview is an Obsidian plugin that runs queries over page frontmatter. If your LLM adds YAML frontmatter to wiki pages (tags, dates, source counts), Dataview can generate dynamic tables and lists.
• The wiki is just a git repo of markdown files. You get version history, branching, and collaboration for free.

Why this works

The tedious part of maintaining a knowledge base is not the reading or the thinking — it’s the bookkeeping. Updating cross-references, keeping summaries current, noting when new data contradicts old claims, maintaining consistency across dozens of pages. Humans abandon wikis because the maintenance burden grows faster than the value. LLMs don’t get bored, don’t forget to update a cross-reference, and can touch 15 files in one pass. The wiki stays maintained because the cost of maintenance is near zero.

The human’s job is to curate sources, direct the analysis, ask good questions, and think about what it all means. The LLM’s job is everything else.

The idea is related in spirit to Vannevar Bush’s Memex (1945) — a personal, curated knowledge store with associative trails between documents. Bush’s vision was closer to this than to what the web became: private, actively curated, with the connections between documents as valuable as the documents themselves. The part he couldn’t solve was who does the maintenance. The LLM handles that.

Note

This document is intentionally abstract. It describes the idea, not a specific implementation. The exact directory structure, the schema conventions, the page formats, the tooling — all of that will depend on your domain, your preferences, and your LLM of choice. Everything mentioned above is optional and modular — pick what’s useful, ignore what isn’t. For example: your sources might be text-only, so you don’t need image handling at all. Your wiki might be small enough that the index file is all you need, no search engine required. You might not care about slide decks and just want markdown pages. You might want a completely different set of output formats. The right way to use this is to share it with your LLM agent and work together to instantiate a version that fits your needs. The document’s only job is to communicate the pattern. Your LLM can figure out the rest.