# LLM wiki vs RAG vs vector memory: when to compile

> Compare an LLM-maintained wiki, RAG over chunks and vector memory. See when to compile knowledge into pages, when to retrieve, and how Cozy fits in.

Updated 2026-11-08. Canonical: https://cozy.cloud/llm-wiki-vs-rag

> Use RAG or vector memory when you have a large, changing pile of documents and need fast lookups of specific passages. Use an LLM wiki when knowledge is reused often, needs to stay consistent, and should be readable and correctable by you. An LLM wiki compiles sources into linked pages once; RAG re-derives answers from chunks every time.

## Three ways to give an agent knowledge

**RAG** (retrieval-augmented generation) splits documents into chunks, indexes them, and fetches the best matches at question time. The model reads those chunks and answers. Nothing is synthesized in advance.

**Vector memory** is a store of extracted facts or snippets that an agent writes to and reads from, usually ranked by embedding similarity. Products such as Mem0, Zep and Letta sit in this area. The memories are typically records inside a service, not files you open.

**An LLM wiki** is a folder of Markdown pages that an agent maintains. Sources go in, the agent writes and updates linked pages, and later questions are answered from those pages. The pattern was popularized by [Andrej Karpathy's LLM Wiki gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f), which describes three layers (raw sources, an LLM-owned wiki, and a schema file) and three operations: ingest, query and lint.

## Where compiling wins

Compiling does the synthesis work once and keeps it. That matters in a few situations.

- **Repeated questions.** If the same topic comes up weekly, a page that already holds the answer is cheaper and more stable than re-deriving it from chunks each time.
- **Cross-document facts.** A person, project or decision often spans many sources. A page can join them. A chunk store returns fragments and leaves the join to the model.
- **Contradictions.** A wiki can record a conflict and keep the older value with a date. Chunk retrieval just returns both passages and hopes the model notices.
- **Human review.** A page is something you can read, correct and trust. An embedding is not.
- **Provenance.** Pages can cite the source that backs each claim, so you can check it.

## Where retrieval wins

Be honest about the costs of compiling. Writing and maintaining pages takes agent effort, and the gist's own comment thread asks whether ingest cost pays off against direct retrieval.

- **Large corpora.** Karpathy says index-based navigation works at around 100 articles or 400,000 words (as reported by [VentureBeat via Implicator](https://www.implicator.ai/karpathys-llm-wiki-got-16-million-views-the-code-isnt-the-point-2.md)). Beyond that you need real search.
- **Needle lookups.** Finding one clause in a thousand contracts is a retrieval job.
- **Fast-moving raw data.** Logs, tickets and email streams change faster than anyone would compile them.
- **One-off questions.** If you will ask something once, a page is wasted effort.

## Side by side

| Factor | RAG over chunks | Vector memory | LLM wiki |
| --- | --- | --- | --- |
| Unit of storage | Document chunks | Extracted memories | Linked Markdown pages |
| Synthesis | At query time | At write time, per fact | At ingest and maintenance time |
| Human-readable | Rarely | Varies by product | Yes |
| Handles huge corpora | Yes | Moderate | Needs search alongside |
| Contradictions | Left to the model | Often overwritten | Can be recorded |
| Provenance | Chunk source | Varies | Citations to sources |

## They combine well

Most real setups use more than one. A common split is to keep raw material as sources, compile the parts you reuse into pages, and search across both. The wiki becomes the curated layer and retrieval covers the long tail.

## How Cozy fits

We make Cozy, a private wiki and cloud file space for each person and their AI agents. It sits on the compile side of this comparison, with a few design choices worth stating plainly.

- **No embeddings or vector database.** Cozy search is lexical and paginated. If your main need is semantic similarity over a huge corpus, a dedicated retrieval system is a better fit, and you can run one next to Cozy.
- **Sources and pages are separate.** Material from connectors and importers lands in `sources/` with stable IDs, URLs and timestamps, and is treated as untrusted data. Curated pages live in `wiki/`. See [source citations](/source-citations).
- **The graph comes from Markdown.** Frontmatter, wikilinks and `relation:: [[target]]` lines are parsed on every save. Links into `sources/` count as citations. See [typed links](/typed-links).
- **Lint is built in.** A health check flags broken links, orphans, duplicates, unsourced pages and pages whose cited source changed after the last edit. See [wiki health](/wiki-health).
- **Cozy runs no model.** Your own agent does the writing, including the offline consolidation pass we call dreaming, through the hosted [MCP server](/mcp-server).

Mem0 and similar services are built to be embedded in apps and return memories through an API. Their [pricing page](https://mem0.ai/pricing) lists graph memory and consolidation features on a custom tier. Cozy stores readable files you can export and edit, but it does not offer the vector search those products are built around.

## A practical rule

1. Keep everything you might need as a source.
2. Compile a page when you notice the same facts being re-explained or re-fetched.
3. Let retrieval handle the rest until a topic earns a page.
4. Run a health check regularly so compiled pages do not drift from their sources.

For the broader pattern, read about the [LLM wiki](/llm-wiki) and how it differs from built-in [AI memory](/ai-memory).

## Frequently asked questions

### Is an LLM wiki a replacement for RAG?

Not usually. An LLM wiki suits knowledge you reuse and want to keep consistent, while RAG suits large or fast-changing collections where you need passage lookup. Many setups keep both.

### Does Cozy use embeddings or a vector database?

No. Cozy search is lexical and paginated, and the link graph is built from Markdown conventions. Cozy also runs no model itself, so you bring your own agent.

### How big can an LLM wiki get before it needs search?

Karpathy has said index-based navigation works at roughly 100 articles or 400,000 words. Past that, add search over the wiki and its sources. Cozy provides lexical search through [the MCP server](/mcp-server).

### How do I keep compiled pages from going stale?

Cite sources on each page and run lint regularly. Cozy's [health check](/wiki-health) flags pages whose cited source changed after the page was last edited.
