Guides
LLM wiki lint: what to check and how Cozy scores it
LLM wiki lint is a set of automated checks that find problems in a wiki an AI agent maintains: broken links, stale pages, conflicts, orphans and unsourced claims. In Cozy, the cozy_health tool reports these checks, rolls them into a score from 0 to 100, and lists fixes in priority order so an agent can repair the wiki.
Why an LLM wiki needs linting
In the LLM wiki pattern, an agent ingests sources, writes Markdown pages, and answers questions from them. Karpathy's LLM Wiki gist names three operations: ingest, query and lint. Ingest and query get most of the attention. Lint is the step that keeps the wiki trustworthy after hundreds of agent edits.
Without it, small errors pile up. A renamed page leaves dead links. A source changes and the summary written from it goes quietly out of date. Two pages record different values for the same fact. Nobody notices until an agent repeats the wrong one.
Cozy calls its lint pass Health. It is exposed to agents as the cozy_health tool on the hosted MCP server.
What the checks catch
Each check maps to a failure that is common when agents write most of the pages.
| Check | What it flags |
|---|---|
| Broken links | A wikilink that resolves to no page |
| Duplicate titles or aliases | Two pages that answer to the same name |
| Stale pages | A cited source changed after the page was last edited |
| Conflicts | Pages carrying a CONFLICT: line |
| Orphans | Pages that nothing links to |
| Unreachable pages | Pages that wiki/index.md cannot reach through links |
| Unsourced pages | Pages that cite no source |
| Missing type or description | Frontmatter that is incomplete |
| Oversized pages | Pages too large to read comfortably |
| Uncited sources | Evidence in sources/ that no page uses |
The stale-page check depends on source citations. A page that links into sources/ is recorded as citing that source. When the source is re-imported with new content, the page is older than its evidence and gets flagged.
How links are resolved
Cozy rebuilds the graph from Markdown on every save, so link checks reflect the current files. A name link resolves by path, path without extension, file stem, title or alias, ignoring case and accents. A link that matches none of these counts as broken. Links inside code are ignored, so examples in a page do not create false alarms.
Frontmatter keys whose values are wikilinks become typed relations, and relation:: [[target]] lines work too. The typed links page covers the full set of conventions.
How the health score works
Each page starts at 1.0 and loses weight for every issue found on it. The score is the average across pages, multiplied by 100. A wiki with no issues scores 100. The tool also returns a next list that orders fixes by priority, so an agent does not have to decide where to start.
Cozy does not publish per-issue weights on this page, and the score is only meaningful relative to your own wiki over time. Use it as a trend line and read the issue list for specifics.
Running a health pass with an agent
Cozy runs no model of its own, so your agent does the fixing. A typical loop looks like this:
- Call
cozy_healthand read the score and thenextlist. - Use
cozy_readorcozy_contexton the first flagged page to see its neighbors, evidence and warnings. - Fix the issue with
cozy_edit, passing the page'sexpectedRevision. - Call
cozy_healthagain and confirm the score moved.
Edits require expectedRevision, and concurrent edits return a conflict instead of overwriting. If an agent makes a bad fix, cozy_history and cozy_restore bring back any of the last 50 revisions.
For broken links after a rename, cozy_move keeps history and returns the links that need repair. For duplicates, cozy_resolve maps names to pages and surfaces likely duplicates.
Lint and dreaming
Health tells you what is wrong. Dreaming is the offline consolidation pass your own agent runs to fix it. A dream run orients with cozy_health, scopes work to cozy_changes since the last run, verifies claims against sources, keeps superseded values with dates, records CONFLICT: lines, merges duplicates, writes typed relations, and keeps wiki/index.md short. The last-run time lives in agents/dream/state.md. See memory consolidation for the longer explanation.
What lint cannot do
Lint is structural. It can tell you a page cites no source, but not whether the page is true. It can detect that a source changed, but not whether the page's claim still holds. A human or an agent still has to read the source and decide.
It also does not replace review of what agents write. Scoped credentials limit what an agent can touch, and the revision history lets you undo it, but neither judges the content.
If you keep a wiki in plain Markdown elsewhere, you can approximate some of these checks with scripts. Link checkers and orphan finders are easy to write. Source-versus-page staleness and conflict counting are harder to bolt on, which is where a built-in pass helps. Obsidian and similar tools have strong plugin ecosystems for local vaults, so compare options on the compare page before deciding.
Getting started
Sign in at cozy.cloud, connect an agent to https://cozy.cloud/mcp over MCP. Then ask it to run cozy_health and work through the next list. The FAQ covers setup questions.
Frequently asked questions
What is LLM wiki lint?
It is an automated pass over a wiki that an AI agent maintains, looking for broken links, stale pages, conflicts, orphans and unsourced claims. The term comes from the lint operation in Karpathy's LLM Wiki pattern. In Cozy it is the cozy_health tool.
How is the Cozy health score calculated?
Each page starts at 1.0 and loses weight for every issue on it. The score is the average across pages times 100. The tool also returns a next list of fixes in priority order.
What makes a page stale in Cozy?
A page is stale when a source it cites changed after the page's last edit. The page links into sources/, so Cozy can compare the two timestamps. See source citations.
Does Cozy fix the problems it finds?
No. Cozy runs no model itself, so your own agent reads the report and makes the edits. The offline memory consolidation pass called dreaming is one way to do that on a schedule you choose.
Can lint tell me whether a page is correct?
No. It checks structure, links, citations and timestamps. A person or an agent still has to compare claims with the sources.
Give your agents a memory worth keeping
Cozy gives every account a private wiki that Claude, Codex, ChatGPT and other MCP clients can read and maintain.
Start your wiki