Chapter 03
Memory Substrate
Why a long-running partnership with an LLM needs a persistent store outside the model, and what shape that store should take.
Updated 2026-05-12
The problem with relying on the model
A language model has no persistent state of its own. Context windows are ephemeral, conversation history gets compressed or truncated, and model versions get retired and replaced. Any partnership that treats the model itself as the place where memory lives is building on rented ground.
The only durable component in the system is what the human owns and writes down. Everything else — the weights, the inference endpoint, the chat UI — is replaceable infrastructure. The substrate has to live outside.
What the substrate is
A plain-text repository, version controlled, organized as a tree, read by the agent at the start of every session.
That is the entire definition. Nothing about the substrate is exotic. The properties that make it work are properties that plain text and git already give you:
- Diffable. Every state change is a commit. The history is the audit trail.
- Greppable. A regex over the working tree is a serviceable retrieval system before any embedding pipeline.
- Human-readable and model-readable in the same format. No serialization layer to maintain.
- Local-first. The human owns the bytes. No vendor lock-in, no service shutdown risk.
- One source of truth. No syncing problem between "the app's memory" and "what actually happened."
A repository like this is boring on purpose. The point is not to be a clever knowledge graph. The point is to outlast every other component.
A worked structure
A reasonable shape for a personal life-repo, abstracted away from any specific user:
| Directory | Purpose | Write cadence |
|---|---|---|
Logs/ | dated journal entries — what happened on day N | daily append |
Notes/ | non-trivial insights or decisions worth keeping | as they happen |
Finance/ | financial snapshots, balance reports, ledger memos | monthly + ad-hoc |
Routines/ | definitions of recurring tasks and their outcomes | rare edits |
Memories/ | curated long-term memory the agent should index | rare appends |
Scripts/ | automation that lives next to the data it operates on | as needed |
Chatlogs/ | exported conversation transcripts worth keeping | per-conversation |
The exact tree is unimportant. What matters is that every category has a single canonical home, and that the home is named in a way both human and agent can guess without a lookup.
The boot sector
A single entry-point file — CLAUDE.md, INFO.md, README.md, whatever the convention — sits at the repo root and is loaded into every session. It contains:
- Current goals and constraints.
- Current focus areas.
- Pointers into the subdirectories ("financial state lives in
Finance/, recent thoughts inNotes/YYYY-MM-DD-*.md"). - A short profile of the human, in case the agent has none.
Treat this file like the boot sector of a disk. Every session pays the token cost of loading it, so it has to stay short. Anything that does not change session-to-session belongs here; anything else belongs deeper in the tree.
Write discipline
Both the human and the agent are allowed to write. The discipline is in deciding what gets written.
- The human writes raw input. Corrections, decisions, opinions, things the agent has no way to observe.
- The agent writes structured outcomes. Memos at the end of non-trivial sessions, summarizing what was decided and why. These compound into the agent's own long-term memory across model versions.
- Filenames carry semantic information.
YYYY-MM-DD-topic-slug.mdmakes the tree self-indexing.gray-matter-style frontmatter at the top of each file carries type and tags. - Never overwrite. Append a new file dated today rather than editing yesterday's. The git history exists for a reason; using it as time travel is cheaper than mental archeology over an edit chain.
What this is not
This is not a second brain, an Obsidian vault, or a PKM system in the marketed sense of those terms. Those frameworks optimize for the human re-reading their own notes. The memory substrate optimizes for an agent reading them at session start.
It is also not a RAG store, although a vector index can be built on top if the tree grows past the size of a context window. The plain-text repository is the primary; embeddings are a secondary cache.
It is also not version-of-truth ceremony. The agent does not need elaborate metadata, schemas, or graph relationships to use the tree. It needs filenames, dates, and prose.
The compounding property
The substrate has no value in week one. In week one it is a directory of three files and a README.
In year three it contains the full record of every decision the partnership made, every constraint the human added, every correction the agent received. At that point the substrate is irreplaceable in a way the model is not: any frontier model can be swapped in, but no model can be retrofitted with three years of the human's actual context.
This is the entire reason to build it on plain text in a git repo. The model layer churns on the timescale of months. The substrate layer needs to churn on the timescale of decades. The only format that has demonstrated that lifespan is plain text under version control.
Implementation notes
- Backups. The substrate is the most valuable asset in the partnership. Encrypted off-site backup is not optional. Treat it like financial records.
- Privacy. The substrate will accumulate sensitive personal data. Keep it in a private repository, encrypt the backup, and review what the agent writes before committing if the agent has commit access.
- Index files. A
MEMORY.mdor equivalent at a known path, listing the most-recently-relevant memory entries with one-line descriptions, lets the agent decide what to pull into context without scanning the whole tree. - Scripts live with the data. Automation that reads or writes the substrate belongs inside the repo, not in a separate tooling project. The substrate and its operators travel together.