The Memory Index That Quietly Stopped Being An Index

If you give a coding agent persistent memory, you will end up with two things: the memories themselves, and something that tells the agent what it has. Most people — me included — start by making that second thing a markdown file. It's the obvious choice: human-readable, trivially editable, and it loads into context automatically at the start of every session.

It worked for a long time. Then I counted.

Sixty of two hundred and forty-one memory files were unreachable from the index. Not deleted. Not corrupted. Every single one of them sat on disk exactly as written, and every one was mirrored in a separate document store. Nothing was lost. What was lost was recall — and that is a much worse failure than it first sounds, because it looks like nothing at all.

The failure mode that looks like success

A memory you cannot find is a memory you do not have. But an index that has quietly shed a quarter of its entries doesn't announce it. It still opens with a confident heading. It still lists plenty of things. Every link in it still resolves. There is no error, no warning, no broken state to notice — the agent simply never learns that some fact exists, and behaves exactly as it would have if the fact had never been written.

Compare that to a memory file that gets deleted. That one is loud: something references it, the reference breaks, you go looking. The silent version generates no signal whatsoever. In my case the only reason I found it was that I sat down and counted files on disk against links in the index, for an unrelated reason.

I would encourage you to go and do that count now, on whatever your agent uses. It takes a minute and I suspect you will not like the answer.

The actual root cause: one file, two jobs

The index was doing two genuinely different jobs, and only one of them was in its job title.

Job one — the pointer. A short, human-readable orientation: here is what memory is, here is how to search it, and here are the handful of rules that matter so much they need to be read without being asked for.

Job two — the catalogue. One line per memory, all of them, so nothing is unreachable.

Job one wants to be small. Job two grows without bound. In a single document, job two eventually crowds out job one — and long before that, it exceeds the size at which anyone (or anything) is actually reading the whole file carefully. Entries got added, entries got reworded, sections got reorganised, and somewhere in the accumulated churn of a document nobody ever read end-to-end, sixty lines stopped existing. I cannot tell you which edit dropped which line, and that is the point.

An index that is also a document has a size ceiling, and past that ceiling it sheds entries silently. Not because anyone is careless. Because maintaining a complete list by hand, inside a file that is also prose, is a job with no feedback loop.

The fix, and the three parts of it worth stealing

Split the jobs. Keep the pointer small, human-readable and always loaded. Move the catalogue into a derived index — in my case a small local SQLite database holding, per memory, its name, description, type, tags, path, links, plus a full-text index over the body.

That's the obvious half. The three details that make it actually hold up:

Rebuild wholesale, never incrementally. The build throws the entire database away and recreates it from whatever is on disk right now. Incremental updates are how a derived copy drifts from its source — one missed delete, one failed partial write, and you are back to an index that disagrees with reality and won't tell you. A full rebuild cannot drift, by construction. It costs a second or two.

Refuse to write an empty index over a good one. If the build finds zero memory files, that is overwhelmingly likely to be a broken path or a bad working directory, not a genuine emptiness. So it refuses and exits loudly rather than faithfully writing out nothing. This one rule converts the worst possible outcome — a rebuild that destroys the catalogue — into an error message.

Treat an empty query result as a cache miss, not as proof. The index is derived data. When it returns nothing, the honest statement is "the index has nothing, and I checked the source directory to confirm," not "there is nothing." Those are different claims, and only one of them is an answer.

The part that actually transfers: always-on versus lookup

Here is the design judgement I did not expect to have to make, and it is the reason this isn't just a story about putting data in a database.

Nothing auto-queries a database.

A searchable index is wonderful for anything the agent knows to go and look for. "Have we decided how to handle X?" is a lookup. "What did we learn about this library?" is a lookup. The agent recognises a question, runs a query, gets an answer. That is most of memory, and moving most of memory into a queryable store is a straightforward win.

But some things are not lookups. Some rules only work if they are in front of you without your having known to ask — because the situations they govern are exactly the situations where you don't realise a rule applies. A constraint on how work gets committed only helps if it is read before the commit, and an agent that doesn't know the rule exists has no reason to search for it. Put that in a database and you have filed it perfectly and defeated its purpose.

So the split isn't "human-readable pointer versus machine-readable catalogue." It's:

• Always-on — the small set of things that must be in context unprompted, because acting correctly on them depends on seeing them without asking. These stay in the always-loaded file, stated in full, and the cost of keeping them there is that the file must stay small enough to actually be read.
• Lookup — everything else, which is the overwhelming majority, and which belongs in the index precisely because it will be searched for by something that already knows it wants it.

Deciding which memories are which is real work, and it is the part I'd spend the most time on. The temptation is to mark everything important as always-on. Resist it: the always-on set has a budget, and blowing that budget is how the original file got into trouble in the first place.

The honest counterweight

This adds a third place the same facts live: the memory files, the derived index, and (in my case) a separate document store that also mirrors them. Derived-copy drift is a real cost, and I don't want to pretend otherwise — a second copy of anything is a second thing that can be wrong.

That cost is exactly why rebuild-only and the empty-write refusal are in the design rather than bolted on later. A derived copy is safe in proportion to how cheap and how total its rebuild is. The moment you find yourself writing incremental update logic for a cache you could regenerate in two seconds, you have traded away the property that made it trustworthy.

What I'd suggest

If you're running an agent with persistent memory:

1. Count. Files on disk against entries in your index. Today. The gap is the interesting number and there is no other way to see it.
2. Ask what your index file is actually for. If the answer is "orientation and a complete catalogue," it is doing two jobs and one of them will lose.
3. Sort your memories into always-on and lookup, explicitly, and keep the always-on set small enough that it genuinely gets read.
4. Make the catalogue derived, rebuilt wholesale, and unable to overwrite itself with nothing.

None of this is sophisticated. The sophistication was in the failure — a system that degraded without producing a single symptom, in a component whose entire purpose is to make sure nothing gets forgotten. Hopefully this saves you the count I had to do — and if you run it and your numbers come back clean, I'd genuinely like to know what you're doing differently, because I don't think my setup was unusual.