My AI memory, my rules — Aliaksei Novikau

Since 2023 I’ve had 5,362 conversations with ChatGPT. Claude doesn’t know any of them.

I use ChatGPT, Claude and Codex almost every day, and over time I’ve given them an enormous amount of context: projects, technical decisions, preferences, parts of my personal history. But each assistant knows a different version of me. ChatGPT remembers things Claude doesn’t, Claude has context Codex can’t reach, and the valuable parts are buried in thousands of conversations. I keep explaining things I’ve already explained.

So I’m running an experiment: what if my AI memory belonged to me, not to an AI provider?

The design: boring on purpose

Plain Markdown in a private git repository is the source of truth: structured knowledge with a full history of every change.
Raw material stays out of git: exports, files and recognised text live in a separate cloud folder.
OCR and indexing only where they add value.
Any assistant can read it: ChatGPT, Claude, Codex.
Imports go through review: a new export is processed on a branch, and I read the diff before merging.
No vector database, no RAG pipeline, no custom MCP server. Not yet. They can come later, on top of the files, without locking the knowledge into them.

A few rules turned out to matter more than the tooling:

Two memories, not one. Instructions on how agents should work with me live in a separate repository. This one is about my life and work.
Some things are never stored: passwords, tokens and keys, document numbers and personal IDs, bank accounts and cards.
Facts age at different speeds. Fast-changing things get re-checked after a month, the current state of things after six, preferences and opinions after a year, plans when they’re due.
Every fact links back to the exact message it came from.
Contradictions are flagged, not resolved silently. My own stories disagree: I’ve given three different years for when I started university. The system doesn’t pick one; it asks me.

The pilot: 5% of ChatGPT

Before processing everything, I ran a pilot on a 5% sample: 268 conversations spread evenly across the years, plus the start of one long biographical chat.

Most of the history isn’t me. 83% of the text was the AI talking. Skipping most of it, along with code and logs, cut what a model has to read almost sixfold.
Eight extraction agents in parallel finished in under nine minutes. Each took 4–9 minutes regardless of the model.
The fixed cost dominates. Every agent burned 100–150 thousand tokens on instructions and its own context, while the useful input was only 10–30 thousand. Bigger batches per agent are the cheapest optimisation available.
A small model was good enough. On the same personal conversations, Haiku and Sonnet extracted practically the same facts, amounts, dates and statuses; Sonnet was better at nuance and stricter with the rules. At a twentyfold price difference, the small model wins, and the expensive one is kept for merging.
No invented facts in the spot check. Every one of 1,771 source links pointed to a real message; a manual check of 15 random records against the source found nothing made up; the forbidden-data filter caught nothing, and the agents reported themselves what they had skipped.
The whole archive is affordable. Extrapolated to all 5,362 conversations, extraction costs roughly $9–21 in API prices with the small model doing most of the work.

What I don’t know yet

The real question remains: can different assistants work from the same context, and can that knowledge grow without turning into a mess?

If it works, I’ll grow it, maybe into an open-source project or a product. If it doesn’t, I’ll have learned something without spending months on infrastructure.

I don’t want another assistant with a better memory. I want a memory that works with any assistant, and I want to own it.

How do you keep context across different AI tools?