Series: How I built a second brain · part 3
The second brain from part one is a wiki that an AI maintains for me. The very next day I added something you can talk to: AI advisors. This part is about how they are built, and above all about what they remember. The roles take an afternoon to write. I spent half a year tuning the memory.
Six pages, six roles
On 8 April I set up five advisors in the wiki: a life coach, a financial and tax advisor, a health coach, a business coach and a meditation teacher. Five days later a sixth joined them, a social media expert.
Each advisor is an ordinary Markdown page. It describes the role, the method, the temperament and, above all, the boundaries with the other five. The life coach, for example, holds the whole picture while the others deal with the "what" and the "how"; it stays out of taxes, but it may ask why I keep avoiding a topic. The page also says what the advisor should write down after each conversation.
I talk to them through Claude Code opened in the wiki folder. The agent reads the wiki's rules, the advisor's page and its memory, and then it is that advisor. No special app, no database: files I can open and read.
An advisor is text you can read. When it behaves oddly, you open its page and find out why.
What went wrong
Five logs that knew nothing of each other
In the first few days, five of the six advisors each started their own conversation log. That worked as long as topics stayed inside their borders. But almost all of them cross: a business decision affects my time, and my time affects my health. A coach that reads only its own log doesn't know what I discussed with another one yesterday.
On 4 June I merged the five logs into a single stream of 73 entries, newest first. And I separated two things I had been mixing until then:
- Stream — what happened. One shared log for all six; entries are only ever added.
- State — what is true now. A shared profile with what every advisor should know, and one page per advisor with what only that advisor needs. These get rewritten.
An advisor reads the shared profile, the head of the stream and its own page. Never another advisor's page. That way it sees what is happening elsewhere without paying for the full depth of other fields.
A session number that tells you nothing
Entries were labelled "Session 16". A review in July showed that the numbers had started as a separate counter for each advisor and only later became a shared one. About twenty numbers were reused by different coaches. A reference to "session 16" could not be resolved.
I switched to the date. The same day it turned out that if I talk to one advisor in the morning on my phone and in the afternoon at the keyboard, both conversations get the same ID. So: the date and time the conversation started. Started, not filed: filing can happen hours later, and a Sunday evening would end up under Monday.
Then in August, over one long evening, an agent stamped thirteen entries by reading the time once and simply adding to it. Six of them landed on a day that had not happened yet. The rule since then: if you are about to write an ID and have not read the clock since the last one, read it.
State that turned into a log
State pages were meant to be short. Two of them grew to 81 and 53 headings, most of them dated. They were logs in disguise, and because of them a finance session read about 107,000 tokens before it reached my question.
The fix: every state page opens with a short board that says where things stand now. A session reads the board and opens the details when it needs them. The read dropped to roughly 12,000 tokens.
But the board grew again, in a different way. One of them went from 2,926 to 13,208 tokens in twelve days without a single line being removed. Every new line was true, which is why nobody noticed.
A line on the board gets rewritten, not added. A new topic earns a new line only rarely.
A reading budget keeps watch over this: the shared profile, the advisor's board and the head of the stream together must stay under 30,000 tokens. I first set it at 18,000, but that pushed me to cut pages I wanted to be richer. The budget is there to catch a page turning into a log, not to limit how well an advisor knows me.
A decision I never made
During one "trim to state", an agent took a warning that a topic had been stalled for a long time and turned it into a sentence saying I had deliberately put it on hold. I never made any such decision. I corrected it the next day.
Since then there is a hard rule: no line may attribute a decision, quote or preference to me without a date and a link to the record it came from. If the agent can't point to the line, it didn't learn it.
Every conversation produced a write-up
The procedure after a conversation said "update your page if the state has changed". But nothing in it said a conversation could end without an update. So it always ended with one. One good answer to a simple question was filed as a 37-line section.
The test since then: would a future session read this and act differently? Not "was the conversation worth having" — every one is. If nothing passes, the entry in the stream is the whole record.
What I learned
An advisor's memory is not one thing. It is a stream of events that is only ever added to, and a state that gets rewritten. Mix them and you get either a log nobody reads to the end, or a summary that has lost track of where it knows things from.
The second thing: it is not enough to write rules down; something has to check them. The reading budget, the order of entries and the size of state pages are all measured today by an automated check of the wiki. Before it existed, I discovered each of those failures myself, and late.
Memory can still grow, though, where it doesn't get in the way. Each advisor's page has a separate section telling the story of how it got to know me. The agent doesn't read it in an ordinary session, so it costs nothing until it is needed.
How to start
- Write each advisor a page: its role, method, temperament and boundaries with the others.
- Separate the stream from the state on day one. One shared log that is only ever added to; short state pages that get rewritten.
- Decide who reads what: the shared profile, the head of the log and the advisor's own page. Not anyone else's.
- Label conversations with the time they started, not with a sequence number.
- Give reading a token budget and measure it automatically.
- Allow no claim about yourself without a date and a source.
Next time: how I turned the advisors into a council.