Anatomy of a memory for LLMs
Provenance, who said it and how sure we are
The project behind this series: ns-brain.fightclub.pro
In 1986, the morning after the space shuttle Challenger broke apart, the psychologists Ulric Neisser and Nicole Harsch asked 106 students to write down how they had heard the news. Two and a half years later they asked 44 of them again. About a quarter now told a completely different story and most of them were very sure of it: their confidence had nothing to do with whether they were right.
An AI agent's memory has the same weakness, only faster. An agent writes "the tests pass" or "we agreed on a nightly batch" with full confidence, sometimes about a test it never ran or a conversation that never happened. It does this in the same calm tone it uses for the truth. If the memory stores that as a plain fact, the next session believes it. The session after that believes it more and nobody remembers where it came from. Neisser's students had at least their written answers from the first morning. A memory for an agent needs the same thing next to every fact: who said it, when and on what evidence.
When I built ns-brain I made every fact carry its origin: a source (a file, a command, a link, "stated by the user"...), a date, a confidence and a trust level. Confidence has a default (reported, which counts 1.0) and two other values:
- confirmed (counts 1.15 times) needs a source that someone outside the session can check: a file path, a command anyone can run again, a link, a commit or the person themselves. The person is a valid source, because a preference has nothing to cite but the person who stated it. The agent's own word is not a source. When an agent writes "I ran the tests" or "we agreed", nobody can check it later: the next session has no record of that test run or that conversation, only the sentence that claims it happened. So ns-brain refuses to store it as confirmed and its answer says what to do instead: add a source (the command that ran the tests, the file, the link) or store the fact as reported. The terminal session below shows both: the claim is refused, then the same fact goes through once it names the command.
- assumed (counts 0.8 times) is for what nobody has checked: something the agent worked out on its own, a guess or a plan nobody has approved yet. Say the agent notices that invoices are emailed one by one and concludes the mail provider charges per call, without looking at the provider's price list. That may be right but it is still a guess: the label says so. A search still finds it. Its score is multiplied by 0.8, so it ranks a little lower than a checked fact that matches as well. It also arrives with "assumed" printed next to it. The next session reads it as something to check, not as something known.
Figure 1How a fact earns confirmed
ns-brain did not always refuse. At first, when an agent tried to store a fact as confirmed without saying where it came from, ns-brain stored it as reported instead and printed a warning. But most agents use ns-brain as a tool (through MCP, the standard way to plug tools into an AI model) and see only the tool's reply, not the warning. The reply said "stored", so the agent believed its facts were confirmed while ns-brain had saved them as reported. A refusal fixes that, because the refusal is the reply: the agent reads why, adds a source and tries again.
This is a real transcript from a test brain for an invented billing project, one command at a time.
First the user's rule. The user said invoice emails go out the moment an invoice is paid, so the agent stores it as confirmed with the user as its source. "stated by the user" is enough here, because the person who set the rule is the one who can vouch for it.
$ ns-brain remember --type decision --scope billing --title "Invoice emails go out the moment an invoice is paid" --body "Customers asked for the receipt at once." --source "stated by the user" --confidence confirmed
remembered #1 [billing/decision] Invoice emails go out the moment an invoice is paidNext the agent's own idea: send all invoice emails in one batch at night, to cut calls to the mail provider. Nobody has approved it, so it goes in as assumed. That label needs no source.
$ ns-brain remember --type decision --scope billing --title "Invoice emails move to one nightly batch at 2am" --body "Proposed by the agent to cut calls to the mail provider. Not approved yet." --confidence assumed
remembered #2 [billing/decision] Invoice emails move to one nightly batch at 2amThen the agent tries to record that the tests pass, as confirmed, with nothing behind it but its own sentence "I ran the tests and they all passed." ns-brain refuses. The refusal says why (nobody outside this session can check it) and what to do instead: add a source or store the fact as reported. One line in it says more than ns-brain actually does: confirmed does not outrank everything else. It lifts a fact's score for as long as the fact is stored, but only above facts that match the question equally well (that limit is at the end).
$ ns-brain remember --type state --scope billing --title "Invoice tests pass after the VAT fix" --body "I ran the tests and they all passed." --confidence confirmed
refusing 'confirmed': nothing here can be checked by anyone who was not in this session.
'confirmed' outranks everything else forever, so it takes evidence, not a self-assessment.
Add --source "<file, command or URL>", --ref "<path:line>", or --source "stated by the user" (the phrase itself, not a source that contains the word); or store it as --confidence reported.The agent runs the same command again, this time naming the command it ran (go test ./invoice/...) as the source. Anyone can run that command again and see for themselves, so ns-brain accepts the fact as confirmed.
$ ns-brain remember --type state --scope billing --title "Invoice tests pass after the VAT fix" --body "All 212 invoice tests pass." --source "go test ./invoice/..." --confidence confirmed
remembered #3 [billing/state] Invoice tests pass after the VAT fixLast, a search for "invoice emails". Both rules come back. The test result (#3) does not, because it never mentions emails: when some facts hold every word of the question, ns-brain returns only those. Each result shows its label in brackets and its source underneath, so whoever reads it sees that #1 came from the user and #2 is only an assumption.
$ ns-brain recall "invoice emails" --scope billing
#1 [billing/decision] (score 0.773) Invoice emails go out the moment an invoice is paid [confirmed]
Customers asked for the receipt at once.
(stated by the user)
#2 [billing/decision] (score 0.538) Invoice emails move to one nightly batch at 2am [assumed]
Proposed by the agent to cut calls to the mail provider. Not approved yet.Both rules match "invoice emails" equally well and both are decisions, so the only difference in their scores is the label: 0.773 against 0.538 is the same ratio as 1.15 against 0.8.1 The plan still arrives, with "assumed" next to it in the session brief, in every search and in the facts that arrive with each prompt. The next agent sees a proposal, so it does not build the batch as if someone had decided it. When the brain is shared by a team, the label is also what tells a colleague's agent that the batch was only an idea.
Trust is a separate question from confidence: it says where a fact came from rather than how sure it is. A note or a lesson that comes from another project is marked as another project's claim. The agent may report what it says but should not act on it without the person, because any session in any project could have written it.
This has a real cost and I want to define these limits: confidence only decides between facts that match the question equally well (it cannot score a weak match higher than a strong one), so an assumed fact that matches the question much better still comes before a confirmed one that matches it poorly. ns-brain checks that a citation is there, not that it is right: a source that names a command proves that someone wrote the command down, not that it passed. A person can also be as wrong as Neisser's students were, because "stated by the user" means the user said it, which is not the same as true. What ns-brain can do is keep the date and the source next to each fact so that a later session can weigh it. That is what Neisser's students did not have in their heads.
- How the two scores come about. ns-brain multiplies three numbers: how well the words match, how fresh the fact is and how much it counts. Both rules hold "invoice" and "emails" once and are about the same length, so the word match is 0.560 for each. Both were written seconds before the search, so freshness is 1.0 for each. How much a fact counts is its type (1.2 for a decision), its importance (1.0 at the default) and how often it has been used (1.0, as neither has been yet), times its label: 1.2 × 1.15 = 1.38 for the user's rule and 1.2 × 0.8 = 0.96 for the agent's plan. So 0.560 × 1.38 = 0.773 and 0.560 × 0.96 = 0.538. Everything except the label is the same for both rules, which is why the scores have the same ratio as the labels. The ratio stays exact only while that combined weight stays between 0.70 and 1.43, the limit ns-brain puts on it so that what a fact is can never outweigh how well it matched. ↩