Memory & Agency: What Should an Agent Remember?

A result without its history becomes a bad rule

When the content team changes the project-management comparison page’s title and organic clicks rise, an agent can remember two very different things. It can store the event: the title changed, then clicks increased. Or it can store a conclusion: this kind of title improves search performance.

The first is a record. The second is a claim about why something happened. Between them lie other possibilities: seasonality, a search ranking change, a promotion, or ordinary variation. If the agent keeps only the conclusion, it may repeat yesterday’s success as if it were a rule.

This is the problem memory should solve. A useful agent needs more than a store of past outputs. It needs to preserve what happened, how the team interpreted it, what evidence supported that interpretation, and what could make it stop applying. Then it can bring the lesson back when the situation resembles the past, while leaving room to question it.

Here, agency means selecting or proposing what to do next. Persistent memory can inform that choice, but it does not update the model’s parameters, prove that a lesson is reliable, or authorize the system to act.

Part 2 of 3: retaining and revising support. Part 1 follows a claim back to its source. Here, using that claim leads to a recommendation, an intervention, and a new observation. Memory must keep that history available for review. Part 3 asks what changes when the claim concerns a visitor’s goal.

A memory needs more than a takeaway

Return to the changed page title. “Clicks increased” is incomplete without the page, the version that changed, the measurement window, the comparison period, and the source of the data. “The title caused the increase” requires more evidence still. A memory that collapses these statements together makes it hard for a future user or agent to tell observation from explanation.

A useful record keeps those parts connected:

Event: what changed, where, and when.
Context: the page, audience, season, market, and other conditions that could matter.
Outcome: what was measured, over what period, against which baseline.
Interpretation: what the team thinks may explain the outcome.
Evidence and uncertainty: the data supporting the interpretation, alternative explanations, and limits.
Conditions for reuse: what would need to be true before trying the lesson again, and when it should be reviewed.
Reasons to reconsider: corrections, conflicting evidence, changed measurement, or other conditions that undermine the lesson. Philosophers call such undermining reasons defeaters.

These fields are a design proposal, not a guarantee that a system can infer causation. If the team changed several things at once, the record should say so. If the comparison is weak, the memory should preserve that weakness rather than polish it away.

Preserve the record and make room for new reasoning

A philosophical dispute helps sharpen this design: does memory preserve justification, or can it also generate it? These are claims about the support for a belief, not simply about whether stored data changes. Preservationist accounts ask how earlier support is carried forward; generativist accounts allow memory to contribute new justification. Both positions have several forms. The Stanford Encyclopedia’s discussion of memory and justification explains the distinction.

A related dispute separates internalist requirements, which place weight on reasons accessible to the believer, from externalist accounts that can make justification depend on a reliable process. Neither reduces neatly to “explain everything” versus “results are enough.” They ask what makes a belief justified, while this design asks what records let a team inspect and challenge a recommendation.

That engineering question gives preservation and generation separate places. Keep the title edit, source evidence, and observed outcome as versioned records. Let a later review propose a new explanation, such as a difference between comparison pages and support pages. Store the explanation as a new interpretation linked to those records, not as an overwrite of what happened.

The same discipline applies when evidence weakens a lesson. If a measurement error is discovered, preserve the correction and mark the affected interpretation as disputed or withdrawn. Forgetting the reason for doubt must not make the original claim appear stronger. Human memory is often constructive; a generated software summary still needs its own fidelity checks. A human analogy cannot certify the reconstruction.

What should be kept, and what should be compressed?

A system has to select what remains in working context and what can be retrieved later. Shannon’s information theory supplies a precise measure of surprise: I(x) = -log₂ P(x). Under an assigned probability model, an event with probability one quarter has two bits of self-information. An event expected to be rarer carries more. The probability model matters.

Mutual information, I(X;Y) = H(X) - H(X|Y), measures how much knowing one variable reduces uncertainty about another. H denotes entropy, or average uncertainty. These are communication and statistical quantities, not automatic scores for truth or business value. See Shannon’s paper.

For this design, surprise can nominate a title-edit episode for review. It cannot decide its cause or importance: a tracking fault can be surprising, while an ordinary result can determine whether an expensive intervention is worth continuing. To measure information retained through summarisation, the team would need defined variables and a measurement method; inserting the equation does not measure a summary’s faithfulness.

Compression creates another risk. A summary such as “benefit-led headlines worked” is easy to retrieve, but it can erase which pages were tested, what “worked” meant, and what else changed. The system should keep a path from a general lesson back to the original episodes and evidence. A summary is an index into the record, not a replacement for it.

It helps to distinguish three kinds of memory:

Episode memory records a bounded event sequence, its setting, and its observed outcome. Here the unit is an editing intervention and its measurement window. In Part 3 it will be a visitor session; those units should remain explicit.
Pattern memory summarizes a relationship that appears across episodes, with its scope and uncertainty. This plays the generalising role often called semantic memory in agent architectures; it does not turn the pattern into an established fact.
Procedure memory records a tested way to choose or evaluate an action, including when not to use it.

A short-lived working context is different again. It contains the information needed for the immediate task, while durable records remain retrievable. Its budget should follow the model, task, and evidence required. A human working-memory estimate such as seven plus or minus two items is not a capacity rule for an LLM.

More context is not always better-used context. Liu and colleagues’ Lost in the Middle found position-sensitive retrieval and question-answering performance in the models and tasks studied. Use it as a reason to test retrieval and evidence placement in the intended setup, rather than as a fixed limit for every model.

What changes when we say the agent learns?

The series uses “learning” for several different changes:

Mechanism	What changes	What stays distinct
Model training	Parameters learned from a training process	Retrieving a new note is not this process
Retrieval into context	Evidence available for the current request	The underlying model can remain fixed
Memory revision	A saved interpretation, its status, or its conditions for reuse	The original event and correction history remain traceable
Decision-policy learning	How the system chooses among permitted strategies	Better choice under a reward measure does not establish causation or permission

The proposed first version uses retrieval and reviewed memory revision. Building to Learn gives a related lab design: provisional findings pass through evidence review before scoped reuse. That is a concrete application of the record discipline, not proof of an effective learning policy. A bandit is a later option when the intervention and reward can be evaluated credibly.

Fast assistance and slow review

A useful design can separate immediate assistance from slower analysis without pretending that a browser extension and a cloud service are two human minds. The fast part gathers the current page, the marketer’s request, and a small set of relevant memories. It proposes an edit or a test. The slower part compares results across episodes, checks whether context changed, and revises or retires stored patterns.

The page-title example shows why both tempos matter. During editing, a quick suggestion may help. After the change, the outcome needs time and a suitable comparison. A later review can ask whether clicks changed because of the title, whether another event coincided with it, and whether the result held on similar pages. If the evidence remains unclear, the memory should keep the question open.

A browser/cloud split is one possible deployment of this separation. A Chrome extension could supply the editing interface and working context; a service on AWS or another host could keep records and run later comparisons. These are proposed placements, not measured speed claims or two literal cognitive systems.

Responsibility	Information received	Result returned
Immediate assistant	Current page version, request, permitted reference material, relevant prior episodes	Suggested edit with evidence references, uncertainty, and proposal identifier
Event recorder	The proposal, what the marketer actually changed, page/version identifiers, time, measurement plan	A durable intervention record; acceptance is recorded separately from impact
Later review	Intervention records, comparable outcomes, concurrent changes, corrections	A supported, limited, disputed, or inconclusive interpretation with its review conditions
Retrieval service	Current task and authorised scope	Relevant records with source links, status, age, and reasons for the match

Retrieval-augmented generation can bring these external records into the current context. Structured fields help filter by page, period, and source; semantic similarity can help find candidate episodes. Similar wording alone does not establish that the old intervention is comparable. That follows directly from Part 1’s distinction between resemblance and support.

Only necessary, permitted material should move between client and service. Keep page and account scope with the record, prevent cross-client retrieval, and give the quick assistant a way to inspect the evidence behind a compressed lesson. A background review should publish a versioned interpretation, not silently rewrite a recommendation already being measured.

The sequence is straightforward:

A marketer asks for help with a page.
The agent retrieves relevant prior episodes and shows why they match.
The marketer decides whether to use or ignore the suggestion.
The system records what happened, while distinguishing acceptance from measured impact.
A later review tests the pattern and updates, limits, or removes it.

A rejected suggestion is useful feedback about the interaction. It does not prove the underlying lesson was wrong. An accepted suggestion is not proof it was right. The system needs outcome evidence that is independent enough to examine.

Learning from outcomes is experimental work

Memory can also inform reinforcement learning and contextual bandits: an agent chooses among strategies, observes rewards, and balances trying known options with exploring alternatives. This is a useful family of methods for some decision problems. It does not turn every content edit into a valid experiment, and it does not establish why a result occurred.

If an agent chooses a title strategy because it performed well on prior pages, it needs a clear definition of the outcome and a way to compare choices fairly. Pages may differ in traffic, audience, season, or search demand. A change in clicks may follow an algorithm update rather than the edit. A metric can also reward behavior the team did not actually want.

For a first system, begin in recommendation or logging mode. Keep a record of the suggestion, the reason it was made, the marketer’s response, and the outcome measurement. Test one change at a time where practical, compare with an appropriate baseline, and report uncertainty. If the system cannot separate its intervention from the circumstances around it, it should say that the result is inconclusive.

Richard Sutton’s “bitter lesson” concerns the historical advantage of general methods, especially search and learning, that can make use of increasing computation. Applied here, it motivates testing a system that can improve from experience rather than relying only on a growing list of hand-written rules. It does not show that a particular memory architecture works or that a learning algorithm will discover the right objective. An agent can optimize the metric it receives and still produce a result the team regrets.

What the exploration equation actually offers

A simple bandit rule illustrates the optional decision-policy layer. After trying each permitted strategy once, UCB1 selects the strategy with the largest mean_reward(s) + sqrt(2 ln(t) / n(s)). Here t is the number of completed trials and n(s) the trials for strategy s. The second term favours less-tested options. Auer, Cesa-Bianchi, and Fischer analyse this rule for rewards scaled to the interval [0, 1], with a fixed reward distribution for each strategy and independent draws across its trials.

A contextual bandit uses situation features when estimating which choice may work; it is not merely UCB1 with extra prose in a prompt. SEO outcomes are often delayed, affected by concurrent edits, and changed by search conditions. The team needs an appropriate method for those conditions, a defined reward, and an attribution plan before treating page edits as bandit trials. The exploration term does not make an intervention safe: approval and eligible strategies are separate constraints.

When should a memory be revised?

A historic observation can remain accurate while its relevance to the current page declines. Past evidence does not expire on a single universal schedule. A memory may become less relevant when the page, product, audience, season, search environment, or measurement method changes. Those conditions belong alongside the claim so the system can notice when it is time to check again.

Conflicting memories should not be forced into one universal rule. Perhaps benefit-led headlines performed better on one kind of page, while detailed feature language helped on another. The system should preserve the different conditions and compare the original evidence before deciding whether the apparent pattern is real. If it cannot identify a meaningful distinction, the conflict remains unresolved.

A memory can also become self-confirming. The agent recommends the same change because it remembers recommending it; the team accepts it because the system sounds confident; the resulting data is then taken as proof of the original recommendation. Independent baselines, visible provenance, and periodic tests help interrupt that loop.

What could a later memory system investigate?

The review method itself can be evaluated. Does it favour a familiar source despite contrary evidence, overlook unsuccessful interventions, or consolidate conflicting outcomes too quickly? Test revisions to that method on held-out episodes before changing how live recommendations are formed.

Shared memory raises a related question. Several agents may contribute records, but ten summaries of one experiment are still one experiment. Preserve the source relationships, disagreements, and permitted scope when records enter a common library. Agreement among agents is not independent corroboration.

Periodic offline replay can test whether a revised retrieval or review process handles past cases better. This gives the fast/slow design a concrete research direction without assuming that software consolidation works like biological sleep. Reprocessing old episodes can expose a missed distinction; it cannot manufacture the missing comparison needed to establish a cause.

Test whether memory preserves the reasons for reuse

A test portfolio should examine how the whole memory design behaves under pressure:

Trial	What to establish
Working-context pressure	Required source details remain retrievable when context is shortened or distractors are added
Consolidation	A pattern retains links to its episodes, measurement windows, concurrent changes, and limitations
Conflicting outcomes	The system preserves disagreement unless evidence supports a contextual distinction; it does not invent differences between cases to make both results fit
Stale or corrected evidence	The original event survives, while affected recommendations are rechecked or withdrawn
Reward mismatch	More clicks or more accepted suggestions do not count as success when the agreed objective worsens
Retrieval boundaries	Similar records from an unauthorised client cannot enter the current recommendation

Compare recommendations with and without the retrieved history on held-out episodes, using the same available information and review criteria. Inspect factual fidelity, the fit of the old evidence to the new case, and whether the agent notices a reason to abstain. For an intervention, measure the later outcome separately from the quality of the recommendation.

The result of this article is a revisable memory record: event, context, outcome, interpretation, supporting evidence, and conditions for reuse or reconsideration. It supports a new choice without pretending the past is a universal rule.

There is one unresolved question behind the comparison page’s click count. More visits do not establish that the page helped visitors do what they came to do. Part 3, The Geometry of Intention turns from the system’s past interventions to the visitor’s current task. The record-keeping discipline carries over; the object being interpreted changes.

Publication history

Originally published by Performics Labs.