Thursday, November 19, 2026

AI: agent memory, context engineering

a clear trend, usually called agent memory or context engineering. The idea: the context window is the model's working memory, and everything else (past sessions, tool results, decisions, documents) goes into a searchable store. The agent retrieves from it as needed, instead of being bootstrapped each time. 

Representative approaches:MemGPT / Letta: tiered memory. A small working context plus archival memory, which the agent pages in and out itself, much like an operating system's virtual memory.
  • Mem0, Zep, LangMem: memory layers that extract facts from conversations and retrieve them later. Zep keeps a knowledge graph with dates on each fact, so it can tell what was true when.
  • Vendors' own memory: Anthropic's memory tool and context editing, and ChatGPT's memory.
  • Chroma: published research on "context rot". Models get measurably worse as the context grows, even far below the window limit. That's the argument for retrieving a small relevant slice rather than filling a million-token window. Their move toward memory follows from it.

The catch is in the phrase "almost infinite context": storage is unlimited, attention is not. Quality depends entirely on retrieving the right pieces, and the problems are the ones from RAG, plus some new ones:
  • What to write: remembering everything buries the signal.
  • Facts that change: an old decision and its reversal both get retrieved. This is why temporal graphs exist.
  • Contradictions and confident errors in automatically extracted "facts".
  • Privacy: what is kept about whom.

What you do now (bootstraps, memory files, "COLD START" pointers) is the curated, hand-written version of the same thing. The trend automates writing and retrieving memory. The likely direction for you is not replacing it but adding to it:
  • Keep the curated layer: decisions, rules, current state. It's small, you've checked it, and it's always loaded.
  • Add automatic, searchable memory for the long tail: past sessions, logs, superseded plans, retrieved only when relevant.

That combination is where most of these systems seem to be heading: a small memory that is always in context, and a large one that is searched.

No comments: