Representative approaches:MemGPT / Letta: tiered memory. A small working context plus archival memory, which the agent pages in and out itself, much like an operating system's virtual memory.
The catch is in the phrase "almost infinite context": storage is unlimited, attention is not. Quality depends entirely on retrieving the right pieces, and the problems are the ones from RAG, plus some new ones:
What you do now (bootstraps, memory files, "COLD START" pointers) is the curated, hand-written version of the same thing. The trend automates writing and retrieving memory. The likely direction for you is not replacing it but adding to it:
That combination is where most of these systems seem to be heading: a small memory that is always in context, and a large one that is searched.
- Mem0, Zep, LangMem: memory layers that extract facts from conversations and retrieve them later. Zep keeps a knowledge graph with dates on each fact, so it can tell what was true when.
- Vendors' own memory: Anthropic's memory tool and context editing, and ChatGPT's memory.
- Chroma: published research on "context rot". Models get measurably worse as the context grows, even far below the window limit. That's the argument for retrieving a small relevant slice rather than filling a million-token window. Their move toward memory follows from it.
The catch is in the phrase "almost infinite context": storage is unlimited, attention is not. Quality depends entirely on retrieving the right pieces, and the problems are the ones from RAG, plus some new ones:
- What to write: remembering everything buries the signal.
- Facts that change: an old decision and its reversal both get retrieved. This is why temporal graphs exist.
- Contradictions and confident errors in automatically extracted "facts".
- Privacy: what is kept about whom.
What you do now (bootstraps, memory files, "COLD START" pointers) is the curated, hand-written version of the same thing. The trend automates writing and retrieving memory. The likely direction for you is not replacing it but adding to it:
- Keep the curated layer: decisions, rules, current state. It's small, you've checked it, and it's always loaded.
- Add automatic, searchable memory for the long tail: past sessions, logs, superseded plans, retrieved only when relevant.
That combination is where most of these systems seem to be heading: a small memory that is always in context, and a large one that is searched.
No comments:
Post a Comment