When teams say "we need agent memory," they usually mean one thing: the model forgot something. So they bolt on a vector store, embed the conversation, and call it done. The retrieval works. That is the easy 20% of the problem.
The hard 80% shows up months later, in production, and it has almost nothing to do with embeddings. Who owns this agent's memory? What did it actually store, and can you prove it? When two agents write the same fact at the same time, which one wins? And when a user asks to be forgotten, can you actually delete it — not just mark it hidden?
Those are operations questions. Most of the agent-memory tooling in 2026 was not built to answer them.
The piece everyone solved: retrieval
The retrieval half of memory is genuinely solved, and the numbers are good. A selective external memory layer trades about six points of accuracy for huge savings over shoveling the whole history into the context window — roughly 1,764 tokens and 1.44s p95 latency per query instead of ~26,000 tokens and 17s, at about 90% lower cost on the LoCoMo benchmark. Even a two-megabyte context window is not a substitute: cost, latency, and the model reasoning through irrelevant noise all grow with window size.
The ecosystem has coalesced around a memory-layer architecture — vector stores for semantic recall, knowledge graphs for facts that evolve, key-value for preferences, plus MemGPT-style paging for what stays in the live window. Mem0, Zep, Letta, and LangMem all sell roughly this shape. For recalling "the user moved from NYC to San Francisco last quarter," this works.
None of it answers the question above: what did the agent remember, why, and can you trust it enough to act on it unattended.
What the model layer quietly punts on
Three unsolved problems live at the boundary between memory and operations:
Forgetting is not an engineering problem, it's a judgment problem. Storing and retrieving are solved. Deciding what to drop is still an open research question, and unbounded stores degrade over time as stale, contradictory entries crowd out relevant ones. A memory that is confidently wrong is worse than no memory.
Staleness leaks into every new session. A fact that was true — an employer, a location, a preference — stays high-ranked in retrieval long after it changes, and decay heuristics miss high-relevance stale entries. Multi-agent systems that treat a change as overwrite instead of evolution silently drift toward hallucinated recall.
Memory can be poisoned. It is not just bad retention hygiene. Studies of retrieval poisoning show that a handful of injected documents can flip a model to a false answer the large majority of the time. Once a wrong fact is written into an agent's store, it becomes an authority that re-cites itself.
The operations layer nobody owns
This is the gap. Memory-layer products today mostly lack ownership records, decision traces, and provable deletion:
- Ownership. Largely nobody owns an agent's memory. There is no record of who — which agent, which run, which human — wrote a given entry, or under what policy it was kept.
- Traceability. Memory records what was stored, not why a decision happened. When compliance asks for a point-in-time reconstruction — as the EU AI Act's logging requirements push toward from August 2026 — "here's a vector" is not an answer. You need the retrieved items, the stale items, the applied policy, and the reasoning chain.
- Deletion. GDPR Art. 17 gives users an erasure right, and no mainstream vector database provides a provable deletion mechanism. Teams fall back on ephemeral timelines and retention budgets, and treat "deleted" as "hidden."
- Multi-agent consistency. Concurrent writes to a shared store cause memory drift. This is the most-cited unsolved problem in the space, and content-level access — evaluated against the requesting identity, not just the query — is what prevents one unprivileged session from reading another's data.
In other words: the same list of operations problems that plague agents generally — identity, audit, governance, budgets — hit the memory layer hardest, because memory is where the organization's history actually lives.
Treat memory as infrastructure, not a pipeline stage
When you treat agent memory as application code, each agent gets its own private vector store and the ops questions go unasked. When you treat it as infrastructure, the defaults change:
- Each agent owns its memory under a cryptographic identity. Memory is scoped to a specific agent — the same agent next week — and every message it sends or reads is signed and verified, so a wrong fact can be traced to the run that wrote it.
- Memory is auditable by construction. A signed, time-ordered record of what was stored and retrieved is what makes "trust this unattended" defensible. Once writes are attributable, poisoning and drift become findable instead of invisible.
- Budget and retention are platform primitives. A daily cost cap applies to retrieval as much as generation; retention and erasure are callable operations, not hopes.
Alpi is built this way: persistent per-profile memory that survives across sessions, an Ed25519 identity per agent, signed messages that give you a real audit trail, and per-agent budgets. We did not set out to build a memory layer — we set out to operate agents as an organization, and memory is one of the first things that breaks when you do.
If your agent memory is still a vector insert and a prayer, the retrieval works today. Ask yourself, in a quarter, when the fact is wrong and a regulator or an angry user asks what the agent knew and why: can you answer? That is the problem worth solving first.
Alpi is free to install and source-available: uv tool install alpi-agent.