Agent memory systems from first principles — how agents remember across sessions, learn from experience, and forget safely. Why agents forget (LLMs are stateless; the context window is working memory — bounded, ephemeral, unevenly attended — and memory is the system around the model: selection, organization, retrieval, updating/forgetting), the memory hierarchy (working/episodic/semantic/procedural — different jobs, storage, retrieval patterns, and lifecycles; most 'remembered wrong' bugs are type confusions), conversation memory and compaction (the buffer → sliding-window → selective-eviction → compaction ladder; preservation contracts, structured rolling summaries, boundary-triggered timing, and demote-don't-delete into episodic storage), retrieval memory (memory ≠ document RAG; relevance × recency × importance scoring with access reinforcement; write-side hygiene — self-contained memories, dedupe at the door, attached structure; budgeted injection with provenance labels), structured and graph memory (where similarity search breaks — aggregation, multi-hop, identity, current state; entity-centric records with resolved, superseding claims; knowledge-graph edges with temporal validity; the hybrid end state where structure answers precisely and episodes keep the receipts), memory as tools (the OS view — context as RAM, store as disk, the model participating in paging policy; file-based memory as the transparent, diff-reviewable baseline; consolidation and reflection as the highest-privilege writer, reviewed and evaluated like one), learning from experience (rules, playbooks, worked examples, failure memory; the act→attribute→reflect→encode→apply loop and its failure modes — misattribution, overgeneralization, rot, conflict; deploy-grade brakes on procedural writes), and memory in production (personal-data obligations, deletion that cascades along provenance, hard tenant isolation, memory writes as a prompt-injection surface, and layered evaluation ending in the memory-off ablation). Grounded in Wikipedia (working memory, LLM, episodic/semantic/procedural memory, short-term memory, automatic summarization, vector database, information retrieval, knowledge graph, ontology, virtual memory, memory consolidation, transfer learning, GDPR, data retention).
Every production agent has the same embarrassing failure mode: ask it to continue yesterday's work and it has no idea what you're talking about. The model that reasoned brilliantly for two hours retains nothing of those two hours. This is not a bug to patch but an architecture to build — memory is the system around the model, and designing it well is becoming the distinguishing skill of agent engineering. This series builds agent memory from first principles.
LLMs are stateless, and the context window is working memory — bounded, ephemeral, and unevenly attended — not memory. Forgetting costs re-work, broken user relationships, repeated mistakes, and drift, and no model upgrade fixes it: memory is the system around the model. Opens an 8-post series on building it.
Cognitive science spent a century mapping human memory into distinct systems — working, episodic, semantic, procedural — and agent engineering has been rediscovering the same partition from the opposite direction, because the same pressures produce the same shapes. The hierarchy isn't biomimicry for its own sake: each memory type answers a different question, is stored differently, retrieved differently, and fails differently. Confusing them is the root of most memory-system design mistakes.
Agent memory decomposes into four types with different jobs: working (the context window), episodic (what happened — read as history), semantic (what is true — needs revision, not just insertion), and procedural (how to act — evolves like code). They differ in storage, retrieval, and lifecycle, and most memory bugs are type confusions.
The first memory problem every agent hits is the humblest: the conversation itself outgrows the window. Tool outputs pile up, reasoning accumulates, and somewhere past the first hour the agent must start forgetting parts of its own session — the only question is whether it forgets with judgment or at random. Compaction is where memory engineering starts, and it's harder than it looks because the thing doing the summarizing is the thing that needs the summary.
The strategy ladder — buffer, sliding window, selective eviction, compaction — and why the top rung needs engineering: a preservation contract for what survives verbatim, structured rolling summaries, boundary-triggered timing, and the architectural rule that compaction should demote to episodic storage, never delete.
The standard answer to agent memory is "embed it and put it in a vector store" — and it's half right. Semantic search over past experience is genuinely the workhorse of episodic recall. But raw similarity is a weak model of what "relevant memory" means: similar is not important, recent, or true. The difference between a memory system and a junk drawer with an embedding index is the scoring, filtering, and hygiene this post covers.
Memory retrieval is not document RAG: the store is self-generated, time-sensitive, and self-contradicting. Score on relevance x recency x importance, enforce write-side hygiene — self-contained memories, dedupe at the door, attached structure — and inject under a budget with provenance labels so memories are weighed, not absorbed.
Embeddings are a marvel of forgiving representation: throw anything in, get approximate recall out. But some of what an agent must remember isn't approximate — who reports to whom, which service depends on which, what the user decided and when they reversed it. For facts with structure, similarity search is the wrong instrument, and the fix is as old as databases: give the facts a shape. This post is about entity-centric and graph-shaped memory, and when the extra machinery pays.
Similarity search breaks on aggregation, multi-hop, identity, and what-is-true-now. The fix is shape: entity-centric memory with resolved, superseding claims as the 80% answer; knowledge-graph edges (owns, depends_on, supersedes) where relations are the content; and a hybrid end state where the structured layer answers precisely and the episodic store keeps the receipts.
Everything so far treated memory as plumbing the harness operates around a passive model. The most interesting turn in recent agent design flips that: give the model tools to read, write, and reorganize its own memory, and let it decide what's worth keeping. The operating-system analogy arrived early — a small, fast context window backed by a large, slow store, with something paging between them — and it keeps paying dividends. This post covers agentic memory: the OS view, file-based memory, and consolidation.
The agentic turn: context as RAM, store as disk, the model participating in paging policy — plus file-based memory as the transparent, diff-reviewable baseline, and consolidation as the background process that extracts facts, distills lessons, and revises standing memory. The consolidator is the highest-privilege writer in the system; review and evaluate it like one.
Everything before this post helps an agent remember; this one asks whether it can improve. The difference is procedural: an agent that recalls last week's failed deploy has memory, while an agent that deploys differently because of it has learned. Procedural learning is the highest-leverage loop in agent engineering — every improvement compounds across all future tasks — and the most dangerous, for exactly the same reason. The craft is building the loop with brakes.
Learning without weight changes lands in artifacts: rules, playbooks, worked examples, failure memory. The loop breaks at misattribution, overgeneralization, rot, and conflict — and procedural writes deserve deploy-grade brakes: proposal review, provenance, blast-radius scoping, regression evals. Start with failure memory and worked examples; they compound at a fraction of the risk.
A memory system that delights in a demo becomes a liability register in production: it accumulates personal data by design, remembers things users expected to expire, must never leak across tenants, and silently decays in quality unless something measures it. This closing post covers the four disciplines that separate a production memory system from a demo with persistence — privacy, deletion, isolation, and evaluation — and ends where post 1 began: with judgment about what's worth remembering at all.
Production memory is a personal-data system: minimization as selection, answerable 'what do you remember about me', sensitivity tiers, deletion that cascades along provenance edges, hard tenant isolation (memory writes are an injection surface), and layered evals ending in the ablation that proves memory earns its cost. Remembering well is valuable; forgetting well is trustworthy. Closes the series.
This series is part of a larger body of work by Pratik Dhanave, an Agentic AI Architect writing about production AI systems, distributed systems, and cloud-native engineering. Explore all course series, browse every post, or find topics via the tag index.