Which Agent Memory System Should You Choose? Mem0 vs. LangMem, Zep, and Graphiti

AI-agent memory is not merely a way to retain chat history. It is the production system that extracts, scopes, retrieves, corrects, and deletes the facts, preferences, events, and instructions that can shape an agent’s next action. A good memory layer makes an agent feel continuous and useful; a weak one can surface stale data, leak cross-tenant context, or turn untrusted retrieved text into a prompt-injection path.

The architectural choice is therefore consequential: choose Mem0 for a focused layer that extracts and retrieves scoped facts; LangMem for configurable memory formation, consolidation, and prompt refinement in LangChain or LangGraph; a temporal graph when relationships, provenance, and historical truth are central; or a custom vector-database stack when the data model, deployment, and compliance controls must be fully owned. This guide compares those alternatives across memory types, retrieval, governance, lifecycle management, runtime fit, and production trade-offs—so the final choice survives contact with real workloads.

1. First, define what “memory” means

An isolated LLM inference call does not retain state from an earlier call. Some model APIs persist conversation objects or execution state, but an agent still needs an application or provider layer to save, select, and supply prior information as model context. Agent memory is the system that decides what information should persist, retrieves it later, and fits it into a limited context budget. For the broader architectural patterns behind this loop, see memory in agentic systems.

This is not identical to retrieval-augmented generation (RAG), although the two can use the same retrieval techniques and can overlap. RAG commonly retrieves source material such as a handbook or private document collection, while memory commonly retains facts, preferences, plans, and prior outcomes derived from an agent’s interactions. A support assistant might retrieve a return policy from RAG and the customer’s preferred language from memory in the same response.

1.1 The four jobs of a memory system

Before comparing tools, separate the jobs that a memory system can perform:

1. Write: Convert an event or conversation into a durable representation.
2. Organize: Associate the representation with a user, tenant, agent, project, time, source, and retention policy.
3. Retrieve: Select the small set of records that helps answer the current request.
4. Maintain: Correct conflicts, expire stale records, honor deletion, audit access, and prevent unsafe data from reaching the prompt.

Some alternatives cover substantial portions of all four jobs as a managed service. None can completely own application-specific truth, authorization, consent, or end-to-end compliance. Other alternatives provide only storage primitives. A storage primitive is not inferior, but it moves extraction, conflict resolution, and lifecycle policy into your application.

1.2 The memory types change the choice

The LangMem conceptual guide usefully distinguishes three categories:

Memory type	What it preserves	Agent example	Human analogy	Typical representation or storage	Usually needs
Semantic	Facts, concepts, preferences, and relationships that can remain useful across interactions	A user prefers concise updates; Python is associated with programming; a project uses PostgreSQL	Knowing that Python is a programming language	A structured profile for a small stable set of fields, or a searchable collection of atomic facts and entity relationships	Fact extraction, identity scope, deduplication, correction, and relevance-based recall
Episodic	Time-bound experiences, actions, observations, and outcomes	A successful tool-use trace; a summary of an earlier conversation; the steps that resolved error E42	Recalling what happened on the first day at a new job	A chronological or append-oriented collection containing events, summaries, traces, timestamps, and provenance	Event capture, sequence and temporal retrieval, outcome labels, summarization, and retention rules
Procedural	Instructions and learned patterns that shape how the system behaves	A response style, agent persona, tool-use policy, or rule to validate before deployment	Knowing the practiced steps for riding a bicycle	Versioned prompt rules, policies, skills, workflow definitions, or a reviewed collection of reusable procedures	Controlled updates, approval, versioning, rollback, conflict handling, and behavioral evaluation

Mem0 is especially natural for semantic facts and lightweight agent memory. For a focused walkthrough of this use case, see Mem0: Building Persistent Memory for AI Agents. LangMem can manage semantic profiles or collections, retain episodic examples, and refine procedural instructions from feedback. A coding agent that continually improves its instructions may instead fit Letta’s file-oriented approach. A financial or supply-chain agent that must answer “what was true on March 1?” may benefit from a temporal graph. There is no single best memory system independent of the memory type.

2. Mem0 in one minute

Mem0’s core flow is deliberately direct: send messages to add, let the system extract durable memories, then call search before the next LLM request. By default, it stores extracted facts rather than a verbatim transcript. Its documentation describes semantic, keyword, entity, and temporal signals at retrieval time, with capabilities varying between the managed Platform and open-source deployment.

Conceptually, a Mem0 record contains a concise statement, identity scope, and metadata:

{
  "memory": "The user prefers concise weekly updates with risks first.",
  "user_id": "user_42",
  "project_id": "billing-migration",
  "created_at": "2026-09-04T10:30:00Z",
  "source": "explicit_user_statement"
}

Mem0 maintains the facts and metadata in SQL, embeddings in a vector store, and entity information in a separate entity store. Its managed Platform adds features such as Graph Memory, Memory Decay, Temporal Reasoning, and Dream, while the open-source version lets teams select their own model, embedding provider, and vector database. The Platform versus OSS guide is the source of truth for the current feature boundary.

2.1 Where Mem0 is a strong fit

Mem0 is usually a strong starting point when:

• An existing chatbot, workflow, or agent needs persistent user, project, or agent facts without replacing its orchestration framework.
• The team wants add, search, update, and delete operations instead of designing an extraction and retrieval pipeline from scratch.
• The main evidence is conversational, relatively compact, and naturally expressed as reusable facts.
• The application can enforce the appropriate scope on every write and read using one or more consistent identifiers such as user_id, agent_id, run_id, and metadata filters.
• A managed option is attractive, or an open-source SDK with configurable components is required.

2.2 Where Mem0 is not the whole answer

Mem0 is not a canonical database for live account balances, permissions, legal policy, or regulated source records. Use a system of record or a controlled retrieval tool for those facts. It is also not a replacement for document RAG when the model needs citations to source passages, nor is it a complete agent runtime with planning, tool execution, retries, and human approval.

The Mem0 paper reports favorable results on long-conversation benchmarks, but no benchmark can choose a production architecture for you. Compare candidates with your own identities, correction patterns, privacy constraints, and latency budget.

3. The comparison at a glance

The following table compares design centers, not marketing claims or universal rankings. “Application-owned” means the tool provides a useful primitive but your team must decide the write policy, schema, and lifecycle rules.

Option	Design center	Representation	Main advantage	Main trade-off
Mem0	Drop-in long-term memory layer	Extracted facts, metadata, embeddings, entities	Focused add and search memory loop with managed or OSS paths	Still needs application policy for truth, consent, and scope
LangMem	Memory-management library with LangGraph integration	Semantic profiles or collections, episodic examples, procedural prompt rules	Configurable extraction, consolidation, and prompt refinement in hot or background paths	Storage, authorization, scheduling, and runtime integration remain application-owned
Zep	Managed context infrastructure	Temporal Context Graph	Facts, entities, episodes, and validity-aware context for evolving data	Greater platform commitment and a graph-oriented mental model
Graphiti	Open-source temporal graph framework	Entities, dated relationships, provenance episodes	Fine control over time, relationships, and graph backend	More infrastructure and data-model design to operate
Letta	Stateful agent harness	Git-backed Markdown memory files, agent state	Agent can inspect, edit, version, and reorganize its own durable context	Not a simple per-request fact-retrieval API by default
LangGraph	Agent orchestration with memory primitives	Thread checkpoints plus namespaced JSON stores	Tight integration with graph state, custom namespaces, and workflows	Extraction, consolidation, and governance are application-owned
LlamaIndex Memory	Configurable agent memory component	FIFO history plus static, fact, and vector blocks	Composable prompt-budgeted blocks within LlamaIndex agents	Best fit is inside the LlamaIndex ecosystem
Supermemory	Context and knowledge layer with hosted and self-hosted paths	Documents, chunks, temporal vector-graph memories, profile	Combines multimodal ingestion, source chunks, graph memory, and profiles	Asynchronous ingestion and product-specific processing semantics require careful workflow design
Qdrant or another vector database	Retrieval infrastructure	Vectors plus payload metadata	Maximum control and portability of the retrieval layer	You build fact extraction, updates, evaluation, and safe lifecycle management

Capability and ownership matrix

A useful comparison needs more than a single “supports memory” checkbox. The following matrices use the union of the major capabilities offered across these tools. They describe each product’s design center, not every feature that could be added with custom code.

• Built in: The tool directly implements the capability as a primary feature.
• Primitive: The tool supplies a usable building block, but the application must define the schema, policy, trigger, or integration.
• Optional: The capability depends on a managed edition, extension, provider, or adjacent product.
• App: The application team must implement or integrate the capability.

Product features and edition boundaries can change. Treat the matrix as an architectural ownership guide, then verify the selected version and deployment model against current documentation.

Regardless of the selected tool, the application team should explicitly own:

1. Source-of-truth boundaries: Decide which information may be remembered and which must be fetched from an authoritative operational system.
2. Write policy: Define what is durable, what requires confirmation, what is too sensitive to store, and what should remain only in the current conversation.
3. Identity and authorization mapping: Translate the authenticated caller into tenant, user, agent, project, and purpose scopes before every read or write.
4. Conflict policy: Specify whether recency, explicit user correction, source reliability, or human approval wins when records disagree.
5. Context assembly: Set retrieval limits, token budgets, ranking rules, provenance requirements, and defenses against instructions embedded in retrieved data.
6. Lifecycle compliance: Implement retention schedules, legal holds, export, correction, and deletion across primary stores, indexes, caches, backups, logs, and derived memories.
7. Reliability behavior: Define timeouts, retries, fallbacks, and the response shown when memory is unavailable or stale.
8. Evaluation: Measure retrieval quality, temporal correctness, cross-tenant leakage, deletion completion, latency, token usage, and downstream answer quality on representative workloads.
9. Economics and portability: Model ingestion, storage, retrieval, model, and operations cost, and test whether data and metadata can be exported without losing semantics.

4. Mem0 versus memory-first platforms

Memory-first platforms own more of the path from raw events to assembled context than a framework store or vector database. The meaningful comparison is therefore not whether each product can retrieve text. It is which inputs the platform accepts, how it represents change, and how much of the memory lifecycle remains in application code.

4.1 Mem0 vs. Zep: compact facts versus a temporal Context Graph

Shared ground. Mem0 and Zep both turn interactions into persistent, scoped context and retrieve relevant information for later responses. Both can represent entities and changing facts, although Mem0’s graph and temporal features depend on the deployment.

Where they diverge. Zep centers on a temporal Context Graph built from facts, entities, episodes, summaries, and observations. It can ingest messages, business data, documents, and JSON, then preserve the period during which a relationship was valid. That model suits historical and relationship-rich questions, such as which supplier served a warehouse during a given month and which event changed the assignment.

Mem0 centers on a narrower loop: submit messages, extract compact memories, retrieve them by scope, and assemble prompt context. It is usually easier to introduce when an existing application mainly needs preferences, decisions, and confirmed outcomes. Mem0 Platform can extend this model with Graph Memory, but the open-source configuration does not include that managed feature.

Decision rule. Prefer Zep when temporal relationships and source episodes are part of the normal query path. Prefer Mem0 when compact fact recall is the primary need and a smaller integration boundary matters more than a graph-centered representation.

Both products still require application-level authorization, tenant isolation, and access policy.

4.2 Mem0 vs. Supermemory: fact retrieval versus unified context ingestion

Shared ground. Mem0 and Supermemory both derive durable context from interactions, provide mechanisms for scoping memory, and retrieve it for later prompts. Correct isolation still depends on using those mechanisms consistently and enforcing authorization in the application.

Where they diverge. Supermemory expands the boundary to files, URLs, media, and, on its hosted platform, connector data. It can produce source chunks, graph memories, and a profile within a containerTag, making it suitable when grounded source retrieval and personal context should share one ingestion pipeline. Its dynamic mode performs consolidation asynchronously, while instant mode prioritizes immediate availability. This distinction affects read-after-write behavior, so workflows that query newly uploaded content must check ingestion status rather than assume that every derived representation is ready.

Mem0 is the more focused choice when document ingestion and RAG already exist and the missing capability is durable fact memory. Prefer Supermemory when unified, multimodal context ingestion is the requirement; it offers hosted use as well as a self-hosting path. The self-hosted edition provides the Memory API, file ingestion, and hybrid search, but it uses your chosen model and does not include the hosted connectors or Supermemory MCP. Prefer Mem0 when replacing an existing content pipeline would add more complexity than value.

5. Mem0 vs. Graphiti: memory API versus a buildable temporal graph

Mem0 and Graphiti overlap in retrieval, but they sit at different abstraction levels. Mem0 presents a memory API. Graphiti provides an open-source framework for building and querying a temporal knowledge graph. Both turn source events into retrievable context and both require the application to define identity scope and safe context use.

Graphiti models:

• Episodes, the source events or documents that produced knowledge.
• Entities, the people, products, and concepts involved.
• Facts or relationships, represented as edges with validity windows.
• Provenance, which lets a derived fact point back to its source episode.

Its hybrid retrieval combines semantic similarity, keyword matching, and graph traversal. For information that changes over time, a relation can be represented with a validity interval $[t_{\mathrm{valid_from}}, t_{\mathrm{valid_to}})$ rather than a single undifferentiated string. A current-state query should prefer facts whose interval includes now; a historical query should ask for the relevant time slice. This requirement is a form of point-in-time correctness.

$$
\operatorname{active}(f,t) =
\begin{cases}
1, & t_{\mathrm{from}} \leq t < t_{\mathrm{to}} \\
1, & t_{\mathrm{from}} \leq t \ \text{and}\ t_{\mathrm{to}}\ \text{is unknown} \\
0, & \text{otherwise.}
\end{cases}
$$

Validity time should also be distinguished from ingestion time. A contract may be uploaded today but have been valid since January. Storing only created_at answers when the system learned a fact, not when the fact was true.

Graphiti provides explicit control over time-indexed relationships, provenance, graph schemas, and graph backends. It fits fraud investigation, supply chains, incident management, and account histories, but requires a graph database, reliable structured extraction, and more operational expertise. Mem0 has a smaller API surface and a lower integration burden for primarily fact-oriented recall. Its Platform can add graph and temporal capabilities without requiring the graph to become the application’s primary data model.

Decision rule. Choose Graphiti when the graph itself is an application asset that the team wants to model, inspect, and operate. Choose Mem0 when the application wants memory behavior behind an API and does not need direct ownership of graph semantics.

6. Mem0 versus framework-integrated memory

These options are not always mutually exclusive. Framework-integrated memory lives inside an agent harness and participates directly in checkpoints, workflows, and prompt construction. Mem0 can instead act as a shared service used by several runtimes. The choice is primarily about ownership and reuse, not raw retrieval quality.

6.1 Mem0 vs. LangGraph: a standalone service boundary versus programmable primitives

Shared ground. Mem0 and LangGraph both persist scoped information across interactions. They can also work together, with LangGraph orchestrating the agent and Mem0 supplying long-term facts.

Where they diverge. LangGraph separates short-term thread checkpoints from long-term, namespaced JSON stores. It gives the workflow direct control over state, schemas, write triggers, and human approval, but the application must decide what becomes memory or add LangMem for extraction and consolidation. Mem0 provides a framework-independent add and search loop with less custom memory logic.

Choose native LangGraph storage when memory updates are tightly coupled to graph nodes and checkpoint state. Add Mem0 when the same long-term facts must be reused outside one graph or when the team wants a dedicated extraction and retrieval layer.

6.2 Mem0 vs. LangMem: a product memory loop versus a programmable memory manager

Shared ground. Mem0 and LangMem both extract and update semantic memory, organize it by scope, and retrieve it later. Both can form memories during a request or in a background workflow.

Where they diverge. LangMem is a programmable library for schema-constrained profiles, searchable collections, episodic examples, consolidation, and procedural prompt optimization. It offers fine-grained control inside LangChain or LangGraph, including custom storage and background workflows. That control leaves storage, scheduling, prompt rendering, and access enforcement to the application. Mem0 provides a more self-contained, framework-independent product loop with managed and open-source deployment paths.

Choose LangMem when memory formation is part of a programmable LangGraph workflow or when procedural prompt learning is required. Choose Mem0 when several applications need a consistent memory service or when reducing library-level orchestration work is more important than controlling every update step.

6.3 Mem0 vs. LlamaIndex Memory: standalone layer versus composable prompt blocks

Shared ground. Mem0 and LlamaIndex Memory both retain long-term facts and retrieve relevant context for an agent prompt.

Where they diverge. LlamaIndex Memory combines a short-term FIFO queue with prioritized static, fact-extraction, and vector-retrieval Memory Blocks. Its main strength is explicit, token-budgeted prompt composition inside LlamaIndex agents. Mem0 creates an independent service boundary that can provide consistent memory across frameworks and applications.

Choose LlamaIndex Memory when the agent already uses LlamaIndex and the team wants block-level control over the prompt budget. Choose Mem0 when memory must remain portable across runtimes. LangChain or LangGraph teams should also evaluate LangMem before introducing another service boundary.

6.4 Mem0 vs. Letta: retrieved facts versus an agent-owned memory filesystem

Shared ground. Mem0 and Letta both preserve context across turns and let agents reuse or update that context over time.

Where they diverge. Letta’s MemFS is a Git-backed filesystem that agents edit with file tools. It makes project notes, conventions, skills, and other long-lived context visible, versioned, and reviewable. Files in system/ load on every turn, while reference files are read on demand; semantic search requires an optional extension. Mem0 automatically extracts and retrieves scoped facts through an application-controlled API, which better fits products serving many users or services.

Choose Letta when the agent should curate a durable workspace that humans can inspect and roll back. Choose Mem0 when memories should be small records selected automatically for each request.

7. Mem0 versus the build-it-yourself vector approach

Mem0 and a custom vector stack can both embed records, attach metadata, filter by scope, and rank relevant context for later prompts. The difference is that a vector database solves storage and retrieval, while a memory layer also defines how candidate records are created, revised, and retired.

A vector database such as Qdrant gives complete control over schemas, models, deployment, filtering, and hybrid ranking. That control matters when compliance, portability, or domain logic is non-negotiable. However, the application must implement extraction, conflict handling, provenance, retention, and deletion. Mem0 packages more of that lifecycle behind a smaller API, reducing development and operational work. It still leaves truth, consent, authorization, and safe prompt use to the application.

7.1 The retrieval mathematics are not the hard part

Given a query embedding $\mathbf{q}$ and memory embeddings $\mathbf{m}_i$, a simple semantic retriever ranks by cosine similarity:

$$
s_{\mathrm{semantic}}(q,m_i) =
\frac{\mathbf{q}^{\top}\mathbf{m}_i}
{\lVert\mathbf{q}\rVert_2\lVert\mathbf{m}_i\rVert_2}.
$$

You can blend this score with lexical, recency, and trust signals:

$$
s(q,m_i) = \alpha\widetilde{s}_{\mathrm{semantic}} + \beta\widetilde{s}_{\mathrm{lexical}} + \gamma\widetilde{s}_{\mathrm{recency}} + \delta\widetilde{s}_{\mathrm{provenance}}.
$$

The tildes mean that component scores have been normalized to comparable ranges. The coefficients should be chosen and evaluated for your task, not copied as universal constants. See text embeddings and approximate nearest-neighbor search for the underlying retrieval concepts. If you use a separate relevance model after first-stage retrieval, reranking in RAG explains the associated trade-offs.

Authorization must constrain which records can leave the trusted retrieval boundary, and hard scope filters should determine candidate eligibility rather than act merely as ranking signals. An implementation may rank within an already authorized candidate set or apply authorization before returning internally ranked results, but a high similarity score must never make an out-of-scope record eligible. The broader system still has to extract atomic claims, assign scopes, merge corrections, detect prompt injection, retain provenance, implement deletion, and prove isolation. Mem0, LangMem, and the other higher-level alternatives are valuable precisely because they package some portion of that work.

Offline retrieval metrics such as precision at $k$ and recall at $k$ are useful but incomplete. A production evaluation should also test whether the retrieved records improve the final answer, whether stale facts displace current ones, and whether the system correctly returns no memory when none is relevant.

7.2 When custom retrieval wins

Build directly on a vector database when you need one or more of the following:

• A strict domain schema that is already defined in your operational database.
• On-premises or air-gapped deployment with a locally governed model stack.
• A custom ranking function that blends business rules, permissions, structured filters, and multiple evidence types.
• An existing event pipeline that already emits trusted, normalized facts.
• Full ownership of storage topology, retention, observability, and migration paths.

Do not choose this route merely to avoid a dependency. You replace an external dependency with a continuing engineering responsibility.

Conversely, do not choose a higher-level memory product merely to avoid writing retrieval code. A custom stack can be simpler when trusted structured facts already exist, no generative extraction is needed, and the team already operates the required databases and policy controls.

8. A practical decision framework

Treat selection as a sequence of gates, not as a feature-count contest. The first gates remove unsuitable uses of memory. The later steps produce a shortlist and test whether any candidate improves the real workload enough to justify its cost and risk.

8.1 Step 1: remove authoritative facts from the memory decision

Start by labeling every candidate fact as authoritative or assistive:

• Authoritative facts control accounts, shipments, entitlements, payments, permissions, or legal decisions. Read them from the canonical system at decision time. Memory may retain a pointer or conversational summary, but it must not silently become a second source of truth.
• Assistive context improves continuity without controlling the underlying transaction. Preferences, project conventions, prior outcomes, and reusable examples are reasonable memory candidates.

This is a hard gate. If stale or inferred content could authorize an action, move that content out of the memory layer before comparing products.

8.2 Step 2: identify the dominant memory shape

Choose the branch that describes most queries, not every possible input:

Dominant need	Questions to ask	First shortlist
Atomic facts or profiles	Is the answer usually a preference, decision, goal, or compact user or project fact?	Mem0; LangMem profiles or collections; a framework-native store
Temporal relationships	Must the system explain what was true at a particular time, how entities were related, and which episode changed the relationship?	Zep for managed context infrastructure; Graphiti for a self-managed temporal graph
Editable agent knowledge	Should the agent and humans inspect, revise, version, or roll back instructions, skills, and project notes?	Letta for an agent-owned workspace; LangMem when the learned artifact is a prompt inside a LangGraph workflow
Documents, media, and personal context	Should source chunks, multimodal ingestion, graph memories, and profiles share one pipeline?	Supermemory; otherwise retain the existing RAG pipeline and add a focused memory layer only where needed

Mixed workloads do not require one product to own everything. A common architecture keeps operational facts in a system of record, documents in RAG, and personal preferences in a fact-memory layer.

8.3 Step 3: account for the existing agent runtime

Prefer the smallest addition to the architecture that already exists:

• In LangGraph, start with checkpoints and namespaced stores. Add LangMem when extraction, consolidation, episodic examples, or prompt refinement is required.
• In LlamaIndex, test native Memory Blocks before introducing another service boundary.
• Use Mem0 when several runtimes need the same fact-memory API, or when a dedicated extraction and retrieval layer removes significant application logic.
• Use Letta when the durable, editable agent workspace is itself the desired runtime model.

Framework fit is a cost consideration, not an overriding requirement. Do not force a temporal relationship workload into a flat framework store merely to avoid another component.

8.4 Step 4: choose the deployment and ownership boundary

Now apply non-functional constraints to the shortlist:

• Managed service: less infrastructure work, but a vendor processes data and defines some limits for retention, observability, availability, and export.
• Self-hosted library or server: more control over data placement and components, but your team owns scaling, backups, upgrades, security patches, and incidents.
• Custom vector or graph stack: justified when air-gapped deployment, a strict domain schema, custom ranking, existing trusted fact pipelines, or portability requirements cannot be met by the higher-level options.

Check actual edition boundaries. “Open source” does not guarantee feature parity with a hosted product, and “managed” does not remove the application’s authorization or compliance duties.

8.5 Step 5: apply non-negotiable production gates

Every candidate should pass tests for:

1. Scope isolation: a query from tenant A cannot retrieve tenant B’s data.
2. Correction: a recent explicit correction is preferred over a stale claim for present-tense questions.
3. Provenance: operators can explain where a critical memory came from.
4. Deletion: a user’s deletion request reaches every configured store and future retrieval path.
5. Prompt-injection resistance: imported or retrieved text cannot silently become higher-priority instructions.
6. Graceful failure: an unavailable memory service does not cause the agent to invent remembered facts.
7. Export and recovery: the team can export useful data and metadata, restore service, and prevent deleted records from reappearing after recovery.

Reject a candidate that fails any hard gate, even if its average retrieval score is strong. A memory outage should degrade to “I do not have the relevant prior context,” not to a confident fabrication.

8.6 Step 6: run a workload-specific bake-off

Evaluate at least two viable candidates, including the current system or a no-memory baseline, with the same event stream, identity scopes, corrections, and queries. Measure the complete loop rather than search in isolation:

• Memory formation: extraction precision, unsupported claims, deduplication, and time until a write becomes searchable.
• Retrieval: relevant-record recall, irrelevant context rate, temporal correctness, and correct abstention.
• Task outcome: answer accuracy, citation or provenance quality, tool-use success, and human preference.
• Safety and lifecycle: cross-scope leakage, correction behavior, deletion completion, and prompt-injection resistance.
• Operations: latency percentiles, token use, model and storage cost, failure recovery, and export fidelity.

Evaluating RAG systems and agentic-system evaluation provide complementary methods for testing retrieval quality and final agent outcomes. Record the selected architecture, rejected alternatives, assumptions, acceptance thresholds, and an exit plan. A small gain in answer quality may not justify a large increase in latency, privacy exposure, or operational burden.

9. Production practices that matter regardless of tool

• Make scope non-optional: Pass a tenant or user scope on every read and write. Enforce it in server-side authorization and database filtering before retrieved text reaches the LLM. Do not rely on an instruction in the prompt to protect data that should never have been retrieved.
• Treat retrieved memory as untrusted data: Memory can include user text, prior model outputs, imported documents, and extraction errors. Keep it in a visibly labeled prompt section. Never allow a retrieved string to override the system policy, tool permissions, or access-control decision. The threat model is closely related to prompt injection and should be addressed with the same seriousness as other LLM guardrails.
• Design the write policy before the retrieval policy: The cleanest search index cannot compensate for indiscriminate writes. Store explicit durable preferences and confirmed outcomes. Require review or confirmation for inferred preferences and sensitive facts. Never retain secrets, access tokens, or payment-card data. The broader privacy principles in protecting privacy in the age of AI apply directly here.
• Observe the complete memory loop: Trace the event that created a memory, the retrieval call, the selected records, and the response that used them. Record safe identifiers, policy decisions, latency, and token counts, but avoid copying personal memory text into general-purpose logs. OpenTelemetry can join these events across an agent workflow.
• Bound the context contribution: Limit both the number of memories and their token budget. More retrieved text can reduce answer quality by introducing stale or weakly related claims. Prefer a small, explainable set of records and preserve identifiers that let operators inspect why each record was selected.
• Test deletion as a workflow: Deleting a primary record is insufficient if embeddings, graph edges, caches, summaries, backups, or logs can still return its content. Track deletion as an auditable operation across every derived store and define how restored backups avoid resurrecting deleted memory.
• Design a memory-off mode: The application should remain safe when extraction, storage, or retrieval is slow or unavailable. Use bounded timeouts, avoid retry storms, expose degraded behavior to operators, and never convert a retrieval failure into an invented recollection.

Key takeaways

The alternatives divide into clear architectural families:

• Focused fact memory: Mem0 is a strong default when an existing application needs scoped, long-term facts without adopting a new agent runtime.
• Temporal context: Zep and Graphiti fit questions whose answers depend on relationships, provenance, and what was true at a particular time.
• Framework-native control: LangMem, LangGraph, and LlamaIndex fit teams that want memory updates and prompt construction inside the agent workflow.
• Agent-curated workspace: Letta fits long-running agents that should inspect, edit, and version their own context.
• Unified ingestion: Supermemory fits applications that want documents, media, source-grounded retrieval, profiles, and personal memory in one context layer, with hosted and self-hosted paths available.
• Custom infrastructure: A vector database remains the right foundation when complete control over schema, deployment, ranking, and lifecycle matters more than convenience.

Start with one narrow workflow and define the source of truth, write policy, identity scope, correction semantics, and deletion path before selecting a product. Then run the same workload and acceptance tests against two candidates. The best choice is the smallest system that safely improves the real task, not the product with the longest feature list.