How AI Memory Systems Work: RAG, Knowledge Graphs, and Persistent Context

RAG: retrieve, then generate

The 2020 paper by Lewis and colleagues introduced a model architecture combining parametric memory in a pretrained model with non-parametric memory retrieved from an external index. In a practical application, documents are segmented and indexed; a question retrieves candidate passages; a model receives those passages and produces a response. Sources: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)

RAG can make answers depend on current or private source material without retraining model weights for every change. The original paper studied knowledge-intensive NLP tasks and a Wikipedia index. It does not establish that every current commercial RAG implementation is accurate, private or suitable for a particular work context. Sources: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)

Knowledge graphs: represent relationships

A knowledge graph represents entities and typed relations: a person owns a commitment; a decision affects a project; a meeting discussed a customer. Unlike plain similarity search, graph traversal can make explicit connections among records. This can help answer questions requiring multiple hops or exact relationship types.

Graphs require difficult maintenance. Two names may refer to the same person or different people; an inferred relationship may be wrong; old edges may no longer be true. A graph should retain source, time and confidence for important facts, and support corrections and historical state.

Persistent context: what carries across sessions?

A system may store user preferences, summaries, events, entity records or references to source documents. “Persistent memory” is an umbrella description, not a specific algorithm. The product must decide what to retain, how to update it, who can access it, and when it expires. A compact summary is cheap to retrieve but can omit details; a full archive is richer but harder to govern and search.

Many systems combine lexical search, vector similarity, metadata filters and graph relationships. Retrieval may use query rewriting, reranking or a structured database. The model’s context window is not a substitute for durable storage: it holds selected material for one inference, while persistence belongs to the application’s data layer.

A reference architecture

A responsible pipeline can be described in stages: ingest approved sources; normalize and retain provenance; extract candidate entities and events; index text and structured relations; retrieve a permission-filtered set for a query; generate an answer with citations; and let the user inspect, correct or reject it. Every stage can fail.

Test retrieval separately from generation. Does the relevant source appear among candidates? Did permission filters exclude unauthorized records? Did the answer accurately reflect retrieved evidence? Is the record current? A fluent answer can mask a retrieval miss, so evaluation should include adversarial names, conflicting dates and superseded decisions.

ProfMe and implementation boundaries

ProfMe’s public product pages describe proposed workflows for capturing or importing work material, creating transcripts and summaries, organizing professional memory, searching, asking questions with source quotations and preparing for meetings. Those pages do not confirm public availability or prove a particular internal RAG or graph implementation. This article therefore does not attribute a specific architecture to ProfMe. Sources: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020), ProfMe search and provenance

ProfMe’s versioned policy pages describe stated provider routes and data practices, but they do not independently verify which routes or controls are active for every account. Product teams should document actual configurations before making stronger operational claims. Sources: ProfMe Privacy Policy

Tradeoffs and evaluation

RAG can return irrelevant or incomplete evidence. Graphs can amplify extraction errors. Persistent context can become stale or overly broad. Mitigations include source-level permissions, minimum necessary retention, timestamps, explicit uncertainty, deletion workflows and human review for consequential decisions. Benchmarks should use representative questions and measure grounded correctness, retrieval recall, citation support, stale-answer rates and permission violations. Sources: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)

The building blocks: evidence, records, indexes, and answers

A memory system begins with source material: an approved transcript, document, note, or structured event. Ingestion identifies usable content and metadata such as speaker, date, project, and permissions. Processing may split long sources into passages, extract candidate entities or actions, and store links back to the original. Search then retrieves material relevant to a question; an answer component may summarize it. Each stage can introduce a different failure, so a single “AI accuracy” score hides useful detail.

A robust design retains a provenance chain. For an answer claim, a reviewer should be able to see the source, relevant passage, event time, processing status, and whether the fact was extracted or directly recorded. If a source is deleted or access is revoked, derived records and indexes need an explicit lifecycle rule. These are system requirements to check; this article does not assert that ProfMe implements a specific database or pipeline. Sources: ProfMe search and provenance

RAG and structured memory solve different parts of the task

Retrieval-augmented generation retrieves passages and supplies them to a language model when answering. Lewis et al. introduced a research system combining parametric model knowledge with a non-parametric Wikipedia index. The method can provide current external evidence to a generator, but retrieval misses, stale indexes, and unsupported synthesis remain possible. A citation must point to evidence that actually entails the answer, not merely a related passage. Sources: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)

Structured memory stores typed facts and relationships, such as a decision with a date, owner, status, and source. It can answer constrained questions consistently, but a rigid schema may miss nuance or make extraction brittle. A hybrid design can retrieve passages for context while using structured fields for status and relationships. A graph is one representation for those links; it does not guarantee better answers. Choose the smallest representation that supports the questions and governance needs.

Extraction and consolidation need uncertainty controls

Extraction turns natural language into candidate records: “I can send the updated draft by Friday” might become a commitment with an owner, action, and due date. But it may be conditional, sarcastic, or quoted from someone else. The system should preserve the original wording, label confidence or review status, and avoid treating an inferred record as confirmed. Where there is ambiguity, ask a human or abstain.

Consolidation can connect repeated references to the same person or project and reconcile new statements with existing facts. That creates entity-resolution risk: “Alex” might refer to two people; a later mention may be hypothetical; a date may refer to a different release. Keep aliases and merges reversible, preserve conflicting evidence, and track the event time separately from the time the system learned the fact.

Evaluate each stage, not just the final prose

Create a test set from authorized, representative questions with known answers and source pointers. Score retrieval recall and relevance separately from answer correctness and groundedness. Add no-answer cases and time-sensitive questions. Inspect false joins, missed evidence, stale answers, and permission violations. The RAGAS paper is one example of research that separates retrieval and generation dimensions; its framework is not a universal certification and should be adapted to the task. Sources: RAGAS: Automated Evaluation of Retrieval Augmented Generation

Run regression checks when changing an extractor, prompt, model, index, or access rule. Keep evaluation data privacy-safe and authorized; do not copy customer conversations into a public benchmark. For a decision-support system, include human review and a clear correction path. Define what happens if a provider times out, an index is unavailable, a deletion fails, or the evidence conflicts. Reliability includes honest failure behavior, not only average answer quality.

A system-design checklist

For each stage, name the input, output, owner, and failure behavior. What sources are permitted? How is a partial transcript labeled? Which extracted claims need confirmation? How are identities linked and corrected? How are access rights checked during retrieval? What happens when a provider is unavailable? Who verifies deletion across indexes and derived summaries? These questions turn an architecture diagram into an operational design.

Separate evaluation by failure type. A retrieval miss differs from a correct passage paired with an incorrect answer. A permission leak is more serious than a low relevance score. A stale fact is different from an extraction error. Keep representative, authorized examples for each class and report sample size and method when publishing results. Do not use a single blended score to hide critical failures.

Treat provenance as a chain that must survive transformations. Store source identifiers and offsets or quotations where allowed, preserve event timestamps, and make citations open the relevant evidence. A citation that leads to a whole long transcript without locating support may be technically present but operationally weak. Design for a reader who needs to verify the answer quickly.

Choose between retrieval, graphs, and structured records

Use passage retrieval when the question depends on language-rich evidence: what reason did the reviewer give, or which caveat appeared in the source? Use structured records when the answer depends on explicit fields such as current owner, due date, or decision status. Use relationships when questions routinely cross people, projects, and events. A graph can represent these links, but a relational schema or carefully indexed references may be simpler to operate.

A hybrid system can combine keyword search for exact names and phrases, semantic search for paraphrases, filters for time and permission, and structured records for normalized state. The blend should be selected from evaluation results rather than architecture fashion. A larger number of retrieval components can make debugging harder if the system cannot show which route supplied a result.

Temporal data requires at least two useful times: when an event occurred and when the system learned or stored it. Some facts also need a validity period or supersession link. Without those distinctions, a later-imported older record may be mistaken for the latest truth, and a corrected fact may erase the history needed to explain a past decision.

Entity graphs introduce their own lifecycle. A person’s alias may be ambiguous; two projects may share a short name; a connection may be inferred from co-occurrence rather than explicitly stated. Preserve evidence for edges, use conservative merge rules, and make corrections reversible. A graph visualization is not a substitute for source-level verification.

Use abstention as a designed behavior

A system should be allowed to return “not enough evidence” when retrieval finds only related material, when the source is inaccessible, or when two records conflict. Set confidence and escalation rules around the cost of error. A low-stakes query can offer a tentative answer with evidence; a high-stakes commitment may require an explicit human check.

Test abstention with deliberately incomplete examples. If the answer model fills every gap with a plausible continuation, strengthen the retrieval boundary or require a citation that supports each claim. Good failure handling is visible: users should know whether the system found no record, could not search a source, or found contradictory records.

Frequently asked questions

Sources and further reading

Original research and product documentation are labeled separately.

• Primary research Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)
• ProfMe documentation ProfMe Privacy Policy
• ProfMe documentation ProfMe AI & Recording Notice
• ProfMe documentation ProfMe Security Overview
• Primary research RAGAS: Automated Evaluation of Retrieval Augmented Generation
• Official documentation NIST AI Risk Management Framework: Generative AI Profile
• ProfMe documentation ProfMe search and provenance

Article ID: how-memory-systems-work · Canonical publication URL: https://www.profme.in/blog/how-memory-systems-work · Author: Satish Rajnale, Senior AI Product Engineer