Mastra Observational Memory — Agents That Use Prompt Caching Instead of Vector RAG

Ever wonder why AI agents suddenly seem to get dumber or why costs skyrocket during long conversations? The culprit is the snowballing volume of conversation history and complex tool-use logs.

Until now, the primary way to solve this has been to retrieve necessary information from a vector database for every interaction. However, 'Observational Memory,' unveiled by the open-source agent framework Mastra, cleverly compresses conversation context and maximizes prompt caching efficiency without needing complex database queries.

The 4-Stage Evolution of Agent Memory

The way AI agents handle memory has evolved through four main stages. Initially, it was a simple chat history approach where the entire previous conversation was passed along. However, this hit token limits and caused costs to snowball as soon as a conversation got a bit long.

To address this, working memory was introduced, allowing critical information or user preferences to be jotted down like a notepad. Yet, this still wasn't enough to capture complex conversation flows. Eventually, developers began adopting semantic search, or vector RAG, to insert relevant conversation content retrieved in real-time from a database.

However, vector RAG had a fatal flaw. Since the data retrieved varies with every query, the system prompt changes constantly, which completely breaks prompt caching—the most powerful cost-saving weapon for large language models. It also slowed down response times due to constant database lookups.

The final 4th stage, introduced to solve this, is Observational Memory. Instead of searching a database every time, background agents organize real-time conversations and tool logs into condensed summary logs. This keeps the prompt structure consistent, maximizing caching efficiency while significantly reducing token costs.

The Compression Loop: Three Agents Collaborating

The operating principle of Observational Memory is intuitive. Three agents—the Actor, Observer, and Reflector—work in the background with precise teamwork. First, the Actor receives the user's command to execute tools and perform tasks.

In the background, the Observer meticulously monitors the conversation and complex tool-use records in real-time. As soon as the accumulated conversation history exceeds a 30,000-token threshold, the Observer steps in to condense the verbose raw logs into highly dense observation logs.

The final stage is the responsibility of the Reflector. If the summarized logs exceed 40,000 tokens, the Reflector acts to prune old or less important information, keeping only the essential context for the final cleanup. Thanks to this clever background collaboration, raw tool-use logs are remarkably compressed—by up to a factor of 40.

As a result, the agent maintains only the necessary context in an optimized state. With unnecessary text eliminated, prompt caching—a core feature of large language models—is maintained steadily without interruption, leading to faster speeds and dramatically lower token costs.

The Secret Behind 94.87% Performance on GPT-5 mini

This clever compression method has been proven effective in actual benchmarks. In the LongMemEval benchmark, which evaluates long-term memory performance, Observational Memory achieved overwhelming accuracy scores of 94.87% on GPT-5 mini and 84.23% on GPT-4o, thanks to keeping the agent's context window lightweight and optimized.

A key feature that adds to development convenience is the Memory Extractor. It goes beyond simply storing conversation summaries as text, allowing developers to extract core information as precisely structured data based on defined Zod schemas.

Mastra's Memory Extractor can be declared and implemented using simple TypeScript code like this:

// @mastra/memory v1.22.0+ 기준
import { Extractor } from "@mastra/memory";
import { z } from "zod";

const supportProfileExtractor = new Extractor({
  name: "Support profile",
  instructions: "사용자 대화에서 운영체제, Node.js 버전, 발생한 에러를 추출합니다.",
  schema: z.object({
    os: z.string().optional(),
    nodeVersion: z.string().optional(),
    issue: z.string().optional(),
  }),
  onExtracted: async ({ current }) => {
    console.log("추출 완료:", current);
    // CRM 전송이나 Webhook 연동 등의 작업 수행
  },
});

This extraction process is naturally integrated into the background memory cleanup step. Since it runs while the conversation context is already active in prompt caching, there is no need to make new LLM calls to structure the data, making the additional token cost effectively zero.

From the Era of Retrieval to the Era of Caching and Context

The era of reflexively retrieving information from external databases to keep agents stable for long periods is coming to an end. Now, clever designs that keep LLM context windows neat and maximize prompt caching are becoming paramount.

The Observational Memory proposed by Mastra provides a practical solution for developing smart agents that significantly save on token costs without adding complex vector database infrastructure. When designing future agents with long-running loops, I recommend considering this new approach of compressing context to improve caching efficiency rather than querying a database every time.