Spring AI 2.0 & Mastra — Reducing Tokens with AI Agent 'Tool Diet'

When you try to add useful features to an AI agent by giving it every tool under the sun, you often find it suddenly becoming 'dumber' or frustratingly slow. This happens because dozens of API specifications clog up the prompt, leaving the agent with no room to think. In this post, I’d like to discuss the concept of a 'tool diet,' or dynamic tool pruning, which is a key focus area for current AI frameworks and research.

Seeing but not using: The gap between 'looking' and 'picking'

Imagine going to a restaurant and being handed a menu that is dozens of pages long. Even if you read every single page carefully, you’re likely to suffer from decision paralysis when it’s time to order, leading you to pick something random or just give up. The problem that occurs when you arm an AI agent with too many tools is very similar.

We’ve often assumed that if an agent struggles to use tools, it’s due to the 'lost in the middle' phenomenon, where it misses information buried in the middle of a long prompt. However, the research paper 'Looking Is Not Picking,' published in June 2026, presents a fascinating twist. Their experiments showed that Large Language Models actually focus on the correct tool specification about 80% of the time.

The real problem isn't that they can't read the information; it’s that they struggle to decide between too many candidates during the final decision-making step. This is why a 'tool diet'—removing unnecessary tools from the menu rather than just shuffling the prompt order—is far more effective.

Searching only when needed: Spring AI and RAG-MCP

Instead of blindly cramming dozens of API specs into a prompt from the start, the recent trend is moving rapidly toward 'input-stage filtering.' The idea is not to arm the agent with its entire arsenal upfront, but to let it search for and retrieve tools from its armory only when necessary.

The most polished implementation of this approach is the 'ToolSearchToolCallingAdvisor' in Spring AI 2.0. Instead of handing the agent dozens of tools at once, you give it a single 'meta-tool' that can search for others. If the agent needs a specific feature while working, it calls this search tool, and the system dynamically retrieves the most appropriate tool specification from an internal vector index and appends it to the prompt.

The results are impressive. According to official Spring AI benchmarks, applying this method can reduce token usage by 34% to 64%. Since the agent isn't distracted by irrelevant tool specs, both its reasoning accuracy and processing speed improve.

The evolution of these frameworks aligns with academic research trends. When the 'RAG-MCP' team searched for and injected only the necessary tools in a Model Context Protocol (MCP) environment, prompt tokens were reduced by over 50%. Remarkably, the probability of selecting the correct tool jumped more than threefold, from approximately 13% to 43%.

Execution log diet: Context cleanup in Mastra and LangGraph

While the Spring AI approach is 'input-stage filtering'—reducing the menu shown to the agent—'execution-stage filtering,' which prevents token waste after a tool is run, is equally important.

When an agent queries a database or calls an external API, hundreds or thousands of lines of raw data can pile up in the conversation history. If these heavy, messy logs remain in the context, the agent quickly becomes slower and less coherent.

The open-source framework Mastra solves this with a feature called 'ToolCallFilter.' It’s a technique that cleans up unnecessarily long and complex raw arguments or garbage data from the conversation history immediately after a tool execution, before continuing the dialogue.

As shown in the example below, you can keep the agent's context clean with just a simple configuration.

import { Agent } from '@mastra/core/agent';
import { ToolCallFilter } from '@mastra/core/processors';

const agent = new Agent({
  id: 'lightweight-agent',
  name: 'Slim Agent',
  model: 'openai/gpt-4o',
  inputProcessors: [
    new ToolCallFilter({
      filterAfterToolSteps: 2, // 최근 2단계의 도구 실행 내역만 유지합니다.
      preserveModelOutput: true, // 원본 로우 데이터 대신 핵심 요약 정보만 남깁니다.
    })
  ]
});

By doing this, the agent remembers what it just did while keeping its 'mind' light, allowing it to decide on its next move much faster and more accurately.

Meanwhile, LangGraph suggests a way to dynamically connect only the necessary tools for each step (node) in a workflow. It’s a structure that provides a minimal set of tools at specific moments, ensuring the agent focuses only on what needs to be handled right now. These practical patterns—lightening inputs and cleaning execution logs—are becoming essential formulas for stable, large-scale AI services.

Smart agents travel light

Building a powerful AI agent is no longer just about connecting hundreds of tools. It’s about how efficiently you keep its 'headroom'—the context—light and clear, enabling the agent to handle complex reasoning. This is what ultimately determines performance.

If you are planning to design a large-scale agent system, make sure to add 'diet patterns'—dynamic tool selection at runtime and conversation history cleanup—to your blueprint. After all, an agent with a lighter arsenal acts sharper and smarter.