Spring AI 2.0 & Mastra — Reducing Tokens with AI Agent 'Tool Diet'

When you load an AI agent with too many tools to make it more useful, you might find it suddenly getting dumber or frustratingly slow. This happens because dozens of API specifications crowd the prompt, leaving no room for the agent to actually think. I’d like to talk about 'Tool Diet'—or dynamic tool pruning—a technique that AI frameworks and researchers are currently focusing on to solve this problem.

Why Can't They Use It When They Can See It?: The Difference Between 'Looking' and 'Picking'

Imagine going to a restaurant with a menu that is dozens of pages long. Even if you read it from cover to cover, you're likely to get overwhelmed, end up choosing the wrong dish, or give up on making a decision altogether. The problem caused by giving an AI agent too many tools is very similar.

For a long time, we assumed that when agents failed to use tools effectively, it was due to the 'lost in the middle' phenomenon—where they missed information buried in the middle of a long prompt. However, the research paper 'Looking Is Not Picking,' published in June 2026, reveals a fascinating reversal: the results showed that Large Language Models were actually focusing on the correct tool specification with a high accuracy of about 80%.

The real problem isn't the inability to read the information, but the struggle to decide what to pick from too many candidates when making the final decision. That's why 'Tool Diet'—clearing unnecessary tools from the menu that won't be used anyway—is far more effective than temporary fixes like reordering prompts at the input stage.

Search and Retrieve Only When Needed: Spring AI and RAG-MCP

Instead of mindlessly jamming dozens of API specs into a prompt from the start, the recent trend is shifting rapidly toward 'input-stage filtering.' The idea is not to arm the agent with all its weapons upfront, but to have it search the arsenal and retrieve what it needs only when necessary.

The most polished implementation of this approach is the 'ToolSearchToolCallingAdvisor' in Spring AI 2.0. Instead of handing the agent dozens of tools at once, it gives the agent a single meta-tool that can search for other tools. When the agent needs a specific function while working, it calls this search tool, and the system dynamically retrieves the most appropriate tool specification from an internal vector index and adds it to the prompt.

The effects are undeniable. According to Spring AI's official benchmarks, this approach reduced token usage by anywhere from 34% to 64%. Because the agent isn't distracted by irrelevant tool specs, both inference accuracy and processing speed improve simultaneously.

The evolution of these frameworks aligns with academic research trends. When the RAG-MCP team tested searching for and injecting only the necessary tools in a Model Context Protocol (MCP) environment, prompt token usage was reduced by over 50%. Remarkably, the accuracy of selecting the right tool jumped more than threefold, from approximately 13% to 43%.

Execution Log Diet: Context Cleaning in Mastra and LangGraph

While Spring AI’s method focuses on 'input-stage filtering' to reduce the menu provided to the agent, 'execution-stage filtering'—which prevents token waste after tools are used—is just as critical.

Whenever an agent queries a database or calls an external API, hundreds or thousands of lines of raw data pile up in the conversation history. If these heavy, messy logs remain in the context, the agent quickly becomes sluggish and less intelligent.

The open-source framework Mastra elegantly solves this with a feature called 'ToolCallFilter.' It’s a technique that immediately cleans up unnecessarily long and complex raw arguments or garbage data from the conversation history once a tool execution is finished.

As shown in the example below, you can keep the agent’s context clean at all times with very simple configurations.

import { Agent } from '@mastra/core/agent';
import { ToolCallFilter } from '@mastra/core/processors';

const agent = new Agent({
  id: 'lightweight-agent',
  name: 'Slim Agent',
  model: 'openai/gpt-4o',
  inputProcessors: [
    new ToolCallFilter({
      filterAfterToolSteps: 2, // 최근 2단계의 도구 실행 내역만 유지합니다.
      preserveModelOutput: true, // 원본 로우 데이터 대신 핵심 요약 정보만 남깁니다.
    })
  ]
});

This allows the agent to remember what it just did while keeping its 'mind' light, enabling it to decide on its next move much faster and more accurately.

Meanwhile, LangGraph proposes a way to dynamically connect only the tools needed for each step (node) in a workflow. It’s a structure that provides a minimal set of tools at specific moments, allowing the agent to focus only on what needs to be handled at that instant in the overall conversation flow. These practical patterns of lightening inputs and clearing execution records are becoming essential formulas for operating large-scale AI services reliably.

A Smart Agent Carries Light Weapons

Building a powerful AI agent is no longer just about connecting hundreds of tools. How lightly and efficiently you keep the 'context' (the agent's mental space) determines its actual performance, allowing it to engage in complex reasoning.

If you are planning to design a large-scale agent system in the future, be sure to add diet patterns—like dynamic tool selection at runtime and conversation history cleanup—to your blueprint. After all, an agent carrying lighter weapons is an agent that moves sharper and smarter.