o3-mini & Kimi K3 — The 'Empty Response' Agent Bug and How to Reduce Inference Costs

Many people developing AI agents lately have been baffled by a strange bug where they are clearly billed for server costs, yet the agent stops with nothing but an empty response. This happens because the latest reasoning models use up their entire token budget on deep thinking, leaving nothing left to generate the final answer for us. I'll break down why this 'stuck' bug happens and how you can cleverly control your agent loops while saving on costs.

The Culprit: 'Shared Token Budgets' and Hidden Reasoning Steps

In the past, you could safely manage costs by limiting the length of the final response using the max_tokens option. However, the latest reasoning models, including OpenAI's o-series, use an entirely new standard called max_completion_tokens.

Think of it like taking an exam with a shared limit of exactly 100 characters for both your scratchpad and the final answer sheet. The 'hidden reasoning tokens'—the thoughts the AI writes on its scratchpad to solve the problem—share the same budget 'bank account' as the 'final response tokens' we see.

If you set this budget too tightly to save money, something absurd happens: the moment the AI fills up the scratchpad while thinking hard, there is zero budget left for the answer sheet.

In the end, the AI stops after only thinking to itself, leaving us with an empty response. What’s even more frustrating is that we still have to pay for all the thinking the AI did. This shared budget rule is the primary culprit behind the bug that paralyzes agents, as tools or JSON data needed for the next step fail to output entirely.

The Danger of 'Thinking Tax' That Cannot Be Cached

The financial hit is significant, too. People often feel relieved by prompt caching, which drastically reduces costs by remembering past conversation context. However, internal reasoning tokens generated in real-time for each request cannot be cached at all.

The problem is that these hidden reasoning tokens are charged at the most expensive output token rate. In fact, for a simple information extraction task, the Kimi K3 model consumes a massive 66 tokens for hidden thought processes out of a total of 97 tokens by default.

If you run an agent loop without controls while it works through multiple steps, you could face an unexpectedly huge 'thinking tax' bill.

Solution 1: Adjust Thinking Depth with 'reasoning_effort'

The smartest way to prevent this bug is to control the depth of thinking according to the situation. After all, the AI doesn't need to put its full effort into deep thinking at every step of the agent's execution.

Fortunately, OpenAI's o-series models provide a parameter called reasoning_effort. Developers can use this option to freely adjust the amount of thinking the model does, choosing between 'low', 'medium', or 'high'.

The method is simple. During simple data collection or intermediate steps of an agent loop, lower this value to 'low' to force the conservation of thinking tokens. Then, implement a dynamic control pattern where you set it to 'high' only during the final stage or coding tasks that require sophisticated planning, concentrating the thinking budget there.

# OpenAI Python SDK 기준
response = client.chat.completions.create(
    model="o3-mini",
    reasoning_effort="low",  # 작업 단계에 따라 low, medium, high 조율
    max_completion_tokens=1024,
    messages=[{"role": "user", "content": "..."}]
)

By balancing the depth of thinking this way, you can perfectly avoid the misfortune of responses being cut off within a limited token budget while maximizing operational efficiency.

Solution 2: 'Turn Off Thinking' for Simple Tasks

For simple tasks that don't require deep thought, it's best to turn off the reasoning feature entirely. If you let the model think deeply even for simple jobs like extracting dates or names, unnecessary costs will continue to leak.

When running simple information extraction with Moonshot AI's Kimi K3, the default settings waste a whopping 66 out of 97 tokens on hidden thought processes. In cases like this, you must specify the reasoning_effort option as none to explicitly turn off the reasoning step and avoid wasting precious costs and time.

However, you should be aware of a bizarre bug when using the latest OpenAI gpt-5.4 API. If you set reasoning_effort: none while simultaneously setting a total token budget via max_completion_tokens, the bug causes it to ignore the 'turn off thinking' setting and dive into deep thought anyway. As a result, it uses up the entire token budget on thinking and outputs a blank final response.

Until this error is resolved, you need to implement a temporary workaround that removes the token limit setting entirely whenever you disable the reasoning feature in gpt-5.4.

It's Time to Control 'Depth of Thought' Yourself

When designing agents in the future, the rule of 'how deeply the AI should think at this stage' will be just as crucial as the skill of writing good prompts.

Let it handle simple tasks quickly without 'using its brain', and dynamically allocate the thinking budget only to complex problems. This single, simple control pattern will act as a sturdy shield, drastically reducing your agent costs and preventing 'stuck' bugs.