OpenTelemetry and OpenInference — Catching 'Silent Failures' in AI Agents

Retrieval-Augmented Generation (RAG) and multi-agent workflows using LLMs frequently encounter 'silent failures'—where API calls succeed, but the system fails due to incorrect document retrieval or hallucinations. Traditional Application Performance Management (APM) tools rely solely on system metrics like network status or HTTP response codes, leaving them unable to detect these business-layer malfunctions. We explore why a dedicated observability design, combining OpenTelemetry and OpenInference, is essential for transparently visualizing AI agent execution paths and assessing their reliability.

Limits of the OpenTelemetry GenAI Spec and the Value of OpenInference

Conventional APM tools struggle to accurately detect if AI services are functioning correctly. In agentic systems, even when the API server returns a successful response, 'silent failures' frequently occur—resulting in incorrect document retrieval or hallucinations. To catch these malfunctions early, you must precisely record intermediate data at each execution step, such as prompt templates, embeddings, vector database queries, and reranking results.

However, the official OpenTelemetry GenAI semantic conventions are still in development. Despite major revisions, such as the spin-off of relevant specifications into separate repositories in June and July 2026, there are significant structural limitations in representing deeply nested span schemas in production environments. This creates a clear functional gap when trying to express the detailed behavior of complex agent workflows or RAG systems using only standardized specifications.

OpenInference, an open-source project led by Arize AI, has established itself as the optimal extension specification for overcoming these limitations. It provides reliable support for semantic schemas optimized for core agent components like LLM calls, retrieval, and tool usage, allowing for structural capture of complex contexts. A hybrid pipeline—where OpenInference data is collected, passed through a lightweight transformation layer, and sent to enterprise APM platforms—is emerging as the most practical and stable monitoring architecture in the industry.

Distributed Tracing in MCP v2: SEP-414 Standard and _meta Propagation

With the Model Context Protocol (MCP) shifting to a stateless HTTP architecture, seamless tracing between backend servers and independent external tool servers has become critical. While traditional distributed tracing has relied primarily on HTTP headers for context propagation, this approach fails in the standard I/O or local pipe environments commonly used in MCP communication, resulting in broken traces.

The SEP-414 specification was introduced to solve this by defining a transport-agnostic context propagation method. According to the SEP-414 standard, W3C Trace Context information—such as traceparent, tracestate, and baggage—is inserted directly into the _meta object within the JSON-RPC request payload. This ensures that the distributed tracing context is safely maintained regardless of whether the communication uses stdio pipes or HTTP connections.

By using the openinference-instrumentation-mcp library alongside this, the insertion and parsing of metadata can be automated. The entire flow, from the moment an agent client calls a tool to when the MCP server executes it and returns a response, is neatly visualized as a single continuous span in a waterfall chart.

Declarative Agent Monitoring: Using the Latest PydanticAI Package

Manually mixing span logging code into business logic to track internal agent behavior or complex tool calls hurts developer productivity. The openinference-instrumentation-pydantic-ai v0.1.18 version, released on July 30, 2026, allows you to collect the full execution path of PydanticAI-based agents—known for their static type safety—with just a few lines of declarative configuration.

PydanticAI natively emits standard OpenTelemetry traces, and the dedicated processor provided by this new library automatically formats that metadata into an agent-specific schema. By using a background asynchronous exporter, it prevents performance latency in the main business loop while allowing you to monitor prompt template variables, input parameters, and the agent's final output structure in their raw data formats.

import os
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from openinference.instrumentation.pydantic_ai import OpenInferenceSpanProcessor
from pydantic import BaseModel
from pydantic_ai import Agent

# 1. 오픈텔레메트리 트레이서 프로바이더 초기화
provider = TracerProvider()
trace.set_tracer_provider(provider)

# 2. PydanticAI 스팬을 OpenInference 포맷으로 가공하는 프로세서 등록
provider.add_span_processor(OpenInferenceSpanProcessor())

# 3. 비동기 익스포터를 통한 오픈소스 수집기 전송 설정
exporter = OTLPSpanExporter(endpoint="http://localhost:6006/v1/traces")
provider.add_span_processor(BatchSpanProcessor(exporter))

# 4. 타입 검증과 트레이싱이 자동 적용되는 에이전트 구성
class UserProfile(BaseModel):
    preferred_theme: str
    marketing_opt_in: bool

agent = Agent("openai:gpt-4o", result_type=UserProfile)
result = agent.run_sync("사용자 설정에서 어두운 테마 선호 여부와 마케팅 수신 거부를 추출해줘.")

The key advantage of this approach is that no tracing-related boilerplate code permeates your tools or prompt logic. Even in the event of exceptions during data validation or the arrival of data with unexpected structures, the original error information is automatically bound to the corresponding trace span, enabling clear debugging of complex, multi-step agent actions.

Checklist for Stable Production Monitoring

As AI agents become more sophisticated, real-time observability evolves from simple monitoring into essential infrastructure that ensures service reliability. To ensure seamless integration with enterprise APM platforms, you should establish rules using the OpenTelemetry Collector's Transform Processor to normalize OpenInference attributes to standard specifications from the collection stage. Furthermore, in large-scale production environments, we recommend applying smart sampling—focusing on branch points in agent logic or high-cost tool calls—rather than storing all data, to maintain an optimal balance between infrastructure costs and debugging efficiency.

Reference Links