OpenTelemetry and OpenInference — Catching 'Silent Failures' in AI Agents

Retrieval-Augmented Generation (RAG) and multi-agent workflows using LLMs frequently suffer from 'silent failures'—where API calls succeed, but the system retrieves incorrect documents or triggers hallucinations. Standard Application Performance Monitoring (APM) tools cannot detect these business-layer malfunctions because they rely solely on system metrics like network status or HTTP response codes. We explore why a dedicated observability design combining OpenTelemetry and OpenInference is necessary to transparently visualize AI agent execution paths and tool integrations while assessing their reliability.

Limitations of the OpenTelemetry GenAI Spec and the Value of OpenInference

Traditional APM tools struggle to accurately detect when an AI service is failing. In agentic systems, even if an API server returns a successful response, 'silent failures' often occur internally, such as fetching incorrect documents or outputting nonsensical results due to hallucinations. To catch these malfunctions quickly, it is essential to precisely record intermediate data for each execution step, including prompt templates, embeddings, vector database queries, and reranking results.

However, the official OpenTelemetry GenAI semantic conventions are still under active development. Despite major revisions, such as the specifications being moved to a separate repository in June and July 2026, there are significant structural limitations in representing the deeply nested span schemas generated in production environments. As a result, there is a clear functional gap in expressing the intricate behaviors of complex agent workflows or RAG systems using standard specifications alone.

OpenInference, an open-source project led by Arize AI, has established itself as the optimal extension specification to overcome these limitations. It provides robust support for semantic schemas optimized for core agent components—including LLM calls, retrieval, and tool usage—to capture complex contexts structurally. A hybrid pipeline, which collects this OpenInference data and transmits it to existing enterprise APM platforms via a lightweight transformation layer, is gaining attention as the most practical and stable monitoring architecture in the field.

Distributed Tracing in MCP v2: The SEP-414 Standard and _meta Propagation

With the Model Context Protocol (MCP) fully transitioning to a stateless HTTP architecture, it has become critical to maintain seamless communication traces between backend servers and externally isolated tool servers. Traditional distributed tracing primarily relied on HTTP headers to convey trace information. However, this header-based propagation fails in standard I/O or local pipe environments commonly used in MCP communication, resulting in broken traces.

To address this, the SEP-414 specification defines an transport-agnostic context propagation method. Under the SEP-414 standard, W3C Trace Context information—such as _meta—is directly inserted into the traceparent, tracestate, and baggage objects within the JSON-RPC request payload. This allows distributed tracing contexts to be maintained reliably across both stdio pipes and HTTP connections using a consistent format.

When used in conjunction with the openinference-instrumentation-mcp library, this metadata injection and parsing process can be automated. The entire flow—from the moment an agent client calls a tool to the MCP server executing and responding—is cleanly visualized as a single continuous span on a waterfall chart.

Declarative Agent Monitoring: Adoption Case with the Latest PydanticAI Package

Manually embedding span-recording code into business logic to track internal agent behavior or complex tool calls hampers developer productivity. The openinference-instrumentation-pydantic-ai v0.1.18 version, released on July 30, 2026, allows you to collect the entire execution path of PydanticAI-based agents—which provide static type safety—using only a few lines of declarative configuration.

PydanticAI natively emits standard OpenTelemetry traces, and the dedicated processor provided by this latest library automatically transforms this metadata into a schema specialized for agents. By using a background asynchronous exporter to prevent performance latency in the main business loop, you can monitor everything—from prompt template variables and input parameters to the agent’s final output structure—restored in its raw type format.

import os
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from openinference.instrumentation.pydantic_ai import OpenInferenceSpanProcessor
from pydantic import BaseModel
from pydantic_ai import Agent

# 1. 오픈텔레메트리 트레이서 프로바이더 초기화
provider = TracerProvider()
trace.set_tracer_provider(provider)

# 2. PydanticAI 스팬을 OpenInference 포맷으로 가공하는 프로세서 등록
provider.add_span_processor(OpenInferenceSpanProcessor())

# 3. 비동기 익스포터를 통한 오픈소스 수집기 전송 설정
exporter = OTLPSpanExporter(endpoint="http://localhost:6006/v1/traces")
provider.add_span_processor(BatchSpanProcessor(exporter))

# 4. 타입 검증과 트레이싱이 자동 적용되는 에이전트 구성
class UserProfile(BaseModel):
    preferred_theme: str
    marketing_opt_in: bool

agent = Agent("openai:gpt-4o", result_type=UserProfile)
result = agent.run_sync("사용자 설정에서 어두운 테마 선호 여부와 마케팅 수신 거부를 추출해줘.")

A key advantage of this approach is that no tracing-related boilerplate code permeates your tools or prompt logic. Even if exceptions occur during data validation or unexpected data structures are encountered, raw error information is automatically bound to the corresponding trace span, enabling clear debugging of complex, multi-step agent actions.

Checklist for Stable Production Monitoring

As AI agents become more sophisticated, real-time observability evolves from simple monitoring into essential infrastructure that guarantees service reliability. To ensure seamless integration with enterprise APM platforms, you should establish rules using the OpenTelemetry Collector's Transform Processor to normalize OpenInference attributes into standard specifications starting at the collection stage. Furthermore, in large-scale production environments, we recommend applying smart sampling—focusing on key agent branching points or high-cost tool calls—rather than storing all data, to maintain an optimal balance between infrastructure costs and debugging efficiency.

Reference Links