Fastify v6 and @fastify/otel — Building Observability for GenAI Agents

Fastify is a proven choice for building high-performance Node.js backends, but once you introduce complex generative AI agents, traditional HTTP-level monitoring hits its limits. To accurately grasp an agent's multi-step reasoning processes and tool execution flows, you need precise observability that can track deep into the application. We will explore how to build a robust agent tracing system without performance overhead using Fastify v6's isolated scope architecture and the official OpenTelemetry plugin.

The End of Legacy: From @opentelemetry/instrumentation-fastify to @fastify/otel

The community package @opentelemetry/instrumentation-fastify, previously responsible for Fastify monitoring in Node.js environments, has officially been deprecated. You must fully transition to @fastify/otel, the official plugin maintained directly by the Fastify ecosystem as a first-class citizen. It is time to move away from makeshift setups using third-party tools and reshape your architecture based on observability standards integrated organically with the framework core.

@fastify/otel works in close integration with Fastify's internal lifecycle hooks, reliably maintaining request-level context. Instead of complex manual orchestration of asynchronous context propagation, developers can easily control the flow by simply calling the request.openTelemetry() method provided within the router handler.

This method returns an object containing the root span mapped to the current request, the tracer for that request, and the active context. This allows you to seamlessly and precisely connect everything from the initial HTTP entry point down to tool calls and reasoning operations in the downstream asynchronous agent layers as a single distributed trace flow.

Fastify v6 Scope Plugins and Non-Invasive Tracer Propagation

Fastify v6 has moved away from the TypeScript global module augmentation structure to solve the global type pollution issues common in monorepo environments. Previously, decorators registered by plugins were forcibly injected into the global namespace, causing unexpected type collisions. Starting with v6, the framework introduces a registration scope approach, limiting decorator types to the scope where the plugin is registered.

You can implement this isolated type control flexibly using the fastify-plugin helper from the createPlugin package. By defining decorators and types that are valid only within the plugin, you can securely share the OTel Tracer instance registered at the root with downstream LLM agent layers with minimal coupling.

The Core Standard for Latest OpenTelemetry GenAI Semantic Conventions

OpenTelemetry's generative AI observability specifications are now fully separated into an independent repository, open-telemetry/semantic-conventions-genai, and are undergoing standardization. Key changes in this revision include refining ambiguous legacy specifications and strengthening criteria to prevent the leakage of sensitive PII during agent operation.

The most significant change is that the gen_ai.system attribute, previously used to distinguish LLM systems, has been replaced by gen_ai.provider.name to clearly identify providers. Additionally, raw string attributes—which often caused security incidents by storing original prompts and completions without sanitization—have been deprecated or disabled by default.

Instead, a unified attribute based on serialized JSON strings has been introduced to structure and securely track conversation contexts. It is recommended to specify system prompts in gen_ai.system_instructions, and inject user questions and model response metadata as JSON-serialized strings into gen_ai.input.messages and gen_ai.output.messages respectively.

// OpenTelemetry GenAI 스팬 속성 할당
span.setAttributes({
  'gen_ai.provider.name': 'openai',
  'gen_ai.system_instructions': '안전한 보안 가이드를 준수합니다.',
  'gen_ai.input.messages': JSON.stringify([{ role: 'user', content: 'Fastify v6' }]),
  'gen_ai.output.messages': JSON.stringify([{ role: 'assistant', content: '안전하고 빠릅니다.' }])
});

Adhering to these standardized formats prevents indiscriminate exposure of text data within multi-agent systems and maximizes compatibility with visualization tools.

Implementing Decorator Integration in Fastify v6

By utilizing Fastify v6’s isolated scope structure and the @fastify/otel plugin, you can implement a tracing chain that connects seamlessly from the HTTP request entry point to the agent's internal task steps. Notably, @fastify/otel wraps the entire router handler in an active span context, so the OpenTelemetry API automatically handles parent-child relationships without requiring manual context passing from the developer.

The following is an example of a non-invasive implementation that combines Fastify v6 and @opentelemetry/api to track an agent's LLM call steps.

import { type Span } from '@opentelemetry/api';
import type { FastifyPluginAsync } from 'fastify';

export const agentRoutes: FastifyPluginAsync = async (fastify) => {
  fastify.post('/v1/chat', async (request, reply) => {
    // 1. 요청 스코프의 OTel 컨텍스트 획득
    const otel = request.openTelemetry();
    
    if (!otel.enabled || !otel.instrumented) {
      return { response: '텔레메트리 비활성 상태' };
    }

    const { tracer } = otel;

    // 2. 상위 컨텍스트를 자동 상속받는 하위 활성 스팬 생성
    return tracer.startActiveSpan('llm.generate', async (span: Span) => {
      try {
        const userPrompt = (request.body as { prompt: string }).prompt;

        // 3. 최신 시맨틱 규격을 반영하여 제공자 정보와 메시지 설정 (JSON 직렬화)
        span.setAttributes({
          'gen_ai.provider.name': 'openai',
          'gen_ai.request.model': 'gpt-4o',
          'gen_ai.input.messages': JSON.stringify([
            { role: 'user', content: userPrompt }
          ])
        });

        // 실제 LLM 서비스 호출 과정 (예시)
        const mockResponse = `답변 결과: ${userPrompt}`;

        // 4. 출력 데이터 및 토큰 사용량 속성 기록
        span.setAttributes({
          'gen_ai.output.messages': JSON.stringify([
            { role: 'assistant', content: mockResponse }
          ]),
          'gen_ai.usage.input_tokens': 15,
          'gen_ai.usage.output_tokens': 30
        });

        return { response: mockResponse };
      } catch (error) {
        if (error instanceof Error) {
          span.recordException(error);
        }
        reply.code(500);
        return { error: '내부 에러 발생' };
      } finally {
        // 5. 작업 완료 후 반드시 스팬 명시적 종료
        span.end();
      }
    });
  });
};

The core of this implementation is maintaining the scope isolation benefit of Fastify v6. Instead of decorator type merging that pollutes the global namespace, you can manage types by encapsulating them at the route scope level. Furthermore, by collecting potentially sensitive prompt input/output data as serialized JSON string attributes rather than individual events, you structurally minimize the risk of external PII exposure.

Maintaining a Secure Observability Ecosystem Without Performance Loss

Fastify v6 actively utilizes the V8 engine's native serialization optimizations instead of traditional complex schema compilation. To fully enjoy the performance benefits of this ultra-fast framework, you must ensure that real-time tracing processing costs do not hinder the application. In production environments, it is essential to design an architecture that rationally controls trace sampling rates and offloads heavy generative AI payloads using asynchronous batch processing to export data safely in the background, preventing them from blocking the Node.js event loop.

Reference Links