Observability trong Spring Boot 2026: OpenTelemetry, Distributed Tracing va Cau hoi Phong van

Observability trong Spring Boot ket hop logging, metrics va distributed tracing thanh mot he thong thong nhat cho thay cach request di chuyen qua cac microservice. Tu Spring Boot 3, framework nay ap dung Micrometer Tracing (thay the Spring Cloud Sleuth) va gioi thieu Observation API cung cap mot diem instrumentation duy nhat de tao ra ca metrics va trace.

Observation API Ket noi Micrometer va OpenTelemetry nhu the nao

Observation API dong vai tro la facade cho ca metrics va tracing. Khi code goi Observation.createNotStarted(), Spring Boot dinh tuyen observation den cac handler da dang ky: MeterObservationHandler cho Micrometer metrics va TracingObservationHandler cho distributed trace. Kien truc nay co nghia la instrument mot lan co the export ra nhieu noi.

Dependency cau noi micrometer-tracing-bridge-otel ket noi Micrometer Tracing voi SDK cua OpenTelemetry. Trace di qua SdkTracerProvider cua OpenTelemetry va export qua OTLP den cac backend nhu Jaeger, Tempo hoac bat ky collector nao tuong thich voi OpenTelemetry.

@Configuration
public class ObservabilityConfig {

    @Bean
    public ObservationRegistryCustomizer<ObservationRegistry> addLowCardinalityTags() {
        return registry -> registry.observationConfig()
            .observationHandler(new ObservationTextPublisher()); // Logs observations to console
    }
}

Cau hinh tren dang ky mot ObservationHandler ghi lai moi observation vao log. Trong moi truong production, cac handler duoc cau hinh tu dong se day du lieu den Micrometer registry va OpenTelemetry exporter ma khong can code bo sung.

Cac Dependency Can thiet cho Tracing trong Spring Boot 3.4

Spring Boot 3.4 yeu cau cac dependency ro rang cho distributed tracing. spring-boot-starter-actuator cung cap Observation API, nhung tracing can cau noi OpenTelemetry va exporter.

<!-- pom.xml -->
<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
        <groupId>io.micrometer</groupId>
        <artifactId>micrometer-tracing-bridge-otel</artifactId>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-exporter-otlp</artifactId>
    </dependency>
</dependencies>

Artifact micrometer-tracing-bridge-otel ket noi Micrometer Tracing voi API cua OpenTelemetry. opentelemetry-exporter-otlp gui span den collector su dung giao thuc OTLP qua HTTP hoac gRPC. Spring Boot quan ly viec dong bo phien ban thong qua dependency management.

Cau hinh Export OTLP den Jaeger hoac Tempo

Spring Boot tu dong cau hinh OtlpHttpSpanExporter khi dependency OTLP co san. Exporter gui trace den endpoint duoc chi dinh trong application.yml.

# application.yml
management:
  tracing:
    sampling:
      probability: 1.0  # 100% sampling cho dev, giam trong production
  otlp:
    tracing:
      endpoint: http://localhost:4318/v1/traces
  opentelemetry:
    resource-attributes:
      service.name: order-service
      deployment.environment: staging

resource-attributes dinh kem metadata vao moi span. Cac backend nhu Grafana Tempo va Jaeger su dung cac thuoc tinh nay de nhom trace theo service va environment. Dat sampling.probability la 1.0 se bat tat ca request trong qua trinh phat trien, nhung he thong production thuong sample tu 1% den 10% de kiem soat chi phi luu tru.

Tao Custom Observation voi Tag Low va High Cardinality

Observation ho tro hai loai tag: low cardinality cho metrics (gia tri gioi han nhu HTTP method hoac status code) va high cardinality cho trace (gia tri khong gioi han nhu user ID hoac request ID).

@Service
public class PaymentService {

    private final ObservationRegistry observationRegistry;

    public PaymentService(ObservationRegistry observationRegistry) {
        this.observationRegistry = observationRegistry;
    }

    public PaymentResult processPayment(PaymentRequest request) {
        return Observation.createNotStarted("payment.process", observationRegistry)
            .lowCardinalityKeyValue("payment.method", request.getMethod().name())  // enum: CARD, BANK_TRANSFER
            .lowCardinalityKeyValue("currency", request.getCurrency())              // bounded: USD, EUR, GBP
            .highCardinalityKeyValue("payment.id", request.getPaymentId())          // unique per request
            .highCardinalityKeyValue("customer.id", request.getCustomerId())        // high cardinality
            .observe(() -> executePayment(request));
    }

    private PaymentResult executePayment(PaymentRequest request) {
        // Payment gateway call
        return new PaymentResult(true, "TXN-" + UUID.randomUUID());
    }
}

Tag low cardinality xuat hien trong ca metrics va trace. Tag high cardinality chi xuat hien trong trace vi metrics voi chieu khong gioi han se lam bung no bo nho luu tru. Su phan biet nay ngan chan cardinality bomb tren Prometheus trong khi giu cho chi tiet trace phong phu.

Truyen Trace Context Qua cac Async Boundary

Distributed tracing yeu cau truyen context. HTTP header mang trace ID giua cac service, nhung cac thao tac async nhu method @Async hoac chuoi CompletableFuture co the mat context neu khong duoc cau hinh.

Spring Boot 3.4 cung cap truyen tu dong cho reactive stream va method @Async khi duoc cau hinh:

# application.yml
spring:
  reactor:
    context-propagation: auto
  task:
    execution:
      propagate-context: true

Voi cac bean TaskExecutor tuy chinh, boc chung bang ContextPropagatingTaskDecorator:

@Configuration
@EnableAsync
public class AsyncConfig {

    @Bean
    public TaskExecutor taskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(10);
        executor.setMaxPoolSize(50);
        executor.setTaskDecorator(new ContextPropagatingTaskDecorator());
        executor.initialize();
        return executor;
    }
}

Decorator sao chep Observation hien tai va trace context vao thread async. Khong co no, span bat dau trong task async se khong co parent, pha vo cay trace.

Su dung Annotation @Observed va @NewSpan

Spring Boot 3.4 ho tro observability khai bao thong qua annotation. Kich hoat xu ly annotation bang cach them AspectJ weaver:

<!-- pom.xml -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

# application.yml
management:
  observations:
    annotations:
      enabled: true

@Service
public class InventoryService {

    @Observed(name = "inventory.check", contextualName = "checkStock")
    public StockLevel checkStock(String productId) {
        // Database query
        return new StockLevel(productId, 42);
    }

    @NewSpan("inventory.reserve")  // Creates a child span under the current trace
    public ReservationResult reserveStock(String productId, int quantity) {
        // Update inventory
        return new ReservationResult(true, "RES-" + System.currentTimeMillis());
    }
}

@Observed tao ca metric (timer) va span. @NewSpan chi tao span, huu ich khi thao tac da co metrics o noi khac. Ca hai annotation tu dong xu ly ghi lai exception va trang thai span.

Cac Thanh phan duoc Instrument Tu dong trong Spring Boot 3.4

Spring Boot 3.4 tu dong instrument nhieu thanh phan ma khong can thay doi code:

Tai lieu Spring Boot Actuator liet ke tat ca cac thanh phan duoc instrument tu dong. Thu vien ben thu ba nhu Datasource Micrometer them tracing query JDBC.

Loc Observation de Giam Noise

Health check, readiness probe va static resource tao ra noise trong trace. Loc chung bang predicate:

@Configuration
public class ObservationFilterConfig {

    @Bean
    public ObservationPredicate noHealthChecks() {
        return (name, context) -> !name.equals("http.server.requests")
            || !isHealthEndpoint(context);
    }

    private boolean isHealthEndpoint(Observation.Context context) {
        if (context instanceof ServerRequestObservationContext http) {
            String path = http.getCarrier().getRequestURI();
            return path.startsWith("/actuator/health") || path.startsWith("/actuator/ready");
        }
        return false;
    }
}

Ngoai ra, vo hieu hoa observation theo ten trong cau hinh:

# application.yml
management:
  observations:
    enable:
      spring.security: false  # Disable Spring Security observations
      http.server.requests.actuator: false  # Custom predicate name

Lien ket Log voi Trace ID

Spring Boot 3.4 tu dong them trace va span ID vao MDC (Mapped Diagnostic Context) khi Micrometer Tracing hoat dong. Logback va Log4j2 co the bao gom cac ID nay trong output log:

<!-- logback-spring.xml -->
<configuration>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - traceId=%X{traceId} spanId=%X{spanId} - %msg%n</pattern>
        </encoder>
    </appender>
    <root level="INFO">
        <appender-ref ref="CONSOLE" />
    </root>
</configuration>

Voi pattern nay, moi dong log bao gom trace ID. Tim kiem log theo trace ID tra ve tat ca cac message tu mot request tren tat ca service, lien ket log voi span trong Jaeger hoac Tempo.

Cac Cau hoi Phong van Thuong gap ve Observability Spring Boot

Nguoi phong van danh gia ca hieu biet khai niem va kinh nghiem thuc te voi observability. Cac cau hoi nay thuong xuat hien trong cac vi tri backend cao cap.

H: Su khac biet giua Micrometer Tracing va OpenTelemetry la gi?

Micrometer Tracing la API trung lap vendor cho distributed tracing, tuong tu cach Micrometer truu tuong hoa metrics. OpenTelemetry la mot trien khai va giao thuc cu the. Spring Boot su dung Micrometer Tracing lam API va ket noi voi OpenTelemetry de export qua micrometer-tracing-bridge-otel. Cach phan lop nay cho phep ung dung chuyen doi backend ma khong can thay doi code.

H: Tai sao Spring khuyen nghi Observation API thay vi instrumentation OpenTelemetry truc tiep?

Observation API cung cap mot diem instrumentation duy nhat tao ra ca metrics va trace. Goi OpenTelemetry truc tiep chi tao trace. Su dung Observation.observe() tao timer metric va span tu cung mot code, giam trung lap va dam bao dat ten nhat quan.

H: Lam the nao de debug trace co gap giua cac span?

Gap chi ra thieu instrumentation hoac truyen context bi hong. Kiem tra xem code co su dung thao tac async ma khong co ContextPropagatingTaskDecorator khong. Xac minh rang HTTP client duoc instrument (WebClient, RestClient hoac RestTemplate duoc boc thu cong). Voi message queue, xac nhan rang trace header duoc truyen qua cac thuoc tinh message.

H: Dieu gi xay ra neu them tag high cardinality vao metric?

Moi gia tri tag duy nhat tao mot time series moi. Tag user ID voi hang trieu user tao hang trieu series, lam can kiet bo nho trong Prometheus hoac cac backend TSDB khac. Giai phap la su dung highCardinalityKeyValue() thay vi lowCardinalityKeyValue(), chi them tag vao trace noi high cardinality duoc mong doi.

Xay dung Pipeline Distributed Tracing voi Docker Compose

Stack observability local giup xac thuc instrumentation truoc khi deploy len production. Vi du nay su dung OpenTelemetry Collector va Jaeger:

# docker-compose.yml
services:
  otel-collector:
    image: otel/opentelemetry-collector-contrib:0.102.0
    command: ["--config", "/etc/otel-collector-config.yml"]
    volumes:
      - ./otel-collector-config.yml:/etc/otel-collector-config.yml
    ports:
      - "4318:4318"   # OTLP HTTP
      - "4317:4317"   # OTLP gRPC

  jaeger:
    image: jaegertracing/jaeger:2.3
    ports:
      - "16686:16686"  # UI
    environment:
      - COLLECTOR_OTLP_ENABLED=true

  app:
    build: .
    environment:
      - MANAGEMENT_OTLP_TRACING_ENDPOINT=http://otel-collector:4318/v1/traces
    depends_on:
      - otel-collector

# otel-collector-config.yml
receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  batch:
    timeout: 1s
    send_batch_size: 1024

exporters:
  otlp/jaeger:
    endpoint: jaeger:4317
    tls:
      insecure: true

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/jaeger]

Collector nhan span tu ung dung Spring Boot va chuyen tiep den Jaeger. Kien truc nay cho phep them exporter (Tempo, Zipkin, cloud backend) ma khong can thay doi code ung dung.

Nhung Diem Chinh ve Observability Spring Boot

• Observation API thong nhat metrics va tracing: mot lan goi Observation.observe() tao ra timer metric va span, loai bo instrumentation kep.
• Them micrometer-tracing-bridge-otel va opentelemetry-exporter-otlp de kich hoat export OTLP. Spring Boot 3.4 tu dong cau hinh HTTP exporter voi endpoint tu management.otlp.tracing.endpoint.
• Su dung lowCardinalityKeyValue() cho cac chieu xuat hien trong metrics (gia tri gioi han). Su dung highCardinalityKeyValue() cho du lieu chi co trong trace (user ID, request ID).
• Kich hoat truyen context cho code async voi spring.task.execution.propagate-context=true va boc executor tuy chinh bang ContextPropagatingTaskDecorator.
• Loc cac endpoint gay nhieu nhu /actuator/health bang bean ObservationPredicate de giu trace tap trung vao cac thao tac nghiep vu.
• Pattern log nen bao gom %X{traceId} de lien ket dong log voi distributed trace trong backend observability.
• Cac cau hoi phong van ve observability tap trung vao cardinality, loi truyen context va su khac biet giua Observation API va su dung OpenTelemetry truc tiep.