Distributed tracing

Every Deno service emits OpenTelemetry traces and metrics when started with OTEL_DENO=true. The compose.observability.yml overlay enables this on the production and points each service at the Alloy collector:

docker compose -f compose.yml -f compose.prod.yml -f compose.observability.yml up -d

The overlay is opt-in. Local development without the overlay does not ship traces by default, which keeps bandwidth cost zero. Service names are auto-derived from the container hostname inside backend/src/lib/telemetry.ts, so each service appears in Tempo without having to plumb OTEL_SERVICE_NAME into 37 compose blocks individually.

Context propagation

Trace context propagates automatically across both transport types the platform uses:

• HTTP: Deno's auto-instrumentation injects and extracts traceparent on every fetch and Deno.serve call. An order flowing Gateway, OMS, Risk Engine, EMS shows up as one connected trace in Tempo.
• Kafka: the producer in messaging.ts injects traceparent into Kafka message headers; the consumer extracts it before invoking the handler. The chain extends across every bus hop too: orders.routed, orders.child, and orders.filled all appear in one trace.

Custom business spans

Custom spans are added with the withSpan() helper:

import { withSpan } from "@veta/telemetry";

await withSpan("score-signal", async (span) => {
  span.setAttribute("symbol", symbol);
  return scoreSignal(features);
});

When OTEL_DENO is unset, withSpan() is a no-op zero-cost call. It is safe to leave in production code paths.

Pipeline-stage latency

The Journal service exposes GET /metrics/latency returning per-stage p50 / p95 / p99 for the order pipeline (gateway, bus, oms, routedToChild, emsAck, filled). Recent baselines and how they were measured live on the Performance page.