A log tells you what failed. Distributed tracing shows where it failed and why, request by request, service by service. When one request crosses six services and each logs its piece, correlating entries by timestamp becomes guesswork under load; trace IDs stitch the full journey together.

Which concepts hold OpenTelemetry up?

Traces contain spans: named operations with duration, attributes, and status. Parent-child relationships form the execution tree. W3C Trace Context propagates context through traceparent headers across HTTP, gRPC, and message queues; follow that spec from day one.

How do you set up the SDK?

Build a TracerProvider with an OTLP exporter pointed at the collector or vendor endpoint. BatchSpanProcessor delivers production-grade throughput, unlike SimpleSpanProcessor in tests. Resource attributes declare service name, version, and environment; telemetry without origin identification turns into noise.

Auto-instrumentation or manual spans?

Start with zero-code agents for Node, Python, and Java: they cover popular frameworks and HTTP clients without touching your code. Restrict manual spans to business-critical operations, with stable names and domain attributes (order id, customer plan). Names too generic hurt vendor search.

Which sampling strategy fits?

Head sampling between 10% and 100% depending on traffic decides collection at the source. Tail sampling keeps every error trace plus slow outliers while dropping routine successes; the collector owns that decision in tail mode. Without tail sampling, the rare trace of a hard bug slips away.

What does the collector sit in front of?

The agent sidecar plus gateway pattern balances local overhead against central processing. Processors handle batching, filtering, and PII scrubbing before export. Scrubbing at the collector keeps sensitive data away from third-party vendors.

Which vendor should you pick?

Jaeger works for a self-hosted start; Grafana Tempo bets on object storage to make long retention cheap; Datadog and Honeycomb sell managed analysis of high quality. OTLP keeps switching cheap: changing backends means swapping the exporter, not reinstrumenting the codebase.

Which pitfalls sink tracing projects?

  • Propagation lost across async boundaries and thread pools; the trace breaks midway
  • Span explosion from one span per database row; aggregate per query, not per row
  • Clock skew between hosts confusing the waterfall view