OpenTelemetry traces a request from frontend to database and shows where it broke. Without distributed instrumentation, debugging a flow that crosses five services requires manually reading logs in each system. The OpenTelemetry specification (a CNCF project) defines three pillars: traces, metrics, and logs.

The three pillars

Traces: the complete path of a request. Each service adds a span. The trace shows where time was spent and where the error happened.

Metrics: aggregated numbers. Requests per second, p99 latency, error rate. Metrics indicate something is wrong.

Logs: structured events. Logs indicate what happened. Traces indicate where. Metrics indicate how much.

Automatic instrumentation

OTel offers automatic instrumentation for popular frameworks. In Node.js, the @opentelemetry/auto-instrumentations-node package instruments Express, PostgreSQL, Redis, Kafka without changing code. OpenTelemetry's documentation lists auto-instrumentation packages for Java, Python, Go, .NET, and other languages.

Manual instrumentation

For manual code, you add spans explicitly. A span is an operation with a name, timestamp, and attributes. Ex: span "process-payment" with attributes "payment-method", "amount", "currency".

What to instrument

Service boundaries: every HTTP, gRPC, or queue call enters and exits a span. Without this tracing, you don't know which service caused the problem.

Database operations: slow queries are the number one cause of performance problems. Instrument queries with name, table, and duration.

Business operations: checkout, payment, onboarding. These operations have direct business impact. Instrument them with business attributes.

Common mistakes

Excessive instrumentation: trivial operations, cache hits, and health checks don't need spans. Instrument what matters for debugging.

Context not propagated: if the trace ID doesn't propagate between services, you have broken traces. Ensure the traceparent header is propagated in all calls.

Cost ignored: tracing has a cost. Sampling reduces cost. Sample 100% of errors, 10% of successes. You don't need every trace for debugging.

Limitations

OTel doesn't solve lack of logs, lack of metrics, or lack of manual instrumentation in legacy code. The tool standardizes collection and export, but instrumentation quality depends on the team.