What Is OpenTelemetry, and Why It Matters for Agent Observability

Every observability vendor would love to own your traces. OpenTelemetry exists so that none of them have to. If you are instrumenting an agent in 2026 and the tracing code you write only works with one backend, you are signing up for a migration later that you could skip entirely now.

What OpenTelemetry actually is#

OpenTelemetry (OTel for short) is an open source observability framework and toolkit for generating, exporting, and collecting telemetry data: traces, metrics, and logs. It is a CNCF project, born in 2019 from the merger of two earlier standards, OpenTracing and OpenCensus, which had been solving the same problem in incompatible ways.

The part people miss: OTel is not an observability backend. It does not store your traces and it does not draw the charts. Storage and visualization are deliberately left to other tools, whether that is open source (Jaeger, Prometheus, Phoenix) or a commercial vendor. OTel standardizes how telemetry is produced and moved, then gets out of the way.

The project states two principles that explain most of its design:

1. You own the data you generate. There is no vendor lock-in.
2. You learn one set of APIs and conventions, and reuse them everywhere.

For agent builders, both principles land directly. Your traces contain your prompts, your tool outputs, and your users' data. Owning that stream, and being able to point it anywhere, is not a nice-to-have.

The data model, quickly#

OTel deals in three signals:

• Traces: the tree-of-spans record of one operation. A span is a single unit of work with a start time, end time, status, and attributes. I covered how this maps onto agent runs (root span = the run, children = model calls, tool calls, subagents) in tracing in agent observability.
• Metrics: numeric measurements over time. Request counts, error rates, token usage, queue depth.
• Logs: timestamped text records, which under OTel can carry the trace and span IDs of whatever was executing when they were written.

The correlation is the underrated part. When a log line carries a trace_id, "weird log message" becomes "weird log message from this exact step of this exact agent run", one click away from the full tree.

The pieces you actually touch#

The project ships a lot of components. Here is the translation table:

The Collector deserves a sentence more. It sits between your app and your backends and runs pipelines: receive OTLP in, then redact, filter, sample, batch, and fan out to one or more destinations. Want to trial a new tracing vendor next to your current one? Add an exporter to the Collector config. Your application code does not change at all.

The GenAI semantic conventions#

Attribute naming sounds boring until you realize it is the reason a UI can show token counts and costs at all. If one library records llm.tokens.input and another records input_token_count, no dashboard can aggregate them. Semantic conventions fix this by standardizing names.

OTel has a set of GenAI semantic conventions covering exactly the things agents produce: LLM call spans (gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and so on), agent spans, GenAI events and metrics, and, notably, conventions for MCP. The work is active enough that it recently moved out of the main semantic-conventions repository into a dedicated one (open-telemetry/semantic-conventions-genai), which tells you where the energy in the standard is going.

You will also meet OpenInference, the convention set used by Arize Phoenix. It is OTel-compatible (spans travel over OTLP as usual) but defines its own attribute vocabulary for LLM apps. Conventions are still converging in this space. My advice: record the standard gen_ai.* attributes where you can, pin your instrumentation library versions, and expect some renames over the next year.

Why build agent tracing on OTel#

Four concrete reasons, in the order I care about them:

You can change backends without re-instrumenting. Exporting to a different tool is a config change (an endpoint and a header), not a rewrite. Develop against a local Phoenix, ship to a hosted vendor, keep a Jaeger for the platform team, all from the same instrumentation.

Your agent joins the rest of the system's traces. Most companies already run OTel for their services. If your agent emits OTel spans, one trace can show the full path: HTTP request into your API, agent loop, tool call, database query. Agent debugging stops being a separate island with separate tooling.

The boring work is done. Auto-instrumentation covers HTTP clients, databases, and queues, and the LLM ecosystem (framework instrumentors, OpenInference, vendor SDKs) increasingly emits OTel-shaped data out of the box. You write manual spans only for your own logic.

The Collector gives you a control point. Prompt redaction, sampling to control cost, and multi-backend fan-out all happen in the pipeline, not scattered through your codebase.

Here is the minimal Python setup, pointed at a local OTLP endpoint:

pip install opentelemetry-sdk opentelemetry-exporter-otlp

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
 
provider = TracerProvider()
provider.add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:6006/v1/traces"))
)
trace.set_tracer_provider(provider)
 
tracer = trace.get_tracer("my-agent")
 
with tracer.start_as_current_span("llm.chat") as span:
    span.set_attribute("gen_ai.request.model", "gpt-4o-mini")
    span.set_attribute("gen_ai.usage.input_tokens", 1874)
    span.set_attribute("gen_ai.usage.output_tokens", 312)

Swap the endpoint and the same spans land in a different backend. That is the whole pitch in one line of config.

What OTel does not do#

Being honest about the edges:

• No storage, no UI. You still need a backend. For local work I like Phoenix, which is the subject of the next post.
• GenAI conventions are young. Names have changed and will change again. Treat them as mostly-stable, not frozen.
• Framework coverage varies. Some agent frameworks have first-class OTel support, others rely on third-party instrumentors of mixed depth. Check what actually gets captured (tool arguments? streamed outputs?) before trusting it in production.

Wrapping up#

OTel gives you a standard shape for telemetry, a protocol to move it, and a growing GenAI vocabulary for the spans agents produce, while leaving you free to pick and swap backends. Instrument once against the standard and the rest of your observability stack becomes a set of config choices.

If you want the conceptual grounding first, start with tracing in agent observability, and for what to do with the data once it flows, see observability for LLM apps. Next up in this cluster: putting all of this to work locally with Arize Phoenix.