OpenTelemetry for LLM apps: trace prompts, tools and cost
OpenTelemetry's GenAI conventions give every model call, tool use and agent step a standard span, with the tokens it used. How to turn tracing on for an LLM app, what the spans hold, how to keep prompts out of them, and how to add up tokens without counting them twice.
Tecyfy · Published · 8 min read

OpenTelemetry traces an LLM app the way it traces any other service: each step becomes a span, and the spans of one request form a trace. What makes it work for model calls is a set of shared names, the GenAI semantic conventions. A model call becomes a span such as chat claude-sonnet-5-5 with gen_ai.usage.input_tokens and gen_ai.usage.output_tokens on it, a tool call becomes execute_tool search_tickets, and any platform that knows the conventions can show them without custom parsing.
That is the case for using it over a vendor's own tracing format: the same spans go to Datadog, Grafana, Honeycomb, Langfuse or your own collector, and your dashboards survive a change of model provider. Anthropic adopted it for Claude on 14 October 2026, when it launched OpenTelemetry export in beta for the Claude API and its SDKs.
What a trace of an LLM app looks like
A trace of one agent request nests like this, using the span names the conventions recommend:
invoke_agent support_agent 3.5 s
├─ chat claude-sonnet-5-5 1.3 s 3,200 in / 180 out finish: tool_use
├─ execute_tool search_tickets 0.4 s
└─ chat claude-sonnet-5-5 1.7 s 4,100 in / 260 out finish: end_turn
From that you can read where the time went (the model, not the tool), what the request cost in tokens, and why the first call stopped (it asked for a tool). With content recorded, you can also read what the user asked and what the model said. That is what the TracingTracingRecording every step an LLM app or agent takes for one request, each model call, tool call and retrieval with its input, output, tokens and time, so you can see why an answer went wrong. concept means by a trace: the record of one request through every step, which you then turn into ObservationObservationA piece of information an agent receives after executing an action. data for debugging and evals.
The widget below is that trace. Press each span to see its attributes, switch content on and off, and add up its tokens 2 ways.
- invoke_agent support_agent3480 ms
- chat claude-sonnet-5-51310 mscounted
- anthropic.messages.create1240 mscounted
- execute_tool search_tickets420 ms
- chat claude-sonnet-5-51650 mscounted
- anthropic.messages.create1580 mscounted
chat claude-sonnet-5-5 · kind CLIENT
- gen_ai.operation.name
- chat
- gen_ai.provider.name
- anthropic
- gen_ai.request.model
- claude-sonnet-5-5
- gen_ai.usage.input_tokens
- 3200
- gen_ai.usage.output_tokens
- 180
- gen_ai.response.finish_reasons
- ["tool_use"]
- gen_ai.system_instructions
- (not recorded)
- gen_ai.input.messages
- (not recorded)
- gen_ai.output.messages
- (not recorded)
No prompt text leaves
Summing every span that carries tokens gives 14,600 input tokens. That is twice the real figure: each model call has a client span from the SDK and a server span from the API, both with the same usage.
A made-up trace in the shape of OpenTelemetry's GenAI conventions and the Claude SDK's spans. Attribute names follow the conventions; values are invented.
The span names and attributes that matter
The GenAI conventions have Development status, so names can still change. They live in their own repository, open-telemetry/semantic-conventions-genai, with general conventions plus provider-specific ones for Anthropic, OpenAI, AWS Bedrock and Azure AI Inference, and pages for MCP and the A2A protocol.
Model calls. The client inference span should be named {gen_ai.operation.name} {gen_ai.request.model}, with kind CLIENT. Its attributes, by requirement level:
| Attribute | Level | Holds |
|---|---|---|
gen_ai.operation.name | Required | chat, execute_tool, invoke_agent and other known values |
gen_ai.provider.name | Required | The provider, such as anthropic |
gen_ai.request.model | If available | The model asked for |
gen_ai.response.model | Recommended | The model that answered |
gen_ai.response.finish_reasons | Recommended | Why it stopped |
gen_ai.usage.input_tokens | Recommended | Input tokens, cached ones included |
gen_ai.usage.output_tokens | Recommended | Output tokens |
gen_ai.usage.cache_read.input_tokens | When available | Tokens read from the prompt cache |
gen_ai.usage.reasoning.output_tokens | When available | Tokens spent on reasoning |
gen_ai.input.messages, gen_ai.output.messages | Opt-in | The prompt and the reply |
Tool calls. An execute_tool {gen_ai.tool.name} span, with gen_ai.tool.name required and the tool call id when the model gave one. The arguments and the result are opt-in.
Agents. An agent run gets an invoke_agent {gen_ai.agent.name} span over its steps, and gen_ai.conversation.id ties the spans of one conversation together across requests.
Turn it on in Python
Most of the work is OpenTelemetry's ordinary setup: a tracer provider and an exporter. Install the SDK and the OTLP exporter:
pip install --upgrade anthropic opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
Then register the provider before the first call you want traced. A current Claude SDK finds it and emits a span for each request, and one for each tool its tool runner runs:
from anthropic import Anthropic
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")))
trace.set_tracer_provider(provider)
client = Anthropic()
tracer = trace.get_tracer("support-agent")
with tracer.start_as_current_span("invoke_agent support_agent") as span:
span.set_attribute("gen_ai.operation.name", "invoke_agent")
span.set_attribute("gen_ai.agent.name", "support_agent")
message = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Why was I charged twice?"}],
)
The SDK's span nests under your invoke_agent span, so the model call appears inside the request that caused it. The endpoint here is an OpenTelemetry Collector on its default HTTP port; point it at your platform's OTLP endpoint instead. Your own tools get spans the same way: wrap each in execute_tool <name> with gen_ai.tool.name set.
Anthropic's SDKs do the same in TypeScript, C#, Go, Java, PHP and Ruby. During the beta it warns that options and span names can change in a new SDK version, and advises pinning the version so a release does not break your dashboards.
For spans from inside the API, such as each model run within a request and the server tools it ran, an admin adds an OpenTelemetry destination in the Claude Console. With both on, the SDK and API spans of a request join in one trace.
Claude Code has its own tracing, also in beta and off by default: set CLAUDE_CODE_ENABLE_TELEMETRY=1, CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 and OTEL_TRACES_EXPORTER. Our Claude Code costs page covers what it spends.
Keep prompts out unless you mean to store them
Prompts and replies are the most useful part of a trace and the most sensitive. The conventions make every attribute that holds them opt-in, and say plainly that they are likely to contain personal data. The tools follow:
- The Claude SDK and API default to metadata only: identifiers, timing, status, token counts and the names you chose for agents, tools and MCP servers. Full content adds the system prompt, the messages and the reply.
- Claude Code redacts prompt text, tool input and tool content in its spans unless you set
OTEL_LOG_USER_PROMPTS,OTEL_LOG_TOOL_DETAILSorOTEL_LOG_TOOL_CONTENT.
Anthropic points out the catch with full content: your observability platform stores it under its own access and retention rules, so check those before you switch it on. The conversation id is exported in both modes, so keep personal data out of it too.
Size is the other limit. With full content, Anthropic caps each span at 0.75 MB by default, enough for a 200k-token context of English text, and below the 1 MB request limit of many platforms. A full 1M-token context needs about 4 MB, so raise the limit only if your platform accepts it. When content is cut, the span says so with anthropic.content.truncated.
Add up tokens once, then price them
A trace makes cost per request, per user or per feature easy to see, because the token counts sit on the spans next to everything else. The conventions record tokens, not money: multiply by the model's prices yourself, from our model price table.
2 traps make the sum wrong:
- The same call appears twice. With both SDK and API tracing on, each Claude request has a client span from the SDK and a server span from the API, both named
anthropic.messages.createand both carrying the usage. Anthropic's rule is to sum one source per request: the API spans when you have both, told apart by span kind (CLIENTorSERVER). The conventions say the same for any provider: group inference spans byservice.namewhen counting. - Cached tokens are already inside input tokens.
gen_ai.usage.input_tokensincludes the tokens read from and written to the prompt cache. Addingcache_readon top counts them twice; instead, split them out and price them at the cache rates, which on Claude are lower for reads and higher for writes. The prompt caching calculator shows what that difference is worth.
Spans can also arrive twice when a delivery is retried, so count each by its trace and span id. And traces are not a bill: Anthropic says delivery is best effort and a span can be lost, and points to its Usage and Cost Analytics API for a complete record. Use traces to explain spend, and the provider's usage data to total it. Our worked AI agent cost per month shows what those totals add up to for a real agent.
Where traces pay off
The first win is usually debugging: a slow request shows whether the model, a tool or a retry took the time, and a wrong answer shows the tool result it was built on. The second is evals. With content recorded, failed traces become test cases, which is how our guide to evaluating an AI agent builds its test set. Start with metadata only, find the requests worth reading, and record content only where you need it.
Questions
- What is OpenTelemetry for LLMs?
- It is OpenTelemetry, the vendor-neutral standard for traces, metrics and logs, used with its semantic conventions for generative AI. The conventions name the spans and attributes for model calls, tool executions and agent steps (for example gen_ai.request.model and gen_ai.usage.input_tokens), so any observability platform that supports them can read an LLM app's traces.
- Are the OpenTelemetry GenAI semantic conventions stable?
- No. The GenAI semantic conventions have Development status, so names and attributes can still change. They now live in their own repository, open-telemetry/semantic-conventions-genai, which extends the core semantic conventions.
- What span does an LLM call get in OpenTelemetry?
- A client inference span, named after the operation and the model, such as chat claude-sonnet-5-5. It must carry gen_ai.operation.name and gen_ai.provider.name, and is recommended to carry the request and response model, finish reasons and the input and output token counts. A tool call gets a span named execute_tool followed by the tool's name.
- Does OpenTelemetry record the prompt and response?
- Not by default. In the GenAI conventions the attributes that hold prompts, responses, system instructions and tool arguments are opt-in, because they can hold personal data. The Claude SDK and Claude API follow the same idea: metadata only is the default, and full content has to be switched on.
- How do I trace the Claude API with OpenTelemetry?
- Register an OpenTelemetry tracer provider in your app and use a current Claude SDK, which then emits a span for each request and each tool the tool runner runs. For spans from the API itself, an admin adds an OpenTelemetry destination in the Claude Console. Both are in beta since 14 October 2026.
- How do I get the cost of an LLM call from a trace?
- Add up the token attributes of the model call spans, counting each call once, and multiply by the model's prices. Input tokens already include tokens read from and written to the prompt cache, which are priced differently, so split those out with the cache attributes before applying prices.
Sources
Where each fact comes from
- 1Semantic conventions for generative AI systems
- 2OpenTelemetry GenAI Semantic Conventions (repository)
- 3Semantic conventions for generative AI client inference
- 4Execute tool span
- 5Semantic conventions for GenAI agent spans
- 6OpenTelemetry tracing for the Claude API
- 7Set up OpenTelemetry with the Claude SDK
- 8Claude API release notes: October 14, 2026
- 9Monitoring (Claude Code)
- 10Exporters (Python)