LLM Observability (OTEL / Langfuse)
AI Engineer path · lesson 94 of 124Explore the standard trace hierarchy — session → trace → span → generation — read OTEL GenAI semantic conventions, and debug a real failing session by clicking through spans, tool errors, and feedback.
The same generation as span g1 in the Trace hierarchy tab — by far the largest slice of the timeline.
Key insight
A span is nothing more than a name, a start time, and a duration — attributes and a parent link are optional extras. Lay every span in a trace out by its start time and you get a waterfall: the same view Datadog, Langfuse, and Jaeger use to show exactly where a request's time went. Notice the small gaps between bars, too — that's real time (network hops, queueing) that no span covers. Click "Simulate slow vector DB" to watch a bottleneck appear.
Nest a trace correctly
Order these from the outermost span to the innermost.
Tap a piece, then tap where it goes — or drag it. Tap the tray to take one back.
- 1Drop a step here
- 2Drop a step here
- 3Drop a step here
- 4Drop a step here
Test your knowledge
In the standard trace hierarchy, what nests inside what?
The short version
An observability tool for LLMs is a plane's black box. When something goes wrong you don't want to re-fly the flight — you want to open the recorder, read what the pilot said, which button they pressed, and what each instrument reported at that moment.
In more depth
LLM observability
An LLM observability stack (Langfuse, Arize, Helicone, Datadog LLM Observability, or a homegrown OTEL setup) records every layer of what your product actually did in production.
Anatomy of a span
A span is the atomic unit of a trace: a name, a start time, and a duration, plus optional attributes (model, tokens, cost, error) and a link to its parent span. A single RAG request is really several spans back to back — parse the query, embed it, search a vector DB, assemble context, call the model — each with its own timing. Lay those spans out on a time axis by their start offset and you get a waterfall: the same trace-viewer layout every observability tool (Datadog, Langfuse, Jaeger) uses, because it makes a bottleneck visually obvious — one span balloons, and everything after it visibly shifts right.
The trace hierarchy
OTEL GenAI semantic conventions
Standardized attribute names — gen_ai.request.model, gen_ai.request.temperature, gen_ai.usage.input_tokens, gen_ai.response.finish_reasons, session.id, user.id — so instrumenting once works with every OTEL-compatible backend.
Above the traces
Below the traces
Interview Questions
References & Further Reading
Related topics
Love this visualization?
Sign up to track your progress, earn achievements, use the AI assistant, and access your personal learning dashboard.
100% free • No credit card required