Read a trace

The rest of this guide measures runs systematically, across a whole dataset. The manual version comes first, and you should never skip it. Reading a single trace by hand is how you build the intuition the systematic tools depend on: you cannot write a useful evaluator for a failure you have never actually looked at.

What a trace is

A trace is the full record of a single run: every step, with the exact input, output, latency, and cost. For a single LLM call it is one span. For an agent or a RAG pipeline it is a tree of spans, one per step.

Why reading it is the highest-leverage habit

Opening a trace shows you two things you would otherwise be guessing at:

• Where it went wrong. In a multi-step run, the trace tells you which step failed. A wrong RAG answer where the retrieval span already returned the wrong document is a retrieval problem, not a prompt problem, and the trace makes that obvious in seconds.
• What the model actually saw. Not what you assumed. The real, fully-rendered prompt, the real retrieved context, the real tool output. Half of all “the model is dumb” bugs turn out to be “the prompt was not what I thought it was,” and the trace is where you catch it.

In Respan, every run shows up in the Traces view as this tree, with each span’s input and output captured for you. Open a few, especially the ones that went wrong, and read them end to end before you do anything else.

When one trace is not enough

Reading a trace tells you what happened in the case in front of you. What it cannot tell you is whether a change helps in general: fix the prompt for this trace, and you still do not know if it helped the other fifty cases or quietly broke five. That is the question the systematic evals in the rest of this guide answer. Reading traces is how you find a problem; a dataset and an evaluator are how you measure the fix.

What this gives you

• You debug from evidence, not guesses. The trace tells you which step failed and what the model really saw.
• You learn what good and bad look like, which is the prerequisite for writing any evaluator.
• You know when one case is not enough, which is your cue to systematize.

Next steps

• Try a fix fast, on the input you just diagnosed, in the playground.
• Measure a fix across many cases with your first systematic eval, evaluate a single LLM call.