Explore traces

The Trace Explorer is where you search, filter, and inspect every trace flowing through your AI applications. It's available in Studio at Observability → Traces.

Before you can explore traces, your application needs to be instrumented. See Send traces for setup instructions.

What traces contain

What traces contain

Every trace is a tree of spans. Each span represents one operation in the execution chain.

FieldDescription
Trace IDUnique identifier for the full execution chain
SpansIndividual operations: LLM calls, tool calls, workflow steps
DurationTime taken for each span and the total trace
Token countsInput and output tokens per LLM call
StatusSuccess or error, with error messages when applicable
SourceWhich product generated the trace (SDK, Workflows, Vibe Work, Managed Agents)
TagsCustom tags from SDK usage, plus automatic metadata such as model name

Click any trace in the list to open the span tree. Expand individual spans to inspect inputs, outputs, latency, and token usage at each step.

Search and filter

Search and filter

Explorer accepts a search expression language: write conditions on trace fields and combine them with AND, OR, and NOT.

Explorer has two views. Traces searches the trace-level fields listed in Available fields. Spans searches individual spans, including any span attribute.

Operators

Operators

CategoryOperatorsExample
Comparison=, !=, <, <=, >, >=duration_ns > 5000000000
Pattern matchingLIKE, ILIKE, NOT LIKE, NOT ILIKEservice_name LIKE '%agent%'
SubstringCONTAINS, NOT CONTAINSservice_name CONTAINS 'sdk'
RegexREGEXP, NOT REGEXPservice_name REGEXP '^my-'
RangeBETWEEN ... AND ..., NOT BETWEEN ... AND ...duration_ns BETWEEN 1000000 AND 5000000000
ListIN (...), NOT IN (...)status_code IN ('Error', 'Unset')
Null checkEXISTS, NOT EXISTSconversation_id EXISTS
BooleanAND, OR, NOT, ()status_code = 'Error' AND duration_ns > 1000000
Available fields

Available fields

FieldTypeDescription
service_namestringName of the service that produced the trace
status_codestringTrace status: Error if the trace failed, otherwise Unset
duration_nsnumberTotal trace duration in nanoseconds
error_countnumberNumber of spans with an error in the trace
start_timetimestampWhen the trace started (ISO 8601)
models_usedarrayModels called within the trace
Span attribute queries

Span attribute queries

To filter on any OpenTelemetry span attribute, switch the Explorer from Traces to Spans and use bracket syntax:

span_attributes['gen_ai.request.model'] CONTAINS 'mistral-large'
span_attributes['gen_ai.operation.name'] = 'chat'
span_attributes['custom.tag'] EXISTS
Array fields

Array fields

For array fields like models_used:

FunctionExample
CONTAINSmodels_used CONTAINS 'mistral-medium-latest'
has(field, value)has(models_used, 'mistral-medium-latest')
hasany(field, [values])hasany(models_used, ['mistral-large-latest', 'mistral-small-latest'])
hasall(field, [values])hasall(models_used, ['mistral-large-latest', 'mistral-small-latest'])
Common examples

Common examples

# All failed traces from a specific service
service_name = 'my-agent' AND status_code = 'Error'

# Traces slower than 5 seconds
duration_ns > 5000000000

# Traces using a specific model
has(models_used, 'mistral-large-latest')
Add spans to a dataset

Add spans to a dataset

Turn production spans into evaluation records. Use the resulting Studio dataset to drive an offline evaluation.

i
Information

Adding spans to a dataset is part of offline evaluations, available to Enterprise-tier organizations only.

Add a single span

Add a single span

  1. Open a trace and select a span in the span tree.
  2. Open the span actions menu (⋯) and click Add span to dataset.
  3. Select a dataset, or create one from the dataset picker.
  4. Review the payload in Preview and edit. By default, it contains the span's resource and span attributes as JSON.
  5. Optionally click Keep input, output only to reduce the payload to the span's input and output messages, or edit the JSON directly. Click Reset changes to restore the original payload.
  6. Click Add to dataset.

The payload must be a valid JSON object. After the record is created, click View record in the confirmation to open it in the dataset.

Add multiple spans

Add multiple spans

  1. In Explorer, switch the view from Traces to Spans, then filter to the spans you want.
  2. Select spans with the row checkboxes. You can select up to 500 spans at once.
  3. Click Add to dataset in the selection bar, then pick a dataset or create one.
  4. Map span fields to record fields. For each mapping, pick a Span field (a span or resource attribute) and name its Record destination, either an existing field of the dataset or a new one.
  5. Check the Record preview and the import summary, then click Continue and confirm the import.

Each selected span becomes one record that contains only the mapped fields. Spans with no value for any mapping are skipped. The import runs in the background. A notification tracks its progress and links to the dataset when it completes.

Managed Agents traces

Managed Agents traces

Each turn of a Managed Agents session is one trace. The root span is invoke_agent <agent name>, with the turn's model calls (chat) and tool calls (execute_tool) under it. Tool calls made by a run_typescript program are nested under that program's span.

A subagent doesn't start a trace of its own. Its turn appears inside the parent's trace, under the agents.spawn or agents.send_message tool span that started it:

invoke_agent support-bot
├── chat
├── execute_tool agents.spawn
│   └── invoke_agent researcher
│       ├── chat
│       └── execute_tool web_search
└── chat

These attributes link spans to the session they belong to:

AttributeDescription
gen_ai.conversation.idID of the session. A subagent's spans carry the subagent's own session ID.
mistral.turn.idID of the turn within the session. Turn IDs are unique per session only.
mistral.conversation.parent.idOn subagent spans, ID of the parent session.
gen_ai.agent.name, gen_ai.agent.idAgent that ran the turn, on the invoke_agent span.

To open the session a trace belongs to, select the root invoke_agent span and choose Go to agent session from the span actions menu.

To find the trace of one turn, filter on both the session and turn IDs:

span_attributes['gen_ai.conversation.id'] = '<session_id>' AND span_attributes['mistral.turn.id'] = '<turn_id>'
Query traces programmatically

Query traces programmatically

All trace data is accessible via the REST API. The endpoints are documented in the API reference:

  • Traces: search and aggregate traces, get a trace with its spans, list fields and field options.
  • Spans: search and aggregate spans across traces, query span evaluations.
Access

Access

Reading traces requires an Enterprise organization and the Org Admin, Workspace Admin, or Observability Viewer role. For what each role can read, see Access.

FAQ

FAQ