Skip to main content
Clousight Bench traces are OpenTelemetry-native: produced through the OTel SDK, attributed with OTel semantic conventions, and exportable over OTLP. One evaluation run = one trace.
Traces are evidence, never verdicts: evaluators do not read them, benchmark fingerprints exclude them, and a broken exporter can never fail a run.

​
What gets traced

All spans share the run’s W3C trace_id (preset at orchestrator start), so the whole evaluation reads as one tree.

​
Where traces go

  1. Local JSONL (always): <results>/traces/<trace_id>.jsonl — flat OTel spans (hex ids, nanosecond times, resource per line). Inspect with csbench trace list / csbench trace show <run-id>.
  2. Suite trajectory artifact: trajectory.jsonl in the run’s artifacts — rendered as a waterfall by csbench serve (per-query TPC lanes, per-call LLM spans).
  3. Your collector (optional): install the extra and point at OTLP/HTTP —
Third-party exporters register under the clousight_bench.span_exporters entry point — the contract is the OTel SDK’s SpanExporter, so any ecosystem exporter plugs in unmodified.

​
Metrics & logs signals

With the same CLOUSIGHT_OTLP_ENDPOINT (+ [otlp]), a finished run also ships:
  • metrics — one gauge per numeric measurement (csbench.<key>, attributes: unit, reproducibility_class, official, the run’s trace id);
  • logs — one record per run error and finding, severity-mapped, trace-correlated.
Both are one-shot snapshots at finalize and fail-safe (telemetry never fails a run). The endpoint value may be the collector base or a full /v1/traces URL — per-signal paths are derived.

​
Importing external OTel spans

A SUT (or any tool) instrumented with standard OTel can hand its spans to the waterfall: csbench trace import <file> accepts OTLP/JSON (resourceSpans) or flat span-per-line JSONL and produces a validated v3 trajectory. Spans without a semconv discriminator get an honest csbench.ingested marker.

​
Context propagation (traceparent)

LLM-endpoint calls carry a W3C traceparent header. If the service you evaluate is itself OTel-instrumented (most managed gateways are), your APM shows the run’s trace continuing inside your service — the eval process and the SUT’s internal view join into one distributed trace, correlated by the same trace id the record stores under extensions.core.trace_id.

​
Honest boundaries

  • BenchBase (TPC-C) and YCSB run as Java subprocesses: each tool invocation gets one measured phase span (TPC-C: create+load+execute; YCSB: load and run), never internal spans.
  • TPC official spans are reconstructed from the measured intervals (the concurrency model is known: sequential phases, back-to-back queries per stream) — faithful timing, not live instrumentation.
  • The deployed agent-bundle protocol still emits legacy v2 spans; the viewer renders both, new span sources must emit v3 (OTel-native).