> ## Documentation Index
> Fetch the complete documentation index at: https://docs.clousight.com/llms.txt
> Use this file to discover all available pages before exploring further.

# TPC-DS suite (data-warehouse)

> Run TPC-DS through the benchmark-suite contract on the duckdb-local reference platform.

TPC-DS is Clousight Bench's second benchmark suite and the first on the
`data-warehouse` domain — the `benchmark_suite` / `evaluator` contract
generalized beyond agent suites (SWE-bench) to OLAP/data. The first platform is
`duckdb-local`: a provider-less, single-node **reference** engine (the analog of
agent-runtime's `local-sim`) that runs entirely offline.

## Run it (no cloud account)

```bash theme={null}
# offline mock path — no DuckDB needed, exercises the whole stage machine:
printf 'target:\n  mode: mock\n' > mock.yaml
csbench run --domain data-warehouse --benchmark tpc-ds \
  --platform duckdb-local --config mock.yaml

# real DuckDB SF1 run (needs the [tpcds] extra: pip install clousight-bench[tpcds]):
printf 'target:\n  mode: runtime\nparams:\n  scale_factor: 1\n  query_ids: [3, 7, 42]\n' > real.yaml
csbench run --domain data-warehouse --benchmark tpc-ds \
  --platform duckdb-local --config real.yaml
```

`params.scale_factor` defaults to `1`; `params.query_ids` defaults to all 99
queries. Omit `query_ids` to run the full set (\~1.3s at SF1 after a one-time
\~6s data generation).

## What it measures

The `official-tpcds-evaluator` emits three `tpc-ds.*` measurements:

| Measurement | Meaning | reproducibility\_class |
| - | - | - |
| `tpc-ds.queries_passed` | ratio of queries whose result matches the SF-keyed pinned reference (`sf<sf>_digests.json`; only SF1 is shipped — omitted at other SFs) | `deterministic` |
| `tpc-ds.geomean_latency_ms` | geometric mean of per-query wall-time | `environmental` |
| `tpc-ds.total_runtime_ms` | sum of per-query wall-time | `environmental` |

All carry `official=True` — in this framework that is a *provenance* flag
("emitted by the suite's recognized evaluator"), not an audit claim; whether a
number drifts is carried by `reproducibility_class`.

## Correctness: a pinned SF1 reference

DuckDB's `tpcds_answers()` is unusable (no parameters, defaults to SF10), so
correctness compares each query's normalized result digest against a **pinned
SF1 reference** captured once against the pinned DuckDB + extension version
(`suites/tpc_ds/fixtures/reference/sf1_digests.json`, produced by
`scripts/capture_tpcds_reference.py`). This is a deterministic
reproducibility/regression check against a recognized implementation — **not an
externally-audited TPC answer** — and is emitted **only at scale factor 1** (the
reference is SF1-only). The digest normalization rounds numerics and sorts rows
so it is stable across platforms; a real run reproduces the committed reference
in CI.

## Deliberate scope (slice 1)

* Correctness is SF1-only; higher scale factors report performance only.
* An *audited* **QphDS** number is never claimed. The unaudited official-formula
  composite exists: `params.mode: official` runs the full sequence (load + power
  * concurrent throughput streams + data maintenance + ACID gate) and computes
    QphDS\@SF — see the **Official mode** section below.
* Big-data scale and distributed engines (Spark on EMR, cloud data warehouses)
  are a later phase, added as additional `data-warehouse` platforms behind the
  same suite.

## Sibling: TPC-H

TPC-H is the second OLAP suite on the same `data-warehouse` domain and
`duckdb-local` platform — proof that a new suite drops onto the contract cheaply.
It shares the shape of TPC-DS (offline mock + real DuckDB, honest per-query
latency), just with DuckDB's `tpch` extension (`dbgen` + 22 queries via
`PRAGMA tpch(n)`) — and goes further on correctness and composites: SF-keyed
**verified** references at SF 1 / 0.1 / 0.01 (each verified cell-by-cell against
DuckDB's official `tpch_answers()` at capture time) and its own official-formula
QphH\@Size mode. See [TPC-H suite](/docs/tpch-suite).

```bash theme={null}
csbench run --domain data-warehouse --benchmark tpc-h \
  --platform duckdb-local --config mock.yaml   # or a real.yaml with mode: runtime
```

Measurements are `tpc-h.queries_passed` / `tpc-h.geomean_latency_ms` /
`tpc-h.total_runtime_ms`. Needs the `[tpch]` extra for the real path. Unlike
TPC-DS, DuckDB ships usable `tpch_answers()` official answers, so TPC-H
correctness is **verified**, not merely pinned: `scripts/capture_tpch_reference.py`
verifies every reference entry against them at capture time and stamps it
`verified_official` — 22/22 verified at each of the three shipped scale factors
(1, 0.1, 0.01).

## Official mode (QphDS\@SF)

`params.mode: official` runs the TPC-DS official sequence — Load → Power (99
queries, single stream) → Throughput Test 1 → Data Maintenance 1 → Throughput
Test 2 → Data Maintenance 2 → ACID gate — and the
`official-tpcds-qphds-evaluator` computes the official composite:

```
QphDS@SF = ⌊ SF · Q / ⁴√(T_PT · T_TT · T_DM · T_LD) ⌋
```

```bash theme={null}
printf 'target:\n  mode: runtime\nparams:\n  mode: official\n  scale_factor: 1\n  evaluator: official-tpcds-qphds-evaluator\n' > ds-official.yaml
csbench run --domain data-warehouse --benchmark tpc-ds --platform duckdb-local --config ds-official.yaml
```

The `official-tpcds-qphds-evaluator` emits:

| Measurement | Meaning | reproducibility\_class |
| - | - | - |
| `tpc-ds.qphds_at_size` | the QphDS\@SF composite (unaudited caveats in its `notes`) | `environmental` |
| `tpc-ds.load_time_s` | Load Test wall time | `environmental` |
| `tpc-ds.power_test_s` | Power Test — sum of per-query intervals | `environmental` |
| `tpc-ds.throughput_test_s` | Throughput Tests 1 + 2 combined | `environmental` |
| `tpc-ds.maintenance_test_s` | Data Maintenance 1 + 2 combined | `environmental` |
| `tpc-ds.acid_atomicity`, `tpc-ds.acid_isolation` | ACID probe pass/fail (consistency and durability are `n/a` → omitted) | `deterministic` |
| `tpc-ds.queries_passed` | SF1 only — pinned-reference correctness over the Power stream | `deterministic` |

`params.streams` defaults to the official minimum (4). The run also emits its
OTel-native trace (`trajectory.jsonl`) — the viewer waterfall shows the power
queries, the concurrent throughput lanes of both tests, and the DM windows.

<Note>
  **Unaudited, with honestly-declared substitutions**: DuckDB's `tpcds` extension
  has no LF\_\* maintenance functions, so data maintenance is a clousight-generated
  insert+delete round-trip on `store_sales`; no official permutation table is
  bundled, so the query ordering is always the deterministic clousight `generated`
  one (recorded as `ordering_source` and folded into the dataset digest); ACID
  probes atomicity/isolation generically — consistency and durability are `n/a`.
  Every composite carries these caveats in its `notes`.
</Note>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.