# ADR 0014 — The conformance suite has one entry point: `ctest -L conformance`

**Status:** accepted · **Date:** 2026-09-10 · **Lane:** PKG/QA

## Context

The conformance suite could be started two ways, and both were live:

* `tests/conformance/run.sh` — the script a developer runs by hand; the CI `conformance`
  job called it directly.
* `ctest -L conformance` — PROTO wired `tests/conformance/CMakeLists.txt` to register
  `conformance` (which shells out to `run.sh`) and `conformance_cpp` (the finer-grained
  native runner) under that label.

Nothing was wrong with either, but the CI `conformance` job — the one named as the M0
exit gate in `.github/BRANCH_PROTECTION.md` — exercised only the script. The ctest
registration was reached only incidentally, by the unfiltered `ctest` run in the `build`
and `sanitizers` jobs. A change that broke the registration (a deleted `add_test`, a
renamed label, a `WORKING_DIRECTORY` regression) would not fail the check that is supposed
to be about conformance; it would fail a build job, or nothing.

Two entry points also means two answers to "did the suite pass", free to drift.

## Decision

`ctest -L conformance` is the canonical entry point. `run.sh` remains the thing it runs —
`tests/conformance/CMakeLists.txt` registers it verbatim — but CI reaches it only through
ctest, the same way a developer does.

* The `conformance` job runs `cmake --preset dev`, builds just the `velox_conformance_cpp`
  target, then `ctest --preset dev -L conformance --output-on-failure`. This puts PROTO's
  registration on the exercised path.
* The rot guard is `noTestsAction: error` in the `dev` test preset: if the label ever
  matches zero tests, ctest exits non-zero instead of the old silent "0 tests, exit 0".
* The `build` and `sanitizers` jobs exclude the end-to-end test with `-E '^conformance$'`.
  It installs two npm trees and runs a node process against a self-compiled, unsanitized
  `g++` binary — running that once, in the dedicated job, is enough; sanitizing it buys
  nothing. `conformance_cpp` is *not* excluded and still runs on every matrix leg, which
  is where ASan/UBSan/TSan coverage of the generated wire code is worth having.

## Consequences

* One definition of "the suite passed".
* `run.sh` still works standalone and unchanged for local use and for
  `run.sh --uds … --ws-port …` against a real daemon; that path is out of scope for this
  ADR, which is only about what CI treats as authoritative.
* If PROTO ever wants the two labelled tests separated (e.g. a `conformance-e2e` label so
  jobs can select them independently without a name regex), that is a `tests/conformance/`
  change on their side; the `-E '^conformance$'` here is the PKG/QA-side stopgap until
  then.