# tests/conformance — one suite, four runners

**This is a required check on every lane's PR.** It is the mechanism that makes four
parallel lanes safe: the C++ daemon and the TypeScript extension are proved compatible
without either having run against the other.

```sh
./tests/conformance/run.sh                                  # starts its own mockd + veloxd
./tests/conformance/run.sh --uds /run/user/1000/velox/velox.sock --ws-port 52000
```

## The runners

| Runner | Needs | Asserts |
|---|---|---|
| `check_contract.py` | python3, jsonschema | schemas parse and resolve; the documented surface matches the schema surface both ways; every method has a success fixture; every fixture validates; `SettingKey` and `Settings` agree; **committed generated code is not stale** |
| `cpp/` | a C++23 compiler, nlohmann | every golden payload parses into the generated structs, serialises back stably, and goes through the real `dispatch()`; privileged methods are refused `-32003` over the WebSocket |
| `ts/replay.ts` against mockd | node ≥ 20 | the TS client and the fixtures agree with each other — mockd always answers every fixture correctly by construction, so this cannot catch veloxd disagreeing with the contract |
| `ts/replay.ts` against veloxd | the above, plus a C++23 toolchain (`daemon/CMakeLists.txt`'s deps) | the **real** daemon it builds and starts, isolated (its own `XDG_RUNTIME_DIR`/`XDG_DATA_HOME`/`XDG_CONFIG_HOME`), answers every fixture — except ones hitting a still-stubbed handler, excused by `veloxd-xfail.json` (see below) |

### `veloxd-xfail.json`

`daemon/docs/deferrals.md` (D1-D4b) lists the handlers still stubbed out (`-32603 not
implemented`); the fixtures that hit them can't pass against veloxd yet and are listed
here with why, keyed by fixture path. This is a maintained allowlist, not a snapshot: a
listed fixture that unexpectedly *passes* is flipped back to a failure by `replay.ts`
(`applyXfail`) rather than silently accepted, so an entry has to be deleted the same PR
that closes the handler — the list can only shrink, never rot into "things nobody checks."

This is also where a real veloxd bug shows up before it reaches anyone else: an
implemented handler returning something the schema forbids (e.g. `download.get` with
`segments: 0`, which `TaskSummary.segments` requires >= 1) is **not** in the allowlist, so
it fails the run for real. That is the point of running against veloxd at all, not just
mockd.

### Current status against veloxd: red, for two reasons — not just the one

As of this wiring, the veloxd runner does not pass, and shouldn't yet:

1. **`download.get` / `download.list` can return `segments: 0`.** `store::TaskRow::eff_segments`
   defaults to `0` and `to_summary` copies it straight into `TaskSummary.segments`
   (`daemon/src/store/tasks.{hpp,cpp}`), which the schema forbids (minimum 1, required).
   This isn't only a post-completion thing — it's any task the engine hasn't segmented yet,
   which includes a task the instant it's added. This is the regression this runner exists
   to catch, and it is deliberately **not** in `veloxd-xfail.json`.
2. **New finding: `download.add` with `startMode: "later"` always fails.** `later` is a
   valid, documented `StartMode` (`contracts/schema/types/StartMode.schema.json`:
   `["now", "later", "queue"]` — "'later' is the File Info dialog's Download Later
   button"), and `on_download_add` stores it verbatim as `tasks.start_mode`
   (`daemon/src/rpc/dispatcher.cpp`). But the `start_mode` column's `CHECK` constraint
   (`daemon/src/store/migrations/0001_initial.sql`) only allows
   `'auto','now','queue','manual'` — no `'later'`, and `'manual'`/`'auto'` aren't contract
   values at all. Every `download.add` with `startMode: "later"` — including this runner's
   own fixture-binding setup, which needs one to exist before it can replay any
   `$taskId`-referencing fixture — fails `-32603` on a SQLite `CHECK` violation. This is not
   in `veloxd-xfail.json` either: it's not a stub (D-list), it's a real, currently-shipping
   bug, and it's why the veloxd run is red across most of the suite right now, not just on
   `download.get`. Filed to DAEMON; not fixed here (out of lane).

`run.sh` also runs one scenario that cannot be shown against a healthy server: with the
daemon answering slower than `capture.offer`'s 750 ms deadline, the client must give up and
let Firefox take the download. **That is the fail-open guarantee, and it is checked here.**

## What "passing" means

The runners check the contract, not the implementation's opinions. Results are compared by
shape and validated against the generated validators; error codes are compared exactly.
Byte-equality with a golden file is deliberately *not* asserted, because a live daemon
returns its own ids and its own clock — see `contracts/fixtures/README.md`.

Adding a method without a fixture fails `check_contract.py`. Regenerating and forgetting to
commit the output fails it too.

## CI

Wired in as `ctest -L conformance` (`.github/workflows/ci.yml`'s `conformance` job, PKG/QA;
see `docs/adr/0014-conformance-runs-through-ctest.md`), required on every branch. It starts
and stops its own `mockd` and its own `veloxd`; nothing else needs to be running.

Needs: `python3` with `jsonschema`, a C++23 compiler, the same deps `daemon/CMakeLists.txt`
needs (SQLite3, nlohmann-json, libsecret — `tools/bootstrap.sh` installs all of it), and
Node ≥ 20.

### One caveat: veloxd's single-instance lock is not isolation-aware

veloxd refuses to start a second copy for the same user — an abstract-namespace socket
keyed by UID only, not by `$XDG_RUNTIME_DIR` (`daemon/src/main.cpp`,
`acquire_single_instance_lock`). This run's isolated veloxd collides with that lock exactly
like any other copy would: if a real veloxd (or another worktree's integration run) is
already up for this user when `run.sh` starts, the veloxd step fails fast with "another
instance is already running for this user" rather than silently testing the wrong daemon.
A fresh CI runner never hits this — only concurrent local runs can. If that turns out to
bite CI in practice (two conformance jobs sharing a runner user, say), the fix belongs in
`daemon/` (scope the lock name to the runtime dir), not here.