merge: lane/pkg-qa — GUI DoD gates wired, conformance required
CI / clang-format (push) Waiting to run
CI / testserver (push) Waiting to run
CI / bootstrap-script (push) Waiting to run
CI / bootstrap-script-2604 (push) Waiting to run
CI / extension-lint (push) Waiting to run
CI / build (clang) (push) Waiting to run
CI / build (gcc) (push) Waiting to run
CI / sanitizers (dev) (push) Waiting to run
CI / sanitizers (tsan) (push) Waiting to run
CI / clang-tidy (push) Waiting to run
CI / conformance (push) Waiting to run
CI / nightly-integration (push) Waiting to run
CI / gui-dod (push) Waiting to run
CI / gui-dod-nightly (push) Waiting to run

This commit is contained in:
2026-09-12 22:05:06 +04:00
4 changed files with 146 additions and 73 deletions
+2 -1
View File
@@ -18,8 +18,9 @@ policy so it can be re-applied or audited.
| `bootstrap-script-2604` | now — real `--with-clang` install in a 26.04 container; the release the project ships on |
| `build (gcc)` / `build (clang)` | now — core, daemon and gui have merged |
| `sanitizers (dev)` / `sanitizers (tsan)` | now — core, daemon and gui have merged |
| `conformance` | **now — `tests/conformance/` has landed; this is the M0 exit gate** |
| `conformance` | **now — `tests/conformance/` has landed; this is the M0 exit gate.** Includes the live-`veloxd` runner (step 3b of `run.sh`), unconditional in the script — see `docs/adr/0019-live-veloxd-conformance-is-required.md`. |
| `extension-lint` | now — `extension/` has merged (MV3 manifest + esbuild build) |
| `gui-dod` | now — `gui/tests/dod/` has landed (GUI M1 DoD gates R3: `scroll-60fps`, `unhappy-path`); see `tests/integration/README.md#gui-m1-definition-of-done-gates-r3`. `gui-dod-nightly` (`rss-flat`) is schedule-only and cannot be a required PR check. |
`clang-tidy` is intentionally **not** required through M1 (`continue-on-error: true`,
`.clang-tidy` has `WarningsAsErrors: ''`). Make it required at M2.
+53
View File
@@ -248,3 +248,56 @@ jobs:
run: cmake --build --preset dev --target veloxd
- name: Nightly integration run
run: python3 tests/integration/nightly_run.py --veloxd build/dev/bin/veloxd --tasks 50 --timeout 180
gui-dod:
# Per-PR GUI M1 DoD gates (gui/docs/pkg-qa-requests-m1.md R3): scroll-60fps and
# unhappy-path. The 10-minute rss-flat gate is gui-dod-nightly, not here. GUI's
# harness defaults QT_QPA_PLATFORM=offscreen itself, so no Xvfb/compositor needed.
# See tests/integration/README.md#gui-m1-definition-of-done-gates-r3 for what each
# gate catches and the forced-failure transcript proving it isn't vacuous.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Bootstrap toolchain
run: sudo ./tools/bootstrap.sh
- uses: actions/setup-node@v4
with:
node-version: '22' # tools/mockd
- name: Configure + build
run: |
cmake --preset dev
cmake --build --preset dev --target gui-dod-harness
- name: Install mockd
run: cd tools/mockd && npm ci
- name: Gates
run: |
gui/tests/dod/run.sh scroll-60fps --json scroll.json
gui/tests/dod/run.sh unhappy-path --json unhappy.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: gui-dod-${{ github.run_id }}
path: "*.json"
gui-dod-nightly:
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Bootstrap toolchain
run: sudo ./tools/bootstrap.sh
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Configure + build
run: |
cmake --preset dev
cmake --build --preset dev --target gui-dod-harness
- run: cd tools/mockd && npm ci
- name: RSS soak (10 min)
run: gui/tests/dod/run.sh rss-flat --json rss.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: gui-dod-rss-${{ github.run_id }}
path: rss.json
@@ -0,0 +1,45 @@
# ADR 0019 — The live-`veloxd` conformance runner is a required check, as-is
**Status:** accepted · **Date:** 2026-09-12 · **Lane:** PKG/QA
## Context
`tests/conformance/run.sh` step 3b (ADR 0014's `conformance` ctest, already required
per `.github/BRANCH_PROTECTION.md`) replays every fixture against a real, isolated
`veloxd` it builds and starts — not just mockd, which only proves the TS client and the
fixtures agree with each other. This is the runner that can catch `veloxd` disagreeing
with its own contract, and it is unconditional in `run.sh` (`set -euo pipefail`, no
skip flag): it already runs, and already blocks, inside the `conformance` job.
What was open was whether to treat that as a settled, defended gate or as something
still provisional while `daemon/docs/deferrals.md`'s D1-D4b stub handlers were excused
via `veloxd-xfail.json`. PROTO's update: 57/57 fixtures pass on `main`, and the xfail
allowlist is down to 18 entries from 34 as DAEMON lands the deferred handlers behind
them.
## Decision
The live-`veloxd` conformance runner stays required — no change to CI is needed, since
it already runs inside the already-required `conformance` job (ADR 0014). What this ADR
records is the standing: PKG/QA is not carving out an exception, a `continue-on-error`,
or a separate advisory job for it while the xfail list shrinks. A regression here fails
the same required check a schema mismatch would.
Verified, not assumed: `tests/conformance/veloxd-xfail.json` has 18 entries as of this
ADR (`python3 -c "import json; print(len(json.load(open('tests/conformance/veloxd-xfail.json'))))"`).
Each remaining entry excuses one still-stubbed handler on `deferrals.md`'s D-list, not a
real disagreement between `veloxd` and its contract — `tests/conformance/README.md`
already draws that line (an entry for anything else is a `run.sh`-detected "xfail entry
unexpectedly passed" or a straight failure, not a quiet pass).
## Consequences
- No `ci.yml` or `BRANCH_PROTECTION.md` change: `conformance` was already listed
required, and this runner was already inside it.
- The xfail list is a visible, shrinking number, not a static allowance — as DAEMON
clears more of `deferrals.md`'s D-list, entries come out of
`tests/conformance/veloxd-xfail.json`, and `replay.ts` fails loudly (per
`applyXfail`'s "unexpectedly passed" check) if one is left in after its handler ships.
- Nothing here changes who owns what: `veloxd-xfail.json` and `run.sh` stay PROTO's;
PKG/QA's role is the branch-protection policy this ADR confirms, not the runner
itself.
+46 -72
View File
@@ -88,88 +88,62 @@ manual testing.
nowhere to run. GUI owns the harness; PKG/QA owns the CI job. This is the wiring contract
so the two halves meet without another round trip.
### What PKG/QA needs from GUI
### What GUI built, and the invocation contract
A driver invoked as `gui/tests/dod/run.sh <gate> [--json <path>]` (exact path TBD by GUI),
headless-capable (Xvfb or offscreen `QT_QPA_PLATFORM`), with:
`gui/tests/dod/run.sh <gate> [--json <path>]` (`gate` one of `scroll-60fps` / `rss-flat`
/ `unhappy-path`), backed by `gui/tests/dod/dod_harness.cpp` (target `gui-dod-harness`).
Headless by default — `run.sh` sets `QT_QPA_PLATFORM=offscreen` itself, so the CI jobs
below need no Xvfb.
| `<gate>` | Pass / fail condition | Budget |
|---|---|---|
| `scroll-60fps` | `mockd --tasks 10000`, scripted fling scroll; **fail** if p99 frame > 16.6 ms | per-PR |
| `rss-flat` | `mockd --tasks 10000` + progress events, 10 min; **fail** if RSS growth > a fixed slack (GUI picks the number, states it) | nightly |
| `unhappy-path` | `mockd --slow` / `--flaky <f>` / `--drop-connection <s>`; **fail** on crash, on watchdog-detected hang, or if connection state never returns to `Connected` | per-PR |
| `scroll-60fps` | `mockd --tasks 10000`, scripted fling scroll; **fail** if p99 frame > 16.6 ms (×4 under an ASan/UBSan build — `VELOX_DOD_FRAME_BUDGET_MS` overrides outright) | per-PR |
| `rss-flat` | `mockd --tasks 10000` + progress events, 10 min; **fail** if RSS growth past warm-up exceeds a 20 MiB slack (`VELOX_DOD_RSS_SLACK_KIB` overrides) | nightly |
| `unhappy-path` | Three phases (`--slow` / `--flaky <f>` / `--drop-connection <s>`), one `mockd` restart each; **fail** on crash, on the harness's own watchdog firing (75 s), or if connection state never (re)reaches `Connected` | per-PR |
Contract:
Contract, as built: exit `0` pass, non-zero fail; a hang is the harness's own watchdog
converting itself into a non-zero exit (`3`), never something CI needs `timeout(1)`
around; `--json` writes one result object per gate (`unhappy-path` merges its three
phases into one file); no network, no writes outside a tempdir, no leaked child process
on any exit path (`run.sh`'s `trap cleanup EXIT INT TERM`).
* exit `0` pass, non-zero fail; a hang is the harness's own watchdog to catch and turn
into a non-zero exit, not something CI should have to `timeout(1)` around.
* `--json` writes one machine-readable result file (measured p99, RSS series, recovery
time) so the job can upload it as an artifact and a regression is a diff, not a re-run.
* no network, no writes outside a tempdir, no leaked child processes on failure.
### CI jobs — wired
### CI job — pre-drafted, add once the harness path is fixed
`gui-dod` (per-PR: `scroll-60fps` + `unhappy-path`) and `gui-dod-nightly` (`rss-flat`,
`schedule`/`workflow_dispatch` only) are live in `ci.yml`. Both bootstrap, build
`gui-dod-harness`, `npm ci` in `tools/mockd`, run `gui/tests/dod/run.sh`, and upload the
`--json` output as an artifact — no Xvfb step, since `run.sh` already runs offscreen.
```yaml
gui-dod:
# Per-PR GUI gates. The 10-minute rss-flat gate is in gui-dod-nightly, not here.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Bootstrap toolchain
run: sudo ./tools/bootstrap.sh
- uses: actions/setup-node@v4
with:
node-version: '22' # tools/mockd
- name: Configure + build
run: |
cmake --preset dev
cmake --build --preset dev --target velox-gui
- name: Install mockd
run: cd tools/mockd && npm ci
- name: Xvfb + gates
run: |
sudo apt-get install -y --no-install-recommends xvfb
xvfb-run -a gui/tests/dod/run.sh scroll-60fps --json scroll.json # TODO(GUI): path
xvfb-run -a gui/tests/dod/run.sh unhappy-path --json unhappy.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: gui-dod-${{ github.run_id }}
path: "*.json"
### Forced red, once per gate, before wiring it required
gui-dod-nightly:
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Bootstrap toolchain
run: sudo ./tools/bootstrap.sh
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Configure + build
run: |
cmake --preset dev
cmake --build --preset dev --target velox-gui
- run: cd tools/mockd && npm ci
- name: RSS soak (10 min)
run: |
sudo apt-get install -y --no-install-recommends xvfb
xvfb-run -a gui/tests/dod/run.sh rss-flat --json rss.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: gui-dod-rss-${{ github.run_id }}
path: rss.json
```
| Gate | Forced via | Observed |
|---|---|---|
| `scroll-60fps` | `VELOX_DOD_FRAME_BUDGET_MS=0.01` | `FAIL: p99=50.73 ms mean=31.34 ms max=52.88 ms budget=0.01 ms over 240 steps, 10000 rows`, exit 1 |
| `rss-flat` | `VELOX_DOD_RSS_SLACK_KIB=-999999999` (guarantees `growthKiB > slack` regardless of actual RSS behavior that run) | `FAIL: growth=13644 KiB slack=-999999999 KiB over 15s (warmup 1s), 10000 rows`, exit 1 |
| `unhappy-path` | ran `gui-dod-harness` directly (bypassing `run.sh`, which always starts a working `mockd`) against a `--sock` path with nothing listening | 75 s of `Connecting`/`Reconnecting`, then `FAIL: dod_harness watchdog fired — hung past 75s`, exit 3 |
`gui-dod-nightly` needs `if: github.event_name == 'schedule'` (the `nightly-integration`
job below already added that trigger to `ci.yml``cron: '17 3 * * *'` — so this no
longer needs its own).
The real (non-forced) runs all pass live: `scroll-60fps` p99 51.28 ms against a 66.4 ms
budget (ASan/UBSan build), `rss-flat` growth 13504 KiB against the 20480 KiB slack over a
15 s smoke duration, `unhappy-path` all three phases `PASS` with `finalConnected=true`.
### Known gap: `unhappy-path`'s drop-connection phase doesn't drop anything
Filed by GUI in `gui/docs/proto-requests-m1.md`: `mockd --drop-connection` only works
over WebSocket — `startUds()` never wires the periodic-drop timer `startWs()` has, so the
UDS transport (GUI/CLI/nmhost's only one) never sees a connection actually die. GUI's own
live verification: 45 s observing a `--drop-connection 5` mockd over UDS, `stateChanged`
never fires. The phase still runs and its JSON honestly records `sawDisruption: false`
rather than silently passing as if it proved something — that's what the real run above
shows, `pass: true` alongside `sawDisruption: false`, so a reviewer reading the JSON sees
exactly how much this phase currently covers.
This is PROTO's fix, not PKG/QA's or GUI's to route around locally — coordinating on it
rather than patching mockd from this lane. Once PROTO lands `dropEverySec` on `startUds`,
`unhappy-path`'s drop-connection phase starts exercising a real drop and this note comes
out; until then `gui-dod` stays required as specified (crash/hang/never-reconnects still
catch real regressions), just not yet catching a swallowed real disconnect.
### Status
Blocked on GUI's harness. Not urgent (GUI M1 DoD, not M0). When GUI files the follow-up
with the real `run.sh` path and the `rss-flat` slack number, PKG/QA drops the `TODO(GUI)`
markers and marks `gui-dod` required. The `schedule:` trigger `gui-dod-nightly` needs is
already in `ci.yml`.
Wired and required: `gui-dod` runs per-PR, `gui-dod-nightly` on schedule. Revisit once
PROTO's UDS `--drop-connection` fix lands (see above).