pkg: wire GUI's DoD harness into CI, mark live-veloxd conformance required

gui-dod (per-PR: scroll-60fps + unhappy-path) and gui-dod-nightly (rss-flat, schedule/workflow_dispatch) are live in ci.yml, driving GUI's newly-landed gui/tests/dod/run.sh + gui-dod-harness. No Xvfb step: run.sh already runs QT_QPA_PLATFORM=offscreen itself. Each gate forced red once before being trusted (tests/integration/README.md has the transcripts): VELOX_DOD_FRAME_BUDGET_MS=0.01 for scroll-60fps, VELOX_DOD_RSS_SLACK_KIB=-999999999 for rss-flat, and — since run.sh always starts a working mockd — a direct gui-dod-harness invocation against an unreachable socket for unhappy-path, which hit the harness's own 75s watchdog exactly as documented. Recorded GUI's live finding (gui/docs/proto-requests-m1.md) that mockd --drop-connection is a no-op over the UDS transport, so unhappy-path's drop-connection phase can't yet exercise a real drop — coordinating with PROTO on the fix rather than working around it locally. gui-dod stays required regardless: its other two phases and the crash/hang/watchdog paths still catch real regressions. Added gui-dod to BRANCH_PROTECTION.md's required-checks table. ADR 0019: the live-veloxd conformance runner (run.sh step 3b, already unconditional inside the already-required conformance job) stays required as PROTO's xfail list shrinks (18 entries now, down from 34; 57/57 fixtures passing on main). No CI change needed — it was already inside a required check; this records the decision not to carve out an exception for it. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01RBPR7iM3YPyxrjWsVtZDPJ
2026-09-12 22:04:09 +04:00
parent 75536304bb
commit 1d359af5a3
4 changed files with 146 additions and 73 deletions
@@ -88,88 +88,62 @@ manual testing.
 nowhere to run. GUI owns the harness; PKG/QA owns the CI job. This is the wiring contract
 so the two halves meet without another round trip.

-### What PKG/QA needs from GUI
+### What GUI built, and the invocation contract

-A driver invoked as `gui/tests/dod/run.sh <gate> [--json <path>]` (exact path TBD by GUI),
-headless-capable (Xvfb or offscreen `QT_QPA_PLATFORM`), with:
+`gui/tests/dod/run.sh <gate> [--json <path>]` (`gate` one of `scroll-60fps` / `rss-flat`
+/ `unhappy-path`), backed by `gui/tests/dod/dod_harness.cpp` (target `gui-dod-harness`).
+Headless by default — `run.sh` sets `QT_QPA_PLATFORM=offscreen` itself, so the CI jobs
+below need no Xvfb.

 | `<gate>` | Pass / fail condition | Budget |
 |---|---|---|
-| `scroll-60fps` | `mockd --tasks 10000`, scripted fling scroll; **fail** if p99 frame > 16.6 ms | per-PR |
-| `rss-flat` | `mockd --tasks 10000` + progress events, 10 min; **fail** if RSS growth > a fixed slack (GUI picks the number, states it) | nightly |
-| `unhappy-path` | `mockd --slow` / `--flaky <f>` / `--drop-connection <s>`; **fail** on crash, on watchdog-detected hang, or if connection state never returns to `Connected` | per-PR |
+| `scroll-60fps` | `mockd --tasks 10000`, scripted fling scroll; **fail** if p99 frame > 16.6 ms (×4 under an ASan/UBSan build — `VELOX_DOD_FRAME_BUDGET_MS` overrides outright) | per-PR |
+| `rss-flat` | `mockd --tasks 10000` + progress events, 10 min; **fail** if RSS growth past warm-up exceeds a 20 MiB slack (`VELOX_DOD_RSS_SLACK_KIB` overrides) | nightly |
+| `unhappy-path` | Three phases (`--slow` / `--flaky <f>` / `--drop-connection <s>`), one `mockd` restart each; **fail** on crash, on the harness's own watchdog firing (75 s), or if connection state never (re)reaches `Connected` | per-PR |

-Contract:
+Contract, as built: exit `0` pass, non-zero fail; a hang is the harness's own watchdog
+converting itself into a non-zero exit (`3`), never something CI needs `timeout(1)`
+around; `--json` writes one result object per gate (`unhappy-path` merges its three
+phases into one file); no network, no writes outside a tempdir, no leaked child process
+on any exit path (`run.sh`'s `trap cleanup EXIT INT TERM`).

-* exit `0` pass, non-zero fail; a hang is the harness's own watchdog to catch and turn
-  into a non-zero exit, not something CI should have to `timeout(1)` around.
-* `--json` writes one machine-readable result file (measured p99, RSS series, recovery
-  time) so the job can upload it as an artifact and a regression is a diff, not a re-run.
-* no network, no writes outside a tempdir, no leaked child processes on failure.
+### CI jobs — wired

-### CI job — pre-drafted, add once the harness path is fixed
+`gui-dod` (per-PR: `scroll-60fps` + `unhappy-path`) and `gui-dod-nightly` (`rss-flat`,
+`schedule`/`workflow_dispatch` only) are live in `ci.yml`. Both bootstrap, build
+`gui-dod-harness`, `npm ci` in `tools/mockd`, run `gui/tests/dod/run.sh`, and upload the
+`--json` output as an artifact — no Xvfb step, since `run.sh` already runs offscreen.

-```yaml
-  gui-dod:
-    # Per-PR GUI gates. The 10-minute rss-flat gate is in gui-dod-nightly, not here.
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - name: Bootstrap toolchain
-        run: sudo ./tools/bootstrap.sh
-      - uses: actions/setup-node@v4
-        with:
-          node-version: '22'          # tools/mockd
-      - name: Configure + build
-        run: |
-          cmake --preset dev
-          cmake --build --preset dev --target velox-gui
-      - name: Install mockd
-        run: cd tools/mockd && npm ci
-      - name: Xvfb + gates
-        run: |
-          sudo apt-get install -y --no-install-recommends xvfb
-          xvfb-run -a gui/tests/dod/run.sh scroll-60fps   --json scroll.json    # TODO(GUI): path
-          xvfb-run -a gui/tests/dod/run.sh unhappy-path    --json unhappy.json
-      - uses: actions/upload-artifact@v4
-        if: always()
-        with:
-          name: gui-dod-${{ github.run_id }}
-          path: "*.json"
+### Forced red, once per gate, before wiring it required

-  gui-dod-nightly:
-    if: github.event_name == 'schedule'
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - name: Bootstrap toolchain
-        run: sudo ./tools/bootstrap.sh
-      - uses: actions/setup-node@v4
-        with:
-          node-version: '22'
-      - name: Configure + build
-        run: |
-          cmake --preset dev
-          cmake --build --preset dev --target velox-gui
-      - run: cd tools/mockd && npm ci
-      - name: RSS soak (10 min)
-        run: |
-          sudo apt-get install -y --no-install-recommends xvfb
-          xvfb-run -a gui/tests/dod/run.sh rss-flat --json rss.json
-      - uses: actions/upload-artifact@v4
-        if: always()
-        with:
-          name: gui-dod-rss-${{ github.run_id }}
-          path: rss.json
-```
+| Gate | Forced via | Observed |
+|---|---|---|
+| `scroll-60fps` | `VELOX_DOD_FRAME_BUDGET_MS=0.01` | `FAIL: p99=50.73 ms mean=31.34 ms max=52.88 ms budget=0.01 ms over 240 steps, 10000 rows`, exit 1 |
+| `rss-flat` | `VELOX_DOD_RSS_SLACK_KIB=-999999999` (guarantees `growthKiB > slack` regardless of actual RSS behavior that run) | `FAIL: growth=13644 KiB slack=-999999999 KiB over 15s (warmup 1s), 10000 rows`, exit 1 |
+| `unhappy-path` | ran `gui-dod-harness` directly (bypassing `run.sh`, which always starts a working `mockd`) against a `--sock` path with nothing listening | 75 s of `Connecting`/`Reconnecting`, then `FAIL: dod_harness watchdog fired — hung past 75s`, exit 3 |

-`gui-dod-nightly` needs `if: github.event_name == 'schedule'` (the `nightly-integration`
-job below already added that trigger to `ci.yml` — `cron: '17 3 * * *'` — so this no
-longer needs its own).
+The real (non-forced) runs all pass live: `scroll-60fps` p99 51.28 ms against a 66.4 ms
+budget (ASan/UBSan build), `rss-flat` growth 13504 KiB against the 20480 KiB slack over a
+15 s smoke duration, `unhappy-path` all three phases `PASS` with `finalConnected=true`.
+
+### Known gap: `unhappy-path`'s drop-connection phase doesn't drop anything
+
+Filed by GUI in `gui/docs/proto-requests-m1.md`: `mockd --drop-connection` only works
+over WebSocket — `startUds()` never wires the periodic-drop timer `startWs()` has, so the
+UDS transport (GUI/CLI/nmhost's only one) never sees a connection actually die. GUI's own
+live verification: 45 s observing a `--drop-connection 5` mockd over UDS, `stateChanged`
+never fires. The phase still runs and its JSON honestly records `sawDisruption: false`
+rather than silently passing as if it proved something — that's what the real run above
+shows, `pass: true` alongside `sawDisruption: false`, so a reviewer reading the JSON sees
+exactly how much this phase currently covers.
+
+This is PROTO's fix, not PKG/QA's or GUI's to route around locally — coordinating on it
+rather than patching mockd from this lane. Once PROTO lands `dropEverySec` on `startUds`,
+`unhappy-path`'s drop-connection phase starts exercising a real drop and this note comes
+out; until then `gui-dod` stays required as specified (crash/hang/never-reconnects still
+catch real regressions), just not yet catching a swallowed real disconnect.

 ### Status

-Blocked on GUI's harness. Not urgent (GUI M1 DoD, not M0). When GUI files the follow-up
-with the real `run.sh` path and the `rss-flat` slack number, PKG/QA drops the `TODO(GUI)`
-markers and marks `gui-dod` required. The `schedule:` trigger `gui-dod-nightly` needs is
-already in `ci.yml`.
+Wired and required: `gui-dod` runs per-PR, `gui-dod-nightly` on schedule. Revisit once
+PROTO's UDS `--drop-connection` fix lands (see above).