proto: wire veloxd into the conformance suite's canonical entry point
run.sh's TS runner only ever started mockd; veloxd existed but nothing in
the ctest -L conformance path touched it, so mockd's always-valid fixtures
were the only thing ts/replay.ts ever saw. That let a real bug through:
veloxd's download.get can return segments: 0, which TaskSummary.segments
forbids (minimum 1, required) — nothing caught it.
Add a 3b step to run.sh (still the one canonical entry, per ADR 0014):
builds veloxd, starts it isolated (its own XDG_RUNTIME_DIR/XDG_DATA_HOME/
XDG_CONFIG_HOME), seeds saveTo.allowedRoots/defaultDir directly into the
isolated velox.db (settings.set is itself a stub, and the default
~/Downloads root doesn't isolate download.add's writes), then replays
every fixture against it over both transports.
Most handlers are still stubs (daemon/docs/deferrals.md D1-D4b). Fixtures
that hit them get an expected-failure entry in the new veloxd-xfail.json,
loaded by replay.ts's new --xfail flag. This is a maintained allowlist,
not a snapshot: a listed fixture that unexpectedly *passes* is flipped
back to a failure (applyXfail), so the list can only shrink as DAEMON
lands handlers, never rot into a list nobody rechecks. download.get's
segments: 0 is deliberately *not* on it — that's the regression this
step exists to catch.
Also hardened setupBindings: a server that can't even complete fixture
binding used to take the whole runner down with an uncaught exception
before a single fixture was checked. It's now a reported Outcome instead,
so the run still produces a coherent report. That robustness fix earned
its keep immediately: veloxd's download.add crashes on startMode
"later" (a valid, documented StartMode — "the File Info dialog's
Download Later button") with a SQLite CHECK constraint violation, because
migrations/0001_initial.sql's start_mode CHECK never had 'later' added to
it (and includes 'manual'/'auto', neither a contract value). That's a
second, more severe bug this wiring found, unrelated to segments: 0 and
currently blocking most of the veloxd run — filed for DAEMON in
tests/conformance/README.md, not fixed here (out of lane). capture.offer
and capture.getRules are also stubs but missing from deferrals.md's
D-list; xfailed with a note asking DAEMON to add the row.
Verified live once against a real, isolated veloxd before this session's
sandbox became persistently contended for veloxd's single-instance lock
(UID-scoped, not namespaced by XDG_RUNTIME_DIR — daemon/src/main.cpp;
documented as a caveat in the README): it built, started isolated, seeded
settings, connected over both transports, and surfaced the startMode bug
above as a real, non-xfailed failure — confirming the whole pipeline
including --xfail end to end. segments: 0 is confirmed by direct reading
of daemon/src/store/tasks.{hpp,cpp} (TaskRow::eff_segments defaults to 0,
copied verbatim into TaskSummary.segments) rather than by a second live
run reaching that specific fixture, since setup itself fails first on the
startMode bug above. mockd path re-verified green after these changes
(200/200, up from 196/196 — the new setup/$taskId outcomes are visible
and passing).
Recommendation for PKG: don't flip this required yet. The existing
`conformance` ctest entry is already a required check, and right now the
startMode bug fails most of the veloxd run, not just the one expected
segments: 0 case — merging as-is would block every lane's PRs on two
DAEMON bugs at once, one of them unrelated to what this task set out to
catch. Required once DAEMON lands a fix for startMode "later" at minimum;
segments: 0 can stay red for a while by design, same as any other tracked
regression.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01SFeUKLbdHizrJjLBeK7ffz
This commit is contained in:
Claude Sonnet 5
@@ -2,13 +2,19 @@
|
||||
#
|
||||
# The conformance suite. This is the command CI runs on every lane's PR.
|
||||
#
|
||||
# ./tests/conformance/run.sh static + C++ + TS against a mockd it starts
|
||||
# ./tests/conformance/run.sh static + C++ + TS against mockd, then veloxd
|
||||
# ./tests/conformance/run.sh --uds PATH --ws-port N against an already-running daemon
|
||||
#
|
||||
# Three runners, one set of fixtures:
|
||||
# Four runners, one set of fixtures:
|
||||
# 1. check_contract.py schemas, fixtures and committed generated code agree
|
||||
# 2. cpp/ the generated C++ parses, serialises and dispatches every fixture
|
||||
# 3. ts/replay.ts a live server answers every fixture over both transports
|
||||
# 2. cpp/ the generated C++ parses, serialises and dispatches every fixture
|
||||
# 3. ts/replay.ts against mockd — mockd always answers every fixture correctly, so
|
||||
# this is the TS client and the fixtures agreeing with each other
|
||||
# 3b. ts/replay.ts against a real, isolated veloxd it builds and starts — the one
|
||||
# runner that can catch veloxd disagreeing with its own contract.
|
||||
# Fixtures that hit a still-stubbed handler (daemon/docs/deferrals.md
|
||||
# D1-D4b) are excused via veloxd-xfail.json; everything else must
|
||||
# pass for real.
|
||||
#
|
||||
# Plus one scenario that cannot be shown against a healthy server: with the daemon
|
||||
# answering slower than capture.offer's 750 ms deadline, the client must give up and let
|
||||
@@ -23,6 +29,7 @@ EXTERNAL_UDS=""
|
||||
EXTERNAL_WS=""
|
||||
MOCKD_PID=""
|
||||
SLOW_PID=""
|
||||
VELOXD_PID=""
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
@@ -46,6 +53,7 @@ stop() {
|
||||
cleanup() {
|
||||
stop "$MOCKD_PID"
|
||||
stop "$SLOW_PID"
|
||||
stop "$VELOXD_PID"
|
||||
rm -rf "$WORK"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
@@ -99,6 +107,78 @@ TS_ARGS=()
|
||||
[ -n "$WS_PORT" ] && TS_ARGS+=(--ws-port "$WS_PORT")
|
||||
( cd "$HERE/ts" && ./node_modules/.bin/tsx replay.ts "${TS_ARGS[@]}" )
|
||||
|
||||
# ------------------------------------------------------------------ 3b. veloxd
|
||||
# The same fixtures against a real, isolated veloxd. mockd (above) always answers every
|
||||
# fixture correctly by construction, so it can only prove the TS client and the fixtures
|
||||
# agree with each other — it cannot catch veloxd disagreeing with its own contract. This
|
||||
# is the runner that closed that gap: it is what would have caught veloxd's download.get
|
||||
# returning `segments: 0`, which TaskSummary forbids (minimum 1, required), before it
|
||||
# shipped rather than after.
|
||||
step "generated TypeScript against a real, isolated veloxd"
|
||||
if [ -z "$EXTERNAL_UDS" ] && [ -z "$EXTERNAL_WS" ]; then
|
||||
if [ ! -f "$REPO/daemon/CMakeLists.txt" ]; then
|
||||
echo "skipped: daemon/CMakeLists.txt not present (lane DAEMON has not landed yet)"
|
||||
else
|
||||
VBUILD="$REPO/build/dev"
|
||||
# cmake --preset dev is idempotent to re-run against an existing build dir; a CI leg
|
||||
# that already configured (the `conformance` job does, before ctest) just reuses it.
|
||||
if [ ! -f "$VBUILD/CMakeCache.txt" ]; then
|
||||
( cd "$REPO" && cmake --preset dev ) >"$WORK/veloxd-configure.log" 2>&1 \
|
||||
|| { echo "veloxd: cmake configure failed:"; cat "$WORK/veloxd-configure.log"; exit 1; }
|
||||
fi
|
||||
cmake --build "$VBUILD" --target veloxd >"$WORK/veloxd-build.log" 2>&1 \
|
||||
|| { echo "veloxd: build failed:"; cat "$WORK/veloxd-build.log"; exit 1; }
|
||||
VELOXD_BIN="$VBUILD/bin/veloxd"
|
||||
|
||||
# Isolated: its own runtime dir (socket, ws.port, single-instance lock), data dir
|
||||
# (velox.db) and config dir, none of them the real user's. veloxd's single-instance
|
||||
# lock is a UID-scoped abstract socket, not namespaced by XDG_RUNTIME_DIR, so this
|
||||
# still collides with a veloxd already running for this user outside the sandbox —
|
||||
# that shows up below as "another instance is already running" and fails loudly
|
||||
# rather than silently testing the wrong daemon.
|
||||
VXDG="$WORK/veloxd-xdg"
|
||||
mkdir -p "$VXDG/runtime" "$VXDG/data" "$VXDG/config" "$VXDG/downloads"
|
||||
|
||||
XDG_RUNTIME_DIR="$VXDG/runtime" XDG_DATA_HOME="$VXDG/data" XDG_CONFIG_HOME="$VXDG/config" \
|
||||
"$VELOXD_BIN" >"$WORK/veloxd.log" 2>&1 &
|
||||
VELOXD_PID=$!
|
||||
VUDS="$VXDG/runtime/velox/velox.sock"
|
||||
for _ in $(seq 1 50); do [ -S "$VUDS" ] && break; sleep 0.2; done
|
||||
[ -S "$VUDS" ] || { echo "veloxd did not start:"; cat "$WORK/veloxd.log"; exit 1; }
|
||||
|
||||
# saveTo.allowedRoots defaults to ["~/Downloads"]; download.add.json (fixture) asks
|
||||
# for a saveDir under $HOME/Downloads, so both that and download.add's own isolated
|
||||
# downloads dir need to be allowed roots, or every download.add fixture fails -32011
|
||||
# before the point of this runner is even reached. settings.set is itself a D3 stub,
|
||||
# so this is written straight into the isolated velox.db rather than over the wire.
|
||||
python3 - "$VXDG/data/velox/velox.db" "$VXDG/downloads" "$HOME/Downloads" <<'PY'
|
||||
import json, sqlite3, sys
|
||||
db_path, isolated_downloads, home_downloads = sys.argv[1:4]
|
||||
db = sqlite3.connect(db_path)
|
||||
db.execute(
|
||||
"INSERT INTO settings(key, value) VALUES(?, ?) "
|
||||
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
|
||||
("saveTo.allowedRoots", json.dumps([isolated_downloads, home_downloads])),
|
||||
)
|
||||
db.execute(
|
||||
"INSERT INTO settings(key, value) VALUES(?, ?) "
|
||||
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
|
||||
("saveTo.defaultDir", json.dumps(isolated_downloads)),
|
||||
)
|
||||
db.commit()
|
||||
PY
|
||||
|
||||
VELOXD_TS_ARGS=(--uds "$VUDS")
|
||||
if [ -f "$VXDG/runtime/velox/ws.port" ]; then
|
||||
VELOXD_TS_ARGS+=(--ws-port "$(cat "$VXDG/runtime/velox/ws.port")")
|
||||
fi
|
||||
( cd "$HERE/ts" && ./node_modules/.bin/tsx replay.ts "${VELOXD_TS_ARGS[@]}" \
|
||||
--xfail "$HERE/veloxd-xfail.json" )
|
||||
fi
|
||||
else
|
||||
echo "skipped: --uds/--ws-port already points at a live daemon"
|
||||
fi
|
||||
|
||||
# ------------------------------------------------- 4. capture fails open
|
||||
step "capture.offer fails open when the daemon is too slow"
|
||||
if [ -z "$EXTERNAL_UDS" ]; then
|
||||
|
||||
Reference in New Issue
Block a user