pkg: make the conformance CI check run and be able to fail
The conformance job was a required check that ran nothing. Two bugs:
1. Its guard tested for tests/conformance/CMakeLists.txt or package.json.
The suite ships as tests/conformance/run.sh; neither file exists, so the
guard was always false and the job took the "skipped" (success) branch.
2. Even forced true, it ran `ctest --preset dev -L conformance` — no test
carries that label, so ctest reported "Total Tests: 0" and exited 0.
Replace the job body with PROTO's intended wiring from
tests/conformance/README.md: bootstrap the toolchain, pin Node 22 (apt ships
< 20; the TS replay runner needs >= 20), and run ./tests/conformance/run.sh
directly. The suite starts its own mockd and builds its own C++ runner, so no
cmake configure is needed. Verified it goes red: an enum-invalid fixture makes
run.sh exit 1; reverting it returns to green.
check_contract.py imports jsonschema and referencing, which bootstrap.sh did
not install. Add python3-jsonschema / python3-referencing to the apt set and
to --check, so one command still provisions the whole suite.
Guards now fail loudly instead of passing quietly:
- conformance has no skip branch any more. run.sh has landed; the job runs
it unconditionally and errors if the entrypoint is missing.
- extension-lint keyed "has EXT landed?" to extension/package.json — the
same single-filename trap. Key it to a manifest instead, and once a
manifest exists, treat a missing package.json as a hard failure rather
than a green skip.
Mark conformance required now in BRANCH_PROTECTION.md — it is the M0 exit gate.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0143aKiohmDiyefJBwHDJJqw
This commit is contained in:
Claude Sonnet 5
@@ -17,7 +17,7 @@ policy so it can be re-applied or audited.
|
||||
| `bootstrap-script` | now |
|
||||
| `build (gcc)` / `build (clang)` | when the first C++ lane merges |
|
||||
| `sanitizers (dev)` / `sanitizers (tsan)` | when the first C++ lane merges |
|
||||
| `conformance` | **when `tests/conformance/` lands — this is the M0 exit gate** |
|
||||
| `conformance` | **now — `tests/conformance/` has landed; this is the M0 exit gate** |
|
||||
| `extension-lint` | when `extension/` lands |
|
||||
|
||||
`clang-tidy` is intentionally **not** required through M1 (`continue-on-error: true`,
|
||||
@@ -30,7 +30,14 @@ policy so it can be re-applied or audited.
|
||||
|
||||
## Note on the "skipped" job steps
|
||||
|
||||
Several jobs (`conformance`, `extension-lint`, `clang-tidy`) short-circuit to a "skipped"
|
||||
echo when their lane hasn't landed. They still report **success**, so they can be marked
|
||||
required now without blocking — they start doing real work automatically on the commit
|
||||
that adds the lane.
|
||||
`extension-lint` and `clang-tidy` short-circuit to a "skipped" echo when their lane
|
||||
hasn't landed. They still report **success**, so they can be marked required now without
|
||||
blocking — they start doing real work automatically on the commit that adds the lane.
|
||||
|
||||
Their guards fail **loudly** (non-zero) once the lane is half-present — e.g. an
|
||||
`extension/manifest.json` with no lintable `package.json`. A guard keyed to a single
|
||||
filename is how a required check ends up green over nothing; the skip branch is only for
|
||||
a lane that is genuinely absent.
|
||||
|
||||
`conformance` no longer has a skip branch. The suite (`tests/conformance/run.sh`) has
|
||||
landed, so the job runs it unconditionally and fails if the entrypoint is missing.
|
||||
|
||||
+41
-20
@@ -53,22 +53,38 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- id: check
|
||||
# "Has EXT landed?" is answered by a manifest, not by extension/package.json:
|
||||
# a guard keyed to one filename passes vacuously the day EXT ships the lane
|
||||
# under any other name. Skip only when the lane genuinely is not here; once a
|
||||
# manifest exists, a missing lint entrypoint is a hard failure, not a skip.
|
||||
run: |
|
||||
if [ -f extension/package.json ]; then echo "present=true" >> "$GITHUB_OUTPUT"
|
||||
else echo "present=false" >> "$GITHUB_OUTPUT"; fi
|
||||
manifest=""
|
||||
for m in extension/manifest.json extension/src/manifest.json extension/public/manifest.json; do
|
||||
if [ -f "$m" ]; then manifest="$m"; break; fi
|
||||
done
|
||||
if [ -z "$manifest" ]; then
|
||||
echo "extension/ has not landed yet (no manifest.json) — skipping web-ext lint."
|
||||
echo "present=false" >> "$GITHUB_OUTPUT"
|
||||
exit 0
|
||||
fi
|
||||
echo "EXT has landed: $manifest"
|
||||
echo "present=true" >> "$GITHUB_OUTPUT"
|
||||
if [ ! -f extension/package.json ]; then
|
||||
echo "::error::$manifest exists but extension/package.json does not — this job" \
|
||||
"cannot lint the extension. Wire web-ext lint in here; do not let the check" \
|
||||
"pass green over an unlinted lane."
|
||||
exit 1
|
||||
fi
|
||||
- uses: actions/setup-node@v4
|
||||
if: steps.check.outputs.present == 'true'
|
||||
with:
|
||||
node-version: '20'
|
||||
node-version: '22'
|
||||
- name: web-ext lint
|
||||
if: steps.check.outputs.present == 'true'
|
||||
working-directory: extension
|
||||
run: |
|
||||
npm ci
|
||||
npx web-ext lint --source-dir .
|
||||
- name: skipped
|
||||
if: steps.check.outputs.present == 'false'
|
||||
run: echo "extension/ has not landed yet — skipping web-ext lint"
|
||||
|
||||
# --- build + test matrix ----------------------------------------------------------
|
||||
build:
|
||||
@@ -143,24 +159,29 @@ jobs:
|
||||
run: echo "no C++ lane has landed a CMakeLists yet — skipping clang-tidy"
|
||||
|
||||
conformance:
|
||||
# Required check on every PR once tests/conformance/ lands (branch protection is
|
||||
# configured in the repo settings, not here — see .github/BRANCH_PROTECTION.md).
|
||||
# The M0 exit gate. Proves the generated C++ daemon surface and the generated TS
|
||||
# extension surface agree with contracts/fixtures without either side having run
|
||||
# against the other. Required on every PR — branch protection is a repo setting,
|
||||
# recorded in .github/BRANCH_PROTECTION.md.
|
||||
#
|
||||
# The suite ships as tests/conformance/run.sh (its own mockd, its own C++ build).
|
||||
# There is no CMakeLists or package.json to guard on, and no ctest label to select;
|
||||
# the earlier guard for those was always false, so this job passed running nothing.
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- id: check
|
||||
- name: Guard — the suite must be present
|
||||
run: |
|
||||
if [ -f tests/conformance/CMakeLists.txt ] || [ -f tests/conformance/package.json ]; then
|
||||
echo "present=true" >> "$GITHUB_OUTPUT"
|
||||
else echo "present=false" >> "$GITHUB_OUTPUT"; fi
|
||||
if [ ! -x tests/conformance/run.sh ]; then
|
||||
echo "::error::tests/conformance/run.sh is missing or not executable. The" \
|
||||
"conformance suite is the M0 exit gate and this check must not pass" \
|
||||
"without running it. If PROTO moved the entrypoint, update this job."
|
||||
exit 1
|
||||
fi
|
||||
- name: Bootstrap toolchain
|
||||
if: steps.check.outputs.present == 'true'
|
||||
run: sudo ./tools/bootstrap.sh
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22' # apt ships < 20; run.sh's TS runner needs >= 20
|
||||
- name: Run conformance suite
|
||||
if: steps.check.outputs.present == 'true'
|
||||
run: |
|
||||
cmake --preset dev
|
||||
ctest --preset dev --output-on-failure -L conformance
|
||||
- name: skipped
|
||||
if: steps.check.outputs.present == 'false'
|
||||
run: echo "tests/conformance/ has not landed yet — skipping"
|
||||
run: ./tests/conformance/run.sh
|
||||
|
||||
@@ -97,6 +97,7 @@ Surveyed on this machine 2026-09-09 — **most of it is already installed**:
|
||||
| libcurl4-openssl-dev · libsqlite3-dev · nlohmann-json3-dev · libssl-dev | ✓ |
|
||||
| libavformat-dev · libavcodec-dev · ffmpeg | ✓ |
|
||||
| clang-format · clang-tidy · python3 · pkg-config | ✓ |
|
||||
| python3-jsonschema · python3-referencing (conformance static runner) | ✓ |
|
||||
|
||||
**Only these four are missing:**
|
||||
|
||||
|
||||
+19
-1
@@ -97,6 +97,11 @@ APT_NODE=(
|
||||
npm
|
||||
)
|
||||
|
||||
APT_PYTHON=(
|
||||
python3-jsonschema # tests/conformance: check_contract.py schema validation
|
||||
python3-referencing # tests/conformance: $ref registry for draft 2020-12
|
||||
)
|
||||
|
||||
APT_CLANG=(
|
||||
clang # optional: libFuzzer targets (M7)
|
||||
llvm
|
||||
@@ -112,7 +117,7 @@ APT_PACKAGING=(
|
||||
|
||||
# --- install --------------------------------------------------------------------
|
||||
|
||||
PKGS=("${APT_CORE[@]}" "${APT_LIBS[@]}" "${APT_GUI[@]}" "${APT_LINT[@]}" "${APT_NODE[@]}")
|
||||
PKGS=("${APT_CORE[@]}" "${APT_LIBS[@]}" "${APT_GUI[@]}" "${APT_LINT[@]}" "${APT_NODE[@]}" "${APT_PYTHON[@]}")
|
||||
[[ $WITH_CLANG -eq 1 ]] && PKGS+=("${APT_CLANG[@]}")
|
||||
[[ $WITH_PACKAGING -eq 1 ]] && PKGS+=("${APT_PACKAGING[@]}")
|
||||
|
||||
@@ -191,6 +196,19 @@ if command -v node >/dev/null 2>&1; then
|
||||
fi
|
||||
fi
|
||||
|
||||
# Python modules the conformance suite's static runner imports directly.
|
||||
if command -v python3 >/dev/null 2>&1; then
|
||||
for mod in jsonschema referencing; do
|
||||
if python3 -c "import $mod" 2>/dev/null; then
|
||||
printf ' \033[32m✓\033[0m %-16s %s\n' "py:$mod" \
|
||||
"$(python3 -c "import importlib.metadata as m; print(m.version('$mod'))" 2>/dev/null)"
|
||||
else
|
||||
printf ' \033[31m✗\033[0m %-16s python3 can'\''t import it\n' "py:$mod"
|
||||
fail=1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# Qt / library dev packages: check via pkg-config where possible.
|
||||
for mod in Qt6Core Qt6Widgets Qt6Svg libcurl sqlite3 libssl libsecret-1 libavformat; do
|
||||
if pkg-config --exists "$mod" 2>/dev/null; then
|
||||
|
||||
Reference in New Issue
Block a user