pkg: make the conformance CI check run and be able to fail
The conformance job was a required check that ran nothing. Two bugs:
1. Its guard tested for tests/conformance/CMakeLists.txt or package.json.
The suite ships as tests/conformance/run.sh; neither file exists, so the
guard was always false and the job took the "skipped" (success) branch.
2. Even forced true, it ran `ctest --preset dev -L conformance` — no test
carries that label, so ctest reported "Total Tests: 0" and exited 0.
Replace the job body with PROTO's intended wiring from
tests/conformance/README.md: bootstrap the toolchain, pin Node 22 (apt ships
< 20; the TS replay runner needs >= 20), and run ./tests/conformance/run.sh
directly. The suite starts its own mockd and builds its own C++ runner, so no
cmake configure is needed. Verified it goes red: an enum-invalid fixture makes
run.sh exit 1; reverting it returns to green.
check_contract.py imports jsonschema and referencing, which bootstrap.sh did
not install. Add python3-jsonschema / python3-referencing to the apt set and
to --check, so one command still provisions the whole suite.
Guards now fail loudly instead of passing quietly:
- conformance has no skip branch any more. run.sh has landed; the job runs
it unconditionally and errors if the entrypoint is missing.
- extension-lint keyed "has EXT landed?" to extension/package.json — the
same single-filename trap. Key it to a manifest instead, and once a
manifest exists, treat a missing package.json as a hard failure rather
than a green skip.
Mark conformance required now in BRANCH_PROTECTION.md — it is the M0 exit gate.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0143aKiohmDiyefJBwHDJJqw
This commit is contained in:
@@ -17,7 +17,7 @@ policy so it can be re-applied or audited.
|
||||
| `bootstrap-script` | now |
|
||||
| `build (gcc)` / `build (clang)` | when the first C++ lane merges |
|
||||
| `sanitizers (dev)` / `sanitizers (tsan)` | when the first C++ lane merges |
|
||||
| `conformance` | **when `tests/conformance/` lands — this is the M0 exit gate** |
|
||||
| `conformance` | **now — `tests/conformance/` has landed; this is the M0 exit gate** |
|
||||
| `extension-lint` | when `extension/` lands |
|
||||
|
||||
`clang-tidy` is intentionally **not** required through M1 (`continue-on-error: true`,
|
||||
@@ -30,7 +30,14 @@ policy so it can be re-applied or audited.
|
||||
|
||||
## Note on the "skipped" job steps
|
||||
|
||||
Several jobs (`conformance`, `extension-lint`, `clang-tidy`) short-circuit to a "skipped"
|
||||
echo when their lane hasn't landed. They still report **success**, so they can be marked
|
||||
required now without blocking — they start doing real work automatically on the commit
|
||||
that adds the lane.
|
||||
`extension-lint` and `clang-tidy` short-circuit to a "skipped" echo when their lane
|
||||
hasn't landed. They still report **success**, so they can be marked required now without
|
||||
blocking — they start doing real work automatically on the commit that adds the lane.
|
||||
|
||||
Their guards fail **loudly** (non-zero) once the lane is half-present — e.g. an
|
||||
`extension/manifest.json` with no lintable `package.json`. A guard keyed to a single
|
||||
filename is how a required check ends up green over nothing; the skip branch is only for
|
||||
a lane that is genuinely absent.
|
||||
|
||||
`conformance` no longer has a skip branch. The suite (`tests/conformance/run.sh`) has
|
||||
landed, so the job runs it unconditionally and fails if the entrypoint is missing.
|
||||
|
||||
Reference in New Issue
Block a user