samiandClaude Opus 5
53421d6cb8
Schemas for the whole v1 surface: 38 methods, 9 events, 25 named types and the
JSON-RPC envelope, with x-privileged / x-transports / x-deadlineMs / x-errors
annotations that both generators emit as data rather than prose.
Four generators over one IR (contracts/codegen/schema_ir.py), so the C++ structs,
the TypeScript types and the OpenRPC document cannot disagree about what the
contract says:
gen_cpp.py -> core/generated/velox_proto.{hpp,cpp}
gen_ts.py -> extension/src/shared/protocol/
gen_openrpc.py -> contracts/openrpc.json
gen_cpp_conformance.py -> tests/conformance/cpp/fixture_dispatcher.hpp
Inbound parsing never throws: parse<T>() returns std::expected<T, ParseError> and
nlohmann's throwing ADL from_json is deliberately not emitted. Schema constraints
(minimum, maxLength, pattern, ...) become real runtime checks in both languages —
the daemon does not trust the extension and the extension does not trust the
daemon.
59 golden fixtures: a success case per method, 12 error cases, 9 events. Replayed
by tests/conformance/ against both the generated C++ and a live server over both
transports. tools/mockd serves the same fixtures with unhappy-path flags so the
GUI and EXT lanes never wait for veloxd.
run.sh also proves capture.offer fails open: with a daemon answering slower than
750 ms the client gives up and lets Firefox take the download.
core/generated/ is libveloxproto, a separate target from libveloxcore, which
still never sees JSON — see docs/adr/0009.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012fgjnqFCS5h5L7gZTZo3rV
53 lines
2.6 KiB
Markdown
53 lines
2.6 KiB
Markdown
# tests/conformance — one suite, three runners
|
|
|
|
**This is a required check on every lane's PR.** It is the mechanism that makes four
|
|
parallel lanes safe: the C++ daemon and the TypeScript extension are proved compatible
|
|
without either having run against the other.
|
|
|
|
```sh
|
|
./tests/conformance/run.sh # starts its own mockd
|
|
./tests/conformance/run.sh --uds /run/user/1000/velox/velox.sock --ws-port 52000
|
|
```
|
|
|
|
## The runners
|
|
|
|
| Runner | Needs | Asserts |
|
|
|---|---|---|
|
|
| `check_contract.py` | python3, jsonschema | schemas parse and resolve; the documented surface matches the schema surface both ways; every method has a success fixture; every fixture validates; `SettingKey` and `Settings` agree; **committed generated code is not stale** |
|
|
| `cpp/` | a C++23 compiler, nlohmann | every golden payload parses into the generated structs, serialises back stably, and goes through the real `dispatch()`; privileged methods are refused `-32003` over the WebSocket |
|
|
| `ts/replay.ts` | node ≥ 20 | a live server answers every fixture over every transport the contract allows, and the reply passes the generated validator |
|
|
|
|
`run.sh` also runs one scenario that cannot be shown against a healthy server: with the
|
|
daemon answering slower than `capture.offer`'s 750 ms deadline, the client must give up and
|
|
let Firefox take the download. **That is the fail-open guarantee, and it is checked here.**
|
|
|
|
## What "passing" means
|
|
|
|
The runners check the contract, not the implementation's opinions. Results are compared by
|
|
shape and validated against the generated validators; error codes are compared exactly.
|
|
Byte-equality with a golden file is deliberately *not* asserted, because a live daemon
|
|
returns its own ids and its own clock — see `contracts/fixtures/README.md`.
|
|
|
|
Adding a method without a fixture fails `check_contract.py`. Regenerating and forgetting to
|
|
commit the output fails it too.
|
|
|
|
## Request to lane PKG/QA
|
|
|
|
`.github/` belongs to PKG/QA, so this suite is not wired into CI by lane PROTO. Please add
|
|
it as a **required status check on every branch**, roughly:
|
|
|
|
```yaml
|
|
conformance:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: actions/setup-node@v4
|
|
with: { node-version: '22' }
|
|
- run: sudo apt-get update && sudo apt-get install -y nlohmann-json3-dev
|
|
- run: pip install jsonschema referencing
|
|
- run: ./tests/conformance/run.sh
|
|
```
|
|
|
|
The suite needs: `python3` with `jsonschema`, a C++23 compiler, `nlohmann-json`, and Node
|
|
≥ 20. It starts and stops its own `mockd`; nothing else needs to be running.
|