Files
samiandClaude Sonnet 5 4e177ec809 proto: stop conformance from downloading real files, ADR the null-clearing gap
1. download.add.json had startMode "now" against a real, large (~6 GB)
   Ubuntu ISO with saveDir hardcoded to /home/sami/Downloads/Programs.
   Against a real veloxd (tests/conformance/run.sh) that's a real
   download into the real user's real home, every single run — it had
   already happened twice. startMode -> "later" (exercises the add path,
   hands nothing to the engine) and saveDir is dropped entirely (resolves
   to saveTo.defaultDir instead, checked against allowedRoots the same
   way). Documented the rule this fixture was breaking in
   contracts/fixtures/README.md so it doesn't happen a third time.

   Auditing the rest for the same shape (now real: capture.offer, D7)
   found a second, subtler instance: capture.offer.take.json's "take"
   admits a real, immediately-started task the same way download.add
   does, and the "Programs" category's saveDir is a migration-seeded
   builtin (~/Downloads/Programs) that no isolated test setup can
   redirect -- so even after pointing the URL at example.org (RFC 2606),
   a real ~6 GB sparse .veloxpart still landed in the real home on the
   declared Content-Length alone. Shrunk to a plausible-but-small 5 MiB.
   Also scoped to "transport": "uds" -- a real "take" persists an active
   task, so replaying the same fixture again on the second live transport
   against the same shared daemon was hitting capture.offer's own
   dedupe-by-URL and failing on a missing taskId, not a bug.
   download.add's other real-URL siblings (errors/*.invalid-path,
   *.invalid-params, *.disk-full) all fail before admission or are
   requires-gated; left alone.

2. ADR 0018: DAEMON can set a nullable field through download.update /
   settings.set but never clear it back to null, because the generated
   C++ parser collapses "absent" and "explicit null" to the same
   std::nullopt for every optional field (contracts/codegen/gen_cpp.py,
   on purpose, and correct for create-style params -- just wrong for
   patch-style ones, which is the only place the schema documents
   "explicit null clears"). Decision: an opt-in x-clearable schema
   annotation makes just those fields std::optional<std::optional<T>> in
   C++ (TS already round-trips this natively); not a blanket rule
   (would retype response fields like TaskSummary.effectiveUrl that have
   no clear-vs-absent distinction to make), not an explicit clear-list
   field (would redesign a wire contract DAEMON already built against
   just to route around a generator gap). Recorded, not implemented here
   -- that's its own PROTO PR (schema annotations + gen_cpp.py + gen_ts.py
   + regeneration + a minor VERSION bump per ADR 0015), not bundled into
   a fixture-safety pass. Left a pointer to the ADR at the generator
   comment it concerns.

3. Re-verified every xfail entry against current deferrals.md rather
   than trust the reasons already on file: D7/D8 (capture.offer/
   getRules), D3d/e/f/g/h/i (rules, queue.reorder, schedule, limiter,
   download.update/refreshUrl) and D9 (settings) have all closed since
   the list was last pruned, so most of it was stale. Removed everything
   that now cleanly passes; kept and re-reasoned everything that doesn't:
   - errors/download.provideAuth.not-found.json stays, as asked: real
     bug, on_download_provideAuth never checks the task exists.
   - category.list.json (mimeTypes -- documented D3a gap), schedule.set.json
     (nextRunAt -- documented D3f gap): unchanged in substance, reason
     text was already accurate.
   - download.probe/get/list/update.json, session.hello.json,
     queue.start/reorder.json, category.remove.json: not bugs -- each
     golden depicts a richer lifecycle/config state (a probed download,
     real queue or category membership, media/grabber capabilities) than
     this harness's fresh, never-started bound tasks and empty isolated
     DB can produce.
   - limiter.get.json: real fixture bug, not a daemon one -- applyToRunning
     is a write-only instruction on limiter.set, on_limiter_get never
     returns it; the golden shouldn't have had it either. Fixed the
     fixture and tools/mockd's own limiter.get, which had the same field
     hardcoded into its in-memory state independent of the fixture file.
   - grabber.*/media.*: still genuinely stub (M4 territory).
   Only remaining unexpected-pass surfaced while re-verifying
   (errors/capture.offer.ignore.json, always "take" instead of "ignore")
   traced to capture.minSizeBytes defaulting to 0 on a fresh daemon,
   making its below-minimum-size scenario unreachable -- not a bug, so
   raised the setting in run.sh's isolated seeding instead of xfailing it.

ctest -L conformance: green, 100% (2/2), ~87s.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01SFeUKLbdHizrJjLBeK7ffz
2026-09-12 16:56:35 +04:00
..

contracts/ — the wire contract

This directory is the interface between every lane. Owner: agent PROTO. Nobody else commits here. Everybody else generates from here.

Status: v1.4.0 (frozen at v1.0.0 on 2026-09-09; minor bumps since)

v1.0.0 froze 38 methods, 9 events, 26 named types. v1.1.0 widened bufferBytes bounds and added the segment-budget settings. v1.2.0 added download.provideAuth (F2). v1.3.0 widened when error is populated on a state change to cover a daemon-initiated paused (for docs/adr/0013-...). v1.4.0 (current) is a C++-binding-only change: the generated Dispatcher gains a HandlerError / HandlerResult<T> error channel so a handler can return -32010 / -32011 / -32013 with their data payloads instead of collapsing to -32603. The wire is byte-identical — no schema or fixture change — but any Dispatcher implementer must swap ResultHandlerResult on regen. Answered in contracts/proto-answers-daemon-m1.md. See also docs/adr/0005-... for the versioning rule and docs/adr/0010-... for the failure taxonomy and segment ranges.

Lane requests are answered in writing: contracts/proto-answers-m1.md responds to core/docs/proto-requests-m1.md point by point.

What each lane can rely on, starting now:

You need It is here
C++ types, parsing, dispatch core/generated/velox_proto.{hpp,cpp} (target libveloxproto)
TypeScript types, typed client, runtime validators extension/src/shared/protocol/
The API document to read contracts/openrpc.json
A daemon to build against today tools/mockd — both transports, 4 Hz progress, unhappy-path flags
Proof you have not drifted ./tests/conformance/run.sh

Changing this is a PR to contracts/ alone. Optional field or new method → minor. Rename, remove or retype → major, plus an ADR. File a request; do not add a field locally.

contracts/
├── VERSION                     # protocol semver — v1.4.0, minor-bumped from the v1.0.0 freeze
├── openrpc.json                # human-readable API doc (generated from schema/)
├── schema/
│   ├── envelope.schema.json    # JSON-RPC 2.0 envelope + our error codes
│   ├── types/                  # Task, Segment, Category, Queue, Settings, CaptureRules…
│   ├── methods/                # one file per method: params + result
│   └── events/                 # one file per server→client notification
├── fixtures/                   # golden request/response pairs, replayed by conformance
└── codegen/
    ├── schema_ir.py            # the one loader/IR all generators share
    ├── gen_cpp.py              # → core/generated/  (structs + to_json + parse + dispatch)
    ├── gen_ts.py               # → extension/src/shared/protocol/ (types, client, validators)
    ├── gen_openrpc.py          # → contracts/openrpc.json
    └── gen_cpp_conformance.py  # → tests/conformance/cpp/fixture_dispatcher.hpp

Each subdirectory has its own README: codegen/ documents the supported JSON Schema subset, fixtures/ documents the fixture shape and the placeholder rules.

Rules

  1. Generated code is committed. No lane may be blocked because it can't run Python.
  2. Hand-editing generated files is a merge blocker. Fix the schema and regenerate.
  3. Every method needs at least one fixture — a success case and, where meaningful, an error case. A method with no fixture is not done.
  4. Versioning: adding an optional field or a new method → minor bump. Removing, renaming, retyping, or changing a default → major bump and a written migration note in docs/adr/. session.hello rejects a major mismatch with error -32001 and a message the GUI renders as "Velox needs updating". "Retype → major" is about the wire — a field a client parses off the socket. A change that leaves the wire byte-identical but breaks a generated binding's source API (a C++ virtual's return type, a struct name) is a minor bump plus a migration note — see docs/adr/0015-generated-binding-changes-and-versioning.md.
  5. Changes arrive as a PR to contracts/ alone, containing: schema edit + fixtures + regenerated code + VERSION bump. Lanes rebase onto it. This is the only synchronization point in the whole project — keep it cheap and frequent rather than big and rare.

Per-method annotations

Every method schema carries these, and both generators emit them as data the code can act on rather than as prose a reader has to honour:

Key Meaning
x-privileged refused over the WebSocket transport with -32003
x-transports which listeners serve it (uds, ws)
x-deadlineMs how long a client waits before giving up
x-errors the error codes this method is documented to return
x-wsRestrictions extra limits when the call arrives from the extension

20 of the 39 methods are privileged: everything that reconfigures the daemon, destroys user data, or names an arbitrary destination path. The extension may request a download; it may not choose where the bytes land.

Transport framing

Client Transport Framing
GUI, CLI $XDG_RUNTIME_DIR/velox/velox.sock newline-delimited JSON (NDJSON)
nmhost ← Firefox stdio 4-byte little-endian length prefix (Firefox's format)
nmhost → daemon same Unix socket NDJSON
Extension (fallback) ws://127.0.0.1:520xx one JSON message per WS text frame

All four carry the same JSON-RPC 2.0 payloads. The framing differences stop at the transport layer; no method behaves differently depending on how it arrived — except that methods marked "privileged": true in the schema are refused over the WebSocket transport.

Method surface (v1.4.0 — expand only via PR)

Session

Method Params → Result
session.hello {clientType, clientName, protocolVersion, token?}{daemonVersion, protocolVersion, capabilities[], sessionId}
session.pair {clientName, extensionId}{token, expiresAt} (WS only; triggers user prompt)
session.subscribe {events[]}{ok}

Downloads

Method Params → Result
download.probe {url, headers?, cookies?, referrer?, userAgent?}{filename, sizeBytes?, mime, resumable, effectiveUrl, suggestedCategoryId}
download.add {url, headers?, cookies?, referrer?, userAgent?, filename?, saveDir?, categoryId?, segments?, bufferBytes?, startMode:"now"|"later"|"queue", queueId?, description?, checksum?}{taskId, state}
download.addBatch {items[], defaults}{taskIds[]}
download.list {filter?, sort?, offset?, limit?}{total, items: TaskSummary[]}
download.get {taskId}TaskDetail (includes segments[])
download.start | .pause | .resume | .cancel {taskIds[]}{updated[]}
download.remove {taskIds[], deleteFile:bool}{removed[]}
download.update {taskId, patch:{filename?, saveDir?, categoryId?, queueId?, description?, segments?, bufferBytes?}}TaskSummary
download.refreshUrl {taskId, url, headers?}{ok} (IDM's "Refresh Download Address")
download.provideAuth {taskId, username, password, save?}{ok} — answers event.auth.required. UDS only; privileged. Credentials go to the Secret Service, never SQLite, never logs

Organisation

category.list · category.upsert · category.remove · queue.list · queue.upsert · queue.start · queue.stop · queue.reorder · rules.list · rules.upsert · schedule.get · schedule.set

Settings & limits

settings.get {keys?} · settings.set {values} · limiter.get · limiter.set {globalBps?, enabled}

Browser integration

Method Notes
capture.offer {url, method, headers, cookies, contentType?, contentLength?, contentDisposition?, tabUrl, filename?}{action:"take"|"ignore", taskId?, reason?}must answer within 750 ms; the extension gives up and lets Firefox handle it otherwise
capture.getRules Extension mirrors the daemon's monitored types so the two never disagree
media.listVariants {manifestUrl, headers}{variants:[{id,resolution,bitrate,codec,sizeEstimate}]}
media.addVariant {manifestUrl, variantId, ...addParams}{taskId}

Grabber

grabber.start {startUrl, depth, includePatterns[], excludePatterns[], fileTypes[]}{jobId}; grabber.status {jobId}; grabber.harvest {jobId, select[]}{taskIds[]}

Events (server → client notifications)

Event Payload
event.task.added / .removed {taskId, summary?}
event.task.state {taskId, state, error?}
event.task.progress Batched array, emitted at ≤4 Hz: [{taskId, downloaded, speedBps, etaSec, segments:[{i,completed,speedBps}]}]
event.speed.global {downBps, activeCount}
event.auth.required {taskId, host, realm, scheme}
event.notify {level, title, body, taskId?}
event.settings.changed {keys[]}
event.grabber.progress {jobId, found, crawled, done}

Two error spaces, and why they are not the same

This trips people up, so it is stated once, loudly:

ErrorCode TaskErrorCode
Says why a call failed why a download failed
Space JSON-RPC integers (-32xxx) strings ("server_file_changed")
Lives in the JSON-RPC envelope's error TaskError.code, on a task
Example -32602 — your params were malformed checksum_mismatch — the bytes arrived and were wrong

A download fails while every RPC involved succeeds. That is the normal case. Never put a -32xxx into a TaskError, and never invent a JSON-RPC code for a transfer failure.

TaskErrorCode's 27 values mirror vdm::Error in core/include/vdm/util/error.hpp by name, so DAEMON's projection from the engine taxonomy is lossless and a new engine failure that has no wire spelling is a visible hole rather than a silent collapse to internal.

Segment ranges are inclusive

Segment.startByte and Segment.endByte describe a closed range [startByte, endByte]: endByte is the last byte, not one past it, and the segment covers endByte - startByte + 1 bytes. The two fields are copied verbatim into Range: bytes=<startByte>-<endByte>, which RFC 9110 defines as inclusive, so there is no arithmetic between the wire and the socket and nowhere for an off-by-one to hide. Conformance enforces contiguity and full coverage; a fixture written half-open fails.

Requested is not effective

DownloadSpec.segments is what a client asked for. TaskSummary.segments is what is in use right now, after the per-host cap and after the demotion to 1 for a non-resumable source. They are routinely different and the GUI must render the effective one.

Error codes

Code Meaning
-32600/-32601/-32602/-32603 Standard JSON-RPC
-32001 Protocol major version mismatch
-32002 Not paired / invalid token
-32003 Method not permitted on this transport
-32010 Task not found
-32011 Invalid destination path (outside allowed roots, or not writable)
-32012 Disk full
-32013 Probe failed (with data.httpStatus)
-32014 Rate limited (pairing brute-force lockout)