14 Commits
Author SHA1 Message Date
samiandClaude Opus 5 9624e685af docs: cmake/platform.cmake belongs to PKG/QA, not PORT (ADR 0020 amendment)
CI / clang-format (push) Canceled after 0s
CI / testserver (push) Canceled after 0s
CI / bootstrap-script (push) Canceled after 0s
CI / bootstrap-script-2604 (push) Canceled after 0s
CI / extension-lint (push) Canceled after 0s
CI / build (clang) (push) Canceled after 0s
CI / build (gcc) (push) Canceled after 0s
CI / sanitizers (dev) (push) Canceled after 0s
CI / sanitizers (tsan) (push) Canceled after 0s
CI / clang-tidy (push) Canceled after 0s
CI / conformance (push) Canceled after 0s
CI / nightly-integration (push) Canceled after 0s
CI / gui-dod (push) Canceled after 0s
CI / gui-dod-nightly (push) Canceled after 0s
As written, ADR 0020 gave PORT ownership of cmake/platform*.cmake while also
making PORT wait for Phase 0 — but Phase 0's seams are exactly what needs
velox_platform_sources() to select platform/linux/*.cpp. PORT cannot start until
Phase 0 lands, and Phase 0 cannot land without the file PORT owns. PKG/QA caught
the circularity before writing anything, which was the right call.

Central OS detection is root build infrastructure, so it stays with PKG/QA and
lands during Phase 0. PORT keeps the per-OS backend directories and the macOS
and Windows packaging, and owns no part of cmake/.

Also drops a hardcoded test count from the PORT definition of done; the suite
number moves every round and the gate is that it stays green.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-15 17:46:48 +04:00
sami 4e87730ce9 merge: lane/daemon — velox-nmhost, systemd units, velox(1) man page
CI / clang-format (push) Canceled after 0s
CI / testserver (push) Canceled after 0s
CI / bootstrap-script (push) Canceled after 0s
CI / bootstrap-script-2604 (push) Canceled after 0s
CI / extension-lint (push) Canceled after 0s
CI / build (clang) (push) Canceled after 0s
CI / build (gcc) (push) Canceled after 0s
CI / sanitizers (dev) (push) Canceled after 0s
CI / sanitizers (tsan) (push) Canceled after 0s
CI / clang-tidy (push) Canceled after 0s
CI / conformance (push) Canceled after 0s
CI / nightly-integration (push) Canceled after 0s
CI / gui-dod (push) Canceled after 0s
CI / gui-dod-nightly (push) Canceled after 0s
2026-09-15 17:38:46 +04:00
samiandClaude Opus 5 facd851824 docs: define the cross-platform porting strategy (ADR 0020, macOS first)
CI / clang-format (push) Canceled after 0s
CI / testserver (push) Canceled after 0s
CI / bootstrap-script (push) Canceled after 0s
CI / bootstrap-script-2604 (push) Canceled after 0s
CI / extension-lint (push) Canceled after 0s
CI / build (clang) (push) Canceled after 0s
CI / build (gcc) (push) Canceled after 0s
CI / sanitizers (dev) (push) Canceled after 0s
CI / sanitizers (tsan) (push) Canceled after 0s
CI / clang-tidy (push) Canceled after 0s
CI / conformance (push) Canceled after 0s
CI / nightly-integration (push) Canceled after 0s
CI / gui-dod (push) Canceled after 0s
CI / gui-dod-nightly (push) Canceled after 0s
The tree is Ubuntu-first, but the Linux-specific surface is about ten files and
already sits at the bottom of the I/O and RPC stacks. Rather than fork the repo
or scatter ifdefs, platform differences go behind narrow seams with one backend
directory per OS, selected centrally in cmake/platform.cmake.

Linux stays the reference implementation and its behaviour does not change: the
owning lanes (CORE, DAEMON) move today's Linux code behind the seams on Ubuntu
first, with the suite still green, before any macOS code exists. A new PORT lane
owns only the per-OS backends, so macOS work never writes shared logic and
CLAUDE.md's lane rule stays intact.

Adds ADR 0020 (the decision and phases), docs/08-porting.md (seam list, API
mapping for macOS and Windows, dependency table, verification gates) and
docs/agents/AGENT-PORT.md (the lane brief), plus the PORT row in the ownership
table.

Co-Authored-By: Claude Opus 5 <[email protected]>
2026-09-15 17:36:27 +04:00
samiandClaude Sonnet 5 eb72aa522c daemon: build velox-nmhost, systemd socket activation + units, velox(1) man page
Closes build order items 7 (the systemd half) and 9 (D11 in deferrals.md).

velox-nmhost (nmhost/src/main.cpp, 185 lines): a poll()-driven byte pump between
Firefox's native-messaging framing on stdio (4-byte native-byte-order length prefix)
and veloxd's own NDJSON framing on the Unix socket. Reframes each direction, no JSON
parsing, no retry/backoff (the extension relaunches a fresh host on its own
reconnect), exits the moment either side closes. Deliberately dependency-free — no
veloxd_* library, no nlohmann_json — since it runs unconfined outside Firefox's
sandbox regardless of packaging format.

Two real bugs found and fixed while getting the integration test to actually pass
rather than hang, both exactly the class of bug a "trivial pump" invites:
1. Never set the pumped fds non-blocking, so the "drain what's available" read loop
   blocked on its own second read() instead of returning to poll().
2. stdin and stdout are two different descriptors (0 and 1), not one — an early draft
   polled POLLOUT on fd 0, which is opened read-only, so EOF and writability were
   never both observable through the same pollfd entry.

packaging/nativehost/com.velox.host.json + its own README.md supersede
AGENT-DAEMON.md's stale "four locations" line: spike S1 / ADR 0003 found only three
manifest locations are real (~/.mozilla/native-messaging-hosts/ for BOTH deb/tarball
and snap Firefox, /usr/lib/mozilla/... for deb/tarball only, the flatpak sandbox path)
— the fourth, ~/snap/firefox/common/.mozilla/..., is not read by snap Firefox at all.
The README spells out the per-user-manifest / postinst enumeration implication for
PKG/QA (postinst runs once as root; the two ~/-relative locations are per-user) and
flags that docs/07-packaging.md's own install-layout line only shows the one
root-owned path.

Socket activation: rpc/systemd_activation.cpp is a from-scratch sd_listen_fds() (env
vars only — LISTEN_PID/LISTEN_FDS, fd 3 — no libsystemd link) that UdsServer::start()
checks first, skipping its own create/bind/chmod/listen when systemd already bound
the socket. packaging/systemd/velox.socket + velox.service are the unit pair,
verified both by systemd-analyze verify and by an actual fork/dup2/execve simulation
of the activation handshake — a real session.hello round-tripped over the handed-off
fd with no bind() ever called inside the daemon for that run. velox.service
deliberately skips ProtectSystem=/ProtectHome=/ReadWritePaths=: saveTo.allowedRoots is
user-configurable to anywhere on the filesystem, and a sandbox here would turn a
legitimately-configured save location into an opaque EROFS/EACCES instead of the
daemon's own clear -32011.

cli/man/velox.1 documents the CLI as it actually exists today (add/ls/pause/resume/rm,
--json) — the queue/settings subcommands AGENT-DAEMON.md's build order originally
sketched aren't implemented in cli/src/main.cpp yet, so the page doesn't claim they
are. Checked warning-free with groff -mandoc -ww -z.

Full ctest: 57/57 (excluding the pre-existing, unrelated conformance failure noted in
earlier commits).

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
2026-09-12 22:46:52 +04:00
sami 3e553f08c0 merge: lane/pkg-qa — GUI DoD gates wired, conformance required
CI / clang-format (push) Canceled after 0s
CI / testserver (push) Canceled after 0s
CI / bootstrap-script (push) Canceled after 0s
CI / bootstrap-script-2604 (push) Canceled after 0s
CI / extension-lint (push) Canceled after 0s
CI / build (clang) (push) Canceled after 0s
CI / build (gcc) (push) Canceled after 0s
CI / sanitizers (dev) (push) Canceled after 0s
CI / sanitizers (tsan) (push) Canceled after 0s
CI / clang-tidy (push) Canceled after 0s
CI / conformance (push) Canceled after 0s
CI / nightly-integration (push) Canceled after 0s
CI / gui-dod (push) Canceled after 0s
CI / gui-dod-nightly (push) Canceled after 0s
2026-09-12 22:05:06 +04:00
samiandClaude Sonnet 5 1d359af5a3 pkg: wire GUI's DoD harness into CI, mark live-veloxd conformance required
gui-dod (per-PR: scroll-60fps + unhappy-path) and gui-dod-nightly (rss-flat,
schedule/workflow_dispatch) are live in ci.yml, driving GUI's newly-landed
gui/tests/dod/run.sh + gui-dod-harness. No Xvfb step: run.sh already runs
QT_QPA_PLATFORM=offscreen itself.

Each gate forced red once before being trusted (tests/integration/README.md
has the transcripts): VELOX_DOD_FRAME_BUDGET_MS=0.01 for scroll-60fps,
VELOX_DOD_RSS_SLACK_KIB=-999999999 for rss-flat, and — since run.sh always
starts a working mockd — a direct gui-dod-harness invocation against an
unreachable socket for unhappy-path, which hit the harness's own 75s
watchdog exactly as documented.

Recorded GUI's live finding (gui/docs/proto-requests-m1.md) that mockd
--drop-connection is a no-op over the UDS transport, so unhappy-path's
drop-connection phase can't yet exercise a real drop — coordinating with
PROTO on the fix rather than working around it locally. gui-dod stays
required regardless: its other two phases and the crash/hang/watchdog paths
still catch real regressions.

Added gui-dod to BRANCH_PROTECTION.md's required-checks table.

ADR 0019: the live-veloxd conformance runner (run.sh step 3b, already
unconditional inside the already-required conformance job) stays required
as PROTO's xfail list shrinks (18 entries now, down from 34; 57/57 fixtures
passing on main). No CI change needed — it was already inside a required
check; this records the decision not to carve out an exception for it.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01RBPR7iM3YPyxrjWsVtZDPJ
2026-09-12 22:04:09 +04:00
sami 75536304bb merge: lane/proto — conformance stops downloading real files 2026-09-12 21:44:55 +04:00
sami 27865888be merge: lane/gui 2026-09-12 21:40:31 +04:00
sami 939a19b7ea merge: lane/core 2026-09-12 21:40:31 +04:00
samiandClaude Sonnet 5 79c29b47e8 core: finish the hostile-mode matrix -- 8 remaining end-to-end cases
Was 7/16 of tools/testserver/README.md's mode table covered by
engine_test.cpp. Adds the rest:

- engine_expiring_signed_url_recovers_via_refresh_url: an expired signed
  URL 403s, the engine asks (paused, decision_calls >= 1) rather than
  failing terminally, and DownloadHandle::refresh_url() with a freshly
  signed URL completes it -- exercises both do_refresh_url() fixes and
  the probe-level referrer retry's second-403 path from the previous
  commit.
- engine_403_without_referer_retries_with_origin: no spec.referrer set,
  the automatic single retry (previous commit) recovers with zero
  decisions asked.
- engine_redirect_chain_follows_to_completion: 5 hops of a plain 302.
  No core-side change needed -- documents that CURLOPT_FOLLOWLOCATION/
  MAXREDIRS (already on, RequestOptions::follow_redirects) cover both
  the probe's and every worker's own request, not just one of the two.
- engine_slow_loris_stall_timeout_fires: proves curl's stall detector
  (CURLOPT_LOW_SPEED_LIMIT/_TIME, download_task.cpp's hardcoded 1024 B/s
  for 30s) actually fires rather than hanging. Needed a real fix, not
  just a test: every other test in this file relies on TestServer's
  short 1s loris dribble to keep runtime down, but 1s of trickle
  followed by full-speed streaming never accumulates curl's required 30
  CONSECUTIVE seconds under the floor, so it would never actually abort
  -- a test built on the default dribble would pass by the download
  merely finishing a bit late, not by observing the stall timeout fire.
  testserver_fixture.hpp's TestServer gained an explicit-loris-seconds
  constructor (default ctor unchanged, still 1s) so this one test can
  ask for a dribble (40s) that genuinely outlasts the threshold.
- engine_401_digest_then_provide_auth_completes: same shape as the
  existing 401-basic test: http_client.cpp already asks libcurl for
  CURLAUTH_ANY regardless of net::AuthScheme, so this needed no core
  change -- it passed on the first run and is here to prove that's true
  end-to-end, not just at the http_client unit level.
- engine_chunked_no_length_completes_single_segment: Transfer-Encoding:
  chunked, no Content-Length anywhere (including HEAD). No core change
  needed -- takes the same size-agnostic "unknown size, one plain-GET
  segment" path as the existing no-range test.
- utf8/legacy-content-disposition: already covered end-to-end by
  probe_reads_utf8_content_disposition and
  probe_reads_legacy_content_disposition in probe_test.cpp (probe-level,
  as these modes only affect the initial request) -- verified passing,
  no new test needed.

All 20 engine_test.cpp cases and all 10 probe_test.cpp cases pass. Every
testserver.py spawned while writing and running this was reaped by
TestServer's destructor; verified no stragglers with `ps aux` after each
run.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
2026-09-12 21:34:24 +04:00
samiandClaude Sonnet 5 8e7d14ba7e core: retry once with the original referrer on a 403, at probe and worker
docs/04-engine-design.md §7's failure policy table has said "403 after
redirect: retry once with the original referrer; many CDNs require it"
since it was written, and Error::forbidden's own enum comment says the
same -- but grepping download_task.cpp and http_client.cpp for 403 turned
up nothing. It was never built.

Implemented at both points a 403 can surface:

- The probe (net::Prober, a separate request path from segment workers):
  on_probe_result() now retries once via restart_probe(false), with
  effective_referrer set to the download URL's own origin (origin_of(),
  via net::split_url()), when the failure is Error::forbidden and this is
  the first retry. A second 403 asks rather than fails outright --
  auto_pause_locked(..., false, true), the same "ask, don't just fail"
  path 416/etag-mismatch already use -- specifically so DownloadHandle::
  refresh_url() stays usable afterward (its own contract requires a
  non-terminal task); this is what makes the expiring-signed-url mode's
  README-documented refresh_url() recovery actually reachable.

- Each segment worker (SegWorker::forbidden, set in seg_head() on a 403
  HEAD): the same one-shot referrer retry via retry_worker(), landing on
  auto_pause_locked() on a second 403 for the same reason.

Both paths route the retry's Referer through a new effective_referrer
field rather than spec.referrer directly, since the origin-retry must not
overwrite what the caller actually asked for -- start_worker_locked() and
restart_probe() were switched to send effective_referrer instead.

do_refresh_url() had two latent bugs surfaced by actually exercising the
expiring-signed-url recovery path end-to-end:

1. It unconditionally proceeded to resume even when the refresh probe
   itself failed -- a bad refresh URL would silently un-pause a task with
   nothing behind it. Now returns (stays paused) on !r.has_value().
2. It only handled "already probed once, just refreshing a few fields" --
   for a task whose first-ever probe never succeeded (every hostile mode
   this commit adds a test for that pauses at the initial probe, not
   mid-download), s->registered was never true, so the existing
   `if (s->registered) set_want()` never fired and nothing happened. Now
   detects !s->have_probe and calls finish_probe_locked() directly, the
   actual first-time registration/segmenter-construction path.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
2026-09-12 21:34:24 +04:00
sami c1c5c82f8b merge: lane/ext — real-veloxd verification, session.subscribe fix 2026-09-12 17:20:19 +04:00
samiandClaude Sonnet 5 4e177ec809 proto: stop conformance from downloading real files, ADR the null-clearing gap
1. download.add.json had startMode "now" against a real, large (~6 GB)
   Ubuntu ISO with saveDir hardcoded to /home/sami/Downloads/Programs.
   Against a real veloxd (tests/conformance/run.sh) that's a real
   download into the real user's real home, every single run — it had
   already happened twice. startMode -> "later" (exercises the add path,
   hands nothing to the engine) and saveDir is dropped entirely (resolves
   to saveTo.defaultDir instead, checked against allowedRoots the same
   way). Documented the rule this fixture was breaking in
   contracts/fixtures/README.md so it doesn't happen a third time.

   Auditing the rest for the same shape (now real: capture.offer, D7)
   found a second, subtler instance: capture.offer.take.json's "take"
   admits a real, immediately-started task the same way download.add
   does, and the "Programs" category's saveDir is a migration-seeded
   builtin (~/Downloads/Programs) that no isolated test setup can
   redirect -- so even after pointing the URL at example.org (RFC 2606),
   a real ~6 GB sparse .veloxpart still landed in the real home on the
   declared Content-Length alone. Shrunk to a plausible-but-small 5 MiB.
   Also scoped to "transport": "uds" -- a real "take" persists an active
   task, so replaying the same fixture again on the second live transport
   against the same shared daemon was hitting capture.offer's own
   dedupe-by-URL and failing on a missing taskId, not a bug.
   download.add's other real-URL siblings (errors/*.invalid-path,
   *.invalid-params, *.disk-full) all fail before admission or are
   requires-gated; left alone.

2. ADR 0018: DAEMON can set a nullable field through download.update /
   settings.set but never clear it back to null, because the generated
   C++ parser collapses "absent" and "explicit null" to the same
   std::nullopt for every optional field (contracts/codegen/gen_cpp.py,
   on purpose, and correct for create-style params -- just wrong for
   patch-style ones, which is the only place the schema documents
   "explicit null clears"). Decision: an opt-in x-clearable schema
   annotation makes just those fields std::optional<std::optional<T>> in
   C++ (TS already round-trips this natively); not a blanket rule
   (would retype response fields like TaskSummary.effectiveUrl that have
   no clear-vs-absent distinction to make), not an explicit clear-list
   field (would redesign a wire contract DAEMON already built against
   just to route around a generator gap). Recorded, not implemented here
   -- that's its own PROTO PR (schema annotations + gen_cpp.py + gen_ts.py
   + regeneration + a minor VERSION bump per ADR 0015), not bundled into
   a fixture-safety pass. Left a pointer to the ADR at the generator
   comment it concerns.

3. Re-verified every xfail entry against current deferrals.md rather
   than trust the reasons already on file: D7/D8 (capture.offer/
   getRules), D3d/e/f/g/h/i (rules, queue.reorder, schedule, limiter,
   download.update/refreshUrl) and D9 (settings) have all closed since
   the list was last pruned, so most of it was stale. Removed everything
   that now cleanly passes; kept and re-reasoned everything that doesn't:
   - errors/download.provideAuth.not-found.json stays, as asked: real
     bug, on_download_provideAuth never checks the task exists.
   - category.list.json (mimeTypes -- documented D3a gap), schedule.set.json
     (nextRunAt -- documented D3f gap): unchanged in substance, reason
     text was already accurate.
   - download.probe/get/list/update.json, session.hello.json,
     queue.start/reorder.json, category.remove.json: not bugs -- each
     golden depicts a richer lifecycle/config state (a probed download,
     real queue or category membership, media/grabber capabilities) than
     this harness's fresh, never-started bound tasks and empty isolated
     DB can produce.
   - limiter.get.json: real fixture bug, not a daemon one -- applyToRunning
     is a write-only instruction on limiter.set, on_limiter_get never
     returns it; the golden shouldn't have had it either. Fixed the
     fixture and tools/mockd's own limiter.get, which had the same field
     hardcoded into its in-memory state independent of the fixture file.
   - grabber.*/media.*: still genuinely stub (M4 territory).
   Only remaining unexpected-pass surfaced while re-verifying
   (errors/capture.offer.ignore.json, always "take" instead of "ignore")
   traced to capture.minSizeBytes defaulting to 0 on a fresh daemon,
   making its below-minimum-size scenario unreachable -- not a bug, so
   raised the setting in run.sh's isolated seeding instead of xfailing it.

ctest -L conformance: green, 100% (2/2), ~87s.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01SFeUKLbdHizrJjLBeK7ffz
2026-09-12 16:56:35 +04:00
samiandClaude Sonnet 5 b5c6e1f47d ext: verify against a real veloxd, not just FakeDaemon
main.cpp's single-instance lock is now keyed to the runtime directory
instead of the euid, so an isolated XDG_RUNTIME_DIR/XDG_DATA_HOME/
XDG_CONFIG_HOME/HOME gets its own veloxd alongside anyone else's.
tests/live/real-veloxd.test.ts spawns one (VELOX_PAIR_AUTO=1 standing
in for the GUI's Allow click) plus tools/testserver/testserver.py, and
exercises the real WebSocketTransport end to end: session.hello,
pairing, the token surviving a reconnect, a wrong token rejected and
then rate-limiting the next pairing attempt, download.add reaching a
real running task, the real capture.offer path (rules table + category
folder + dedupe) taking a monitored download and ignoring its own
duplicate, and fail-open proven by SIGKILLing the daemon mid-offer —
the hook still resolves to {} inside its 750ms budget. A last case
proves fail-open at the transport layer too: a call against a closed
socket rejects instead of hanging.

Guarded behind  so it skips itself (with a clear message)
when no daemon binary is around — npm test and CI are unaffected;
run it with VELOXD_BIN=/path/to/veloxd npx vitest run tests/live.

Real-daemon testing found one actual bug, fixed here: background/
index.ts never called session.subscribe, so event.task.progress and
friends never reached this connection at all — FakeDaemon's tests
never caught it because FakeDaemon broadcasts regardless of
subscription state. Now subscribed to the full event set on every
connect (first connect and every reconnect), which is what the popup's
live-progress path actually depends on against a real daemon.

Per instructions: ctest -L conformance was not run (pending PROTO's
fixture fix).

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Ed8KEmAW48v4YHdxLtqsMB
2026-09-12 16:43:55 +04:00
43 changed files with 2214 additions and 139 deletions
+2 -1
View File
@@ -18,8 +18,9 @@ policy so it can be re-applied or audited.
| `bootstrap-script-2604` | now — real `--with-clang` install in a 26.04 container; the release the project ships on |
| `build (gcc)` / `build (clang)` | now — core, daemon and gui have merged |
| `sanitizers (dev)` / `sanitizers (tsan)` | now — core, daemon and gui have merged |
| `conformance` | **now — `tests/conformance/` has landed; this is the M0 exit gate** |
| `conformance` | **now — `tests/conformance/` has landed; this is the M0 exit gate.** Includes the live-`veloxd` runner (step 3b of `run.sh`), unconditional in the script — see `docs/adr/0019-live-veloxd-conformance-is-required.md`. |
| `extension-lint` | now — `extension/` has merged (MV3 manifest + esbuild build) |
| `gui-dod` | now — `gui/tests/dod/` has landed (GUI M1 DoD gates R3: `scroll-60fps`, `unhappy-path`); see `tests/integration/README.md#gui-m1-definition-of-done-gates-r3`. `gui-dod-nightly` (`rss-flat`) is schedule-only and cannot be a required PR check. |
`clang-tidy` is intentionally **not** required through M1 (`continue-on-error: true`,
`.clang-tidy` has `WarningsAsErrors: ''`). Make it required at M2.
+53
View File
@@ -248,3 +248,56 @@ jobs:
run: cmake --build --preset dev --target veloxd
- name: Nightly integration run
run: python3 tests/integration/nightly_run.py --veloxd build/dev/bin/veloxd --tasks 50 --timeout 180
gui-dod:
# Per-PR GUI M1 DoD gates (gui/docs/pkg-qa-requests-m1.md R3): scroll-60fps and
# unhappy-path. The 10-minute rss-flat gate is gui-dod-nightly, not here. GUI's
# harness defaults QT_QPA_PLATFORM=offscreen itself, so no Xvfb/compositor needed.
# See tests/integration/README.md#gui-m1-definition-of-done-gates-r3 for what each
# gate catches and the forced-failure transcript proving it isn't vacuous.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Bootstrap toolchain
run: sudo ./tools/bootstrap.sh
- uses: actions/setup-node@v4
with:
node-version: '22' # tools/mockd
- name: Configure + build
run: |
cmake --preset dev
cmake --build --preset dev --target gui-dod-harness
- name: Install mockd
run: cd tools/mockd && npm ci
- name: Gates
run: |
gui/tests/dod/run.sh scroll-60fps --json scroll.json
gui/tests/dod/run.sh unhappy-path --json unhappy.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: gui-dod-${{ github.run_id }}
path: "*.json"
gui-dod-nightly:
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Bootstrap toolchain
run: sudo ./tools/bootstrap.sh
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Configure + build
run: |
cmake --preset dev
cmake --build --preset dev --target gui-dod-harness
- run: cd tools/mockd && npm ci
- name: RSS soak (10 min)
run: gui/tests/dod/run.sh rss-flat --json rss.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: gui-dod-rss-${{ github.run_id }}
path: rss.json
+1
View File
@@ -12,6 +12,7 @@ Read this before touching anything. Then read your lane brief in [docs/agents/](
| GUI | `gui/` | everything else |
| EXT | `extension/` | everything else |
| PKG/QA | `packaging/`, `.github/`, `tools/testserver/`, `tests/integration/`, `tests/e2e/`, root build files | any lane's feature code |
| PORT | `**/platform/<os>/**`, `packaging/macos/`, `packaging/windows/` | all logic, and `cmake/` — see [ADR 0020](docs/adr/0020-cross-platform-strategy.md) |
If your task seems to require editing another lane's files, that is a signal the interface
is wrong. **File the request; don't reach across.**
+129
View File
@@ -0,0 +1,129 @@
.TH VELOX 1 "2026-09-12" "Velox" "User Commands"
.SH NAME
velox \- command-line client for veloxd, the Velox download manager daemon
.SH SYNOPSIS
.B velox
.I command
.RI [ options ]
.SH DESCRIPTION
.B velox
talks to
\fBveloxd\fR
over its Unix-domain socket and gives a scriptable, terminal-first view of the same
downloads the GUI and the Firefox extension see. It does no downloading itself \(em every
command is a thin JSON-RPC call; the daemon owns all state.
.PP
.B veloxd
must already be running (see
.B ENVIRONMENT
below for how
.B velox
finds it, and
.BR systemctl (1)
.RI ( "systemctl --user status velox" )
for whether it's up). If nothing is listening on the socket,
.B velox
exits with status 3 rather than hanging.
.SH COMMANDS
.TP
.BI "add " url " [\-\-dir " dir "] [\-\-out " name "] [\-\-segments " n ]
Add a new download. Prints the new task's id and initial state.
.RS
.TP
.BI "\-\-dir " dir
Destination directory. Must resolve inside one of the daemon's configured
.B saveTo.allowedRoots
or the call fails; omit to use the configured default download directory.
.TP
.BI "\-\-out " name
Filename to save as. Omit to derive one from the URL (or, once the daemon has probed it,
from the server's own
.BR Content-Disposition ).
.TP
.BI "\-\-segments " n
Requested connection count for this download, 1\(en32. The daemon may use fewer \(em a
per-host cap or a source that turns out not to support resuming both lower it. The
.B ls
table (and
.RI "\-\-json's " segments
field) show what was actually granted, not what was asked for.
.RE
.TP
.B ls
List every download the daemon knows about: id, state, progress, and filename. With
.BR \-\-json ", the raw " download.list " result instead of the table."
.TP
.BI "pause " id " [" id " ...]"
Pause one or more downloads by id. A download already paused, or already finished, is
left alone \(em not an error.
.TP
.BI "resume " id " [" id " ...]"
Resume one or more paused downloads.
.TP
.BI "rm " id " [" id " ...] " "[\-\-delete\-file]"
Remove one or more downloads from the list. Without
.BR \-\-delete\-file ,
any partial data on disk
.RI ( .veloxpart / .veloxpart.meta )
is discarded but a
.B completed
file is left in place. With
.BR \-\-delete\-file ,
the finished file is deleted too \(em there is deliberately no default for this flag; you
must say which you mean every time.
.SH OPTIONS
.TP
.B \-\-json
Print the raw JSON-RPC result instead of a formatted table. Works with every command;
combine with
.BR jq (1)
for scripting. On error, the JSON form is an
.B {"error": {...}}
object on stdout rather than a message on stderr.
.TP
.B \-h ", " \-\-help
Print usage and exit 0.
.SH EXIT STATUS
.TP
.B 0
Success.
.TP
.B 1
The daemon reached the call but returned a JSON-RPC error (bad task id, path outside the
allowed roots, and so on). The message is on stderr, or in the JSON error object with
.BR \-\-json .
.TP
.B 2
Usage error \(em missing argument, unknown option, or unknown command.
.TP
.B 3
Could not reach
.B veloxd
at all: not running, or its socket is missing or unreachable.
.SH ENVIRONMENT
.TP
.B XDG_RUNTIME_DIR
.B velox
connects to
.IR "$XDG_RUNTIME_DIR/velox/velox.sock" .
If unset, it falls back to
.IR /run/user/ <uid> ,
matching
\fBveloxd\fR's
own resolution \(em the two must agree for the client to find the daemon, so this is
normally left to the desktop session's default rather than set by hand.
.SH FILES
.TP
.I $XDG_RUNTIME_DIR/velox/velox.sock
The daemon's Unix-domain socket, mode 0600, same-UID only (\fBSO_PEERCRED\fR checked on
every connection \(em this is the transport's authorization, not an extra login).
.SH SEE ALSO
.BR systemctl (1),
.BR jq (1)
.PP
.I docs/01-architecture.md
and
.I docs/agents/AGENT-DAEMON.md
in the Velox source tree for the daemon's own build order and the wire protocol
.B velox
speaks.
+6 -1
View File
@@ -506,7 +506,12 @@ def emit_field_parse(f: Field, indent: str) -> list[str]:
f'{i} const auto it = j.find("{f.name}");']
if f.optional:
# Absent and null mean the same thing: the field is not set. A client that omits
# a nullable field and one that sends null are treated identically on purpose.
# a nullable field and one that sends null are treated identically on purpose --
# correct for create-style params, where there is no existing value to distinguish
# "never set" from "explicitly cleared". Patch-style fields need the distinction
# (download.update's patch: "an explicit null clears a nullable field") and get an
# opt-in exception via x-clearable per ADR 0018 (not implemented yet: this is the
# decision record, not the generator change).
o.append(f"{i} if (it != j.end() && !it->is_null()) {{")
o += emit_value_parse(f.type, "(*it)", "val", "fp", i + " ")
o.append(f"{i} out.{m} = std::move(val);")
+25 -1
View File
@@ -21,7 +21,7 @@ fixtures/
```jsonc
{
"name": "download.add — start an ISO now, into the Programs category",
"name": "download.add — add an ISO for later, into the Programs category",
"description": "Why this case is worth pinning.",
"transport": "uds", // optional: replay only on this transport
"requires": "...", // optional: a condition a plain server cannot produce
@@ -81,3 +81,27 @@ cases in `tests/integration/`.
correct response is *no response*: past 750 ms the extension must abandon the offer and let
Firefox download normally. A download manager that eats downloads when its daemon is down
is worse than no download manager.
## No fixture may pair a real external URL with `startMode: "now"`
This suite replays every fixture against a real, live `veloxd` (`tests/conformance/run.sh`),
not just `mockd`. `mockd` never actually fetches anything, so it hid this for a while: a
fixture with `startMode: "now"` (or `"queue"` into a running queue — anything that gets
admitted to the scheduler right away) and a real, resolvable URL makes a **real** daemon
actually start downloading it, for real, onto whatever machine runs the suite. This
happened — twice, with `download.add.json` pointed at a ~6 GB Ubuntu ISO, straight into the
developer's real `~/Downloads`.
The fix in each case is one of:
- `startMode: "later"` — exercises the add path (validation, category assignment, the
event) without ever handing the task to the engine;
- a URL under `example.org`/`example.com` (IANA-reserved for exactly this, RFC 2606) —
resolvable enough to validate as a URL, never a real download source;
- `requires`, if the fixture's entire point needs a real transfer to fail in a specific way
(see `errors/download.add.disk-full.json`) — skipped by default, so it only ever runs
where the condition has actually been arranged.
A real `saveDir` gets the same treatment for the same reason: an absolute path like
`/home/sami/Downloads/...` only means anything on the machine that fixture was written on.
Omit `saveDir` and let `saveTo.defaultDir` apply, or use a relative-feeling path under a
root the runner controls.
+6 -5
View File
@@ -1,22 +1,23 @@
{
"name": "capture.offer — attachment on a monitored type is taken",
"description": "Golden fixture. tests/conformance replays this against the real daemon AND the TS client. If either side drifts, this goes red before the lanes ever integrate.",
"description": "Golden fixture. tests/conformance replays this against the real daemon AND the TS client. If either side drifts, this goes red before the lanes ever integrate. url is example.org (RFC 2606), not a real download source: 'take' against a real veloxd (tests/conformance/run.sh) admits a real task and hands it to the engine for real, and no fixture may do that against a real external URL. contentLength is a plausible-but-small 5 MiB rather than a real ISO's size: the 'Programs' category's saveDir is a migration-seeded builtin (~/Downloads/Programs, daemon/src/store/migrations/0001_initial.sql), not something an isolated test run's settings can redirect, so 'take' always sparse-preallocates into that real path on whatever machine runs this suite -- keeping the declared size small keeps that footprint trivial instead of a real ISO's worth of disk. transport is uds only: a real 'take' persists an active task, so replaying this same fixture again on a second live transport against the same daemon would correctly dedupe against it (capture.offer dedupes by exact URL) and get 'ignore' instead -- an artifact of replaying one fixture against one shared daemon over two transports, not a behaviour to golden.",
"transport": "uds",
"request": {
"jsonrpc": "2.0",
"id": 42,
"method": "capture.offer",
"params": {
"url": "https://releases.ubuntu.com/26.04/ubuntu-26.04-desktop-amd64.iso",
"url": "https://example.org/dl/ubuntu-26.04-desktop-amd64.iso",
"method": "GET",
"tabUrl": "https://releases.ubuntu.com/26.04/",
"tabUrl": "https://example.org/26.04/",
"headers": {
"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:154.0) Gecko/20100101 Firefox/154.0",
"Referer": "https://releases.ubuntu.com/26.04/",
"Referer": "https://example.org/26.04/",
"Accept": "*/*"
},
"cookies": [],
"contentType": "application/octet-stream",
"contentLength": 6228541440,
"contentLength": 5242880,
"contentDisposition": "attachment; filename=\"ubuntu-26.04-desktop-amd64.iso\"",
"filename": "ubuntu-26.04-desktop-amd64.iso",
"origin": "moz-extension://11111111-2222-3333-4444-555555555555"
+6 -7
View File
@@ -1,6 +1,6 @@
{
"name": "download.add \u2014 start an ISO now, into the Programs category",
"description": "The ordinary add path. saveDir is canonicalized and checked against the allowed roots before anything is written.",
"name": "download.add — add an ISO for later, into the Programs category",
"description": "The ordinary add path. saveDir is canonicalized and checked against the allowed roots before anything is written. startMode is 'later' deliberately: this suite replays against a real veloxd (tests/conformance/run.sh), and a real daemon given startMode 'now' would actually start fetching url for real. No fixture may pair a real external URL with startMode 'now' -- see contracts/fixtures/README.md.",
"request": {
"jsonrpc": "2.0",
"id": 11,
@@ -8,10 +8,9 @@
"params": {
"url": "https://releases.ubuntu.com/26.04/ubuntu-26.04-desktop-amd64.iso",
"filename": "ubuntu-26.04-desktop-amd64.iso",
"saveDir": "/home/sami/Downloads/Programs",
"categoryId": "programs",
"segments": 8,
"startMode": "now"
"startMode": "later"
}
},
"response": {
@@ -19,13 +18,13 @@
"id": 11,
"result": {
"taskId": "$uuid",
"state": "connecting",
"state": "paused",
"duplicate": null
}
},
"assertions": [
"the .veloxpart file is created sparse and preallocated at the final size",
"saveDir resolves inside saveTo.allowedRoots, or the call fails -32011 having written nothing",
"saveDir is omitted here on purpose: it resolves to saveTo.defaultDir, which is itself checked against saveTo.allowedRoots the same way an explicit saveDir would be -- see errors/download.add.invalid-path.json for the -32011 case",
"startMode 'later' lands the task in 'paused' and never hands it to the engine, so nothing is fetched and no .veloxpart is created yet -- that only happens once the task is actually started (download.start.json, or startMode 'now'/'queue' against a source this suite controls)",
"event.task.added is emitted to every subscriber before this reply is sent"
]
}
+4 -4
View File
@@ -1,6 +1,6 @@
{
"name": "limiter.get \u2014 the limiter is off",
"description": "globalBps still carries the last configured value so the GUI can restore it when the user re-enables the limit.",
"description": "globalBps still carries the last configured value so the GUI can restore it when the user re-enables the limit. applyToRunning is a write-only instruction on limiter.set (\"retune already-running transfers now\", not a persisted setting), so it never comes back from get.",
"request": {
"jsonrpc": "2.0",
"id": 52,
@@ -12,11 +12,11 @@
"id": 52,
"result": {
"enabled": false,
"globalBps": 2097152,
"applyToRunning": false
"globalBps": 2097152
}
},
"assertions": [
"enabled false means no throttling regardless of globalBps"
"enabled false means no throttling regardless of globalBps",
"applyToRunning is absent, not false: it's meaningless outside a limiter.set call"
]
}
+127 -9
View File
@@ -64,6 +64,19 @@ std::string lower(std::string s) {
return s;
}
// scheme://host[:port] of `url`, with no path/query/fragment -- what docs/04 §7's "403
// after redirect: retry once with the original referrer" retries with as the Referer
// header. Empty on an unparseable URL (the caller just won't get a referrer retry).
std::string origin_of(std::string_view url) {
auto s = net::split_url(url);
if (!s.valid)
return {};
std::string out = s.scheme + "://" + s.host;
if (s.port)
out += ":" + std::to_string(*s.port);
return out;
}
} // namespace
// What to do once every worker has drained (see DownloadTaskState::begin_drain_locked).
@@ -88,6 +101,7 @@ struct SegWorker {
bool needs_auth = false;
bool wrong_status = false;
bool range_bad = false;
bool forbidden = false; // 403 -- docs/04 §7's "retry once with the original referrer"
bool auth_handshake = false; // saw a 401/407 and let libcurl resend with credentials
std::string resp_etag, resp_last_modified; // captured on a wrong_status 200, for demote
std::optional<ErrorInfo> flush_error;
@@ -135,6 +149,16 @@ struct DownloadTaskState : std::enable_shared_from_this<DownloadTaskState> {
std::optional<std::uint64_t> total_size;
std::string origin_host;
// docs/04 §7's "403 after redirect: retry once with the original referrer" -- many
// CDNs 403 a bare/foreign Referer. Starts as spec.referrer (the browser's, verbatim);
// start_worker_locked() sends this, not spec.referrer directly, so a 403 retry can
// override it (to the download URL's own origin) without touching what the caller
// actually asked for. referrer_retried bounds it to exactly once per task -- a second
// 403 with a same-origin Referer already set is a real, honest failure
// (Error::forbidden), not something a referrer swap can fix.
std::string effective_referrer;
bool referrer_retried = false;
std::unique_ptr<segment::Segmenter> seg;
std::unique_ptr<io::SparseFile> file;
std::unordered_map<std::uint32_t, std::unique_ptr<SegWorker>> workers;
@@ -164,7 +188,7 @@ struct DownloadTaskState : std::enable_shared_from_this<DownloadTaskState> {
std::vector<std::function<void()>> deferred;
DownloadTaskState(TaskHost &h, TaskId i, DownloadSpec s, DownloadCallbacks c)
: host(h), id(i), spec(std::move(s)), cbs(std::move(c)) {}
: host(h), id(i), spec(std::move(s)), cbs(std::move(c)), effective_referrer(spec.referrer) {}
// --- deferred callbacks -------------------------------------------------------------
void defer(std::function<void()> fn) {
@@ -286,7 +310,7 @@ void DownloadTaskState::restart_probe(bool with_auth) {
pr.url = spec.url;
pr.headers = spec.headers;
pr.cookies = spec.cookies;
pr.referrer = spec.referrer;
pr.referrer = effective_referrer;
pr.user_agent = spec.user_agent;
pr.proxy = spec.proxy;
if (with_auth)
@@ -299,12 +323,37 @@ void DownloadTaskState::restart_probe(bool with_auth) {
}
void DownloadTaskState::on_probe_result(Result<net::ProbeResult> r) {
bool retry_probe_with_referrer = false;
{
std::unique_lock lk(mu);
if (retired.load() || is_terminal(state))
return;
if (!r.has_value()) {
fail_locked(std::move(r).error());
ErrorInfo e = std::move(r).error();
// docs/04 §7's referrer retry applies here too: a probe (HEAD, or the
// ranged-GET fallback when HEAD is refused -- probe.cpp) can be the request
// that actually gets 403'd, before any segment worker exists to retry it
// (net::Prober builds its own request from ProbeRequest::referrer, not
// through start_worker_locked() -- see restart_probe()'s use of
// effective_referrer below). Same one-shot bound via referrer_retried as the
// worker-level retry (seg_finished's w->forbidden branch) shares.
if (e.code == Error::forbidden && !referrer_retried) {
referrer_retried = true;
effective_referrer = origin_of(spec.url);
retry_probe_with_referrer = true;
} else if (e.code == Error::forbidden) {
// Already retried with the origin referrer and still 403 -- not something
// another blind retry fixes (an expired signed URL, a private resource).
// Ask rather than fail outright, the same "ask, don't just fail" shape as
// wrong_status/range_bad/the worker-level 403 branch: refresh_url() is a
// no-op once the task is terminal, and tools/testserver's expiring-signed-
// url mode (also a bare 403, indistinguishable from any other without
// parsing the body -- CLAUDE.md §3, core never does) is meant to be
// recovered exactly that way.
auto_pause_locked(std::move(e), false, true);
} else {
fail_locked(std::move(e));
}
} else {
probe = std::move(r).value();
have_probe = true;
@@ -319,6 +368,8 @@ void DownloadTaskState::on_probe_result(Result<net::ProbeResult> r) {
}
}
flush_deferred();
if (retry_probe_with_referrer)
restart_probe(false);
}
void DownloadTaskState::finish_probe_locked() {
@@ -472,7 +523,7 @@ void DownloadTaskState::start_worker_locked(std::uint32_t seg_idx) {
req.url = current_url();
req.headers = spec.headers;
req.cookies = spec.cookies;
req.referrer = spec.referrer;
req.referrer = effective_referrer;
req.user_agent = spec.user_agent;
req.proxy = spec.proxy;
req.auth = spec.auth;
@@ -552,6 +603,10 @@ net::DataAction DownloadTaskState::seg_head(std::uint32_t seg_idx, const net::Re
w->range_bad = true;
return net::DataAction::abort;
}
if (h.status == 403) {
w->forbidden = true;
return net::DataAction::abort;
}
if (h.status >= 400)
return net::DataAction::abort;
seg->set_segment_state(seg_idx, segment::SegState::downloading);
@@ -637,7 +692,7 @@ void DownloadTaskState::seg_finished(std::uint32_t seg_idx, Result<net::Transfer
// (content-length-mismatch's honest-length lie, flaky-reset's tail, a proxy RST after
// the last byte). If the segment is fully covered, that's a success.
if (seg && !cancel_requested && !pause_requested && !w->needs_auth && !w->wrong_status &&
!w->flush_error && !w->range_bad) {
!w->flush_error && !w->range_bad && !w->forbidden) {
const std::uint64_t len = seg->segment_end(seg_idx) - seg->segment_start(seg_idx) + 1;
if (len != 0 && seg->segment_completed(seg_idx) >= len) {
r = Result<net::TransferStats>(net::TransferStats{});
@@ -756,6 +811,35 @@ void DownloadTaskState::seg_finished(std::uint32_t seg_idx, Result<net::Transfer
auto_pause_locked(ErrorInfo(Error::range_not_satisfiable, "416"), false, true);
return done();
}
if (w->forbidden) {
// docs/04 §7: "403 after redirect: retry once with the original referrer -- many
// CDNs require it." Bare/foreign Referer is the common cause; origin_of() rebuilds
// it from the (possibly redirected) URL the response actually came from. Exactly
// once per task, not a backoff series -- a second 403 with a same-origin Referer
// already set isn't something another blind retry can fix (a private/expired
// resource, an expiring signed URL past its window, ...). That's not necessarily
// terminal, though: ask (same "ask, don't just fail outright" shape as
// wrong_status/range_bad above) rather than fail_locked() outright, specifically
// so DownloadHandle::refresh_url() -- do_refresh_url() is a no-op once the task is
// terminal -- stays usable for the case tools/testserver's README pairs it with:
// a caller that gets a fresh signed URL and hands it back.
release_slot();
if (!referrer_retried) {
referrer_retried = true;
effective_referrer = origin_of(current_url());
seg->set_segment_state(seg_idx, segment::SegState::stalled);
auto wp = weak_from_this();
host.schedule(std::chrono::steady_clock::now(), [wp, seg_idx] {
if (auto s = wp.lock())
s->retry_worker(seg_idx);
});
if (workers.empty())
transition(EngineState::retry_wait, std::nullopt);
} else {
auto_pause_locked(ErrorInfo(Error::forbidden, "403", w->http_status), false, true);
}
return done();
}
if (!r.has_value()) {
ErrorInfo e = std::move(r).error();
@@ -1215,6 +1299,7 @@ void DownloadTaskState::do_refresh_url(std::string url, std::vector<net::HeaderF
net::ProbeRequest pr;
pr.url = spec.url;
pr.headers = spec.headers;
pr.referrer = effective_referrer;
pr.auth = spec.auth;
pr.proxy = spec.proxy;
host.probe(std::move(pr), [wp](Result<net::ProbeResult> r) {
@@ -1224,10 +1309,43 @@ void DownloadTaskState::do_refresh_url(std::string url, std::vector<net::HeaderF
std::unique_lock lk(s->mu);
if (s->retired.load() || is_terminal(s->state))
return;
if (r.has_value()) {
s->probe.effective_url = r.value().effective_url;
s->probe.etag = r.value().etag;
s->probe.last_modified = r.value().last_modified;
if (!r.has_value()) {
lk.unlock();
s->flush_deferred();
return; // still paused; the caller can retry refresh_url() or decide()
}
if (!s->have_probe) {
// The task's *first* probe never succeeded (e.g. this session's own
// expiring-signed-url path: 403, one referrer retry, still 403 -> ask rather
// than fail outright -- see on_probe_result() -- specifically so this branch
// exists to recover it). finish_probe_locked() is what actually registers the
// task with the budget and builds its Segmenter; nothing downstream of a
// partial field copy would ever start a worker without it.
s->probe = std::move(r).value();
s->have_probe = true;
s->awaiting_auth = false;
s->awaiting_decision = false;
s->finish_probe_locked();
lk.unlock();
s->flush_deferred();
return;
}
s->probe.effective_url = r.value().effective_url;
s->probe.etag = r.value().etag;
s->probe.last_modified = r.value().last_modified;
// refresh_url()'s own contract is "on a live OR PAUSED task, without losing
// progress" -- distinct from do_decide(restart), which discards progress. A task
// can be paused here for any of three reasons (a plain user pause, awaiting_auth,
// or awaiting_decision -- e.g. this session's own 403-after-referrer-retry path,
// or the pre-existing wrong_status/range_bad ones); apply_slot_target()'s guard
// blocks on awaiting_auth/awaiting_decision specifically, so leaving either set
// would have set_want() below recompute a target that nothing ever acts on --
// the caller's new URL re-probed successfully and then the task just sat there.
// Clear both and leave `paused` the same way do_decide(restart) does.
if (s->state == EngineState::paused) {
s->awaiting_auth = false;
s->awaiting_decision = false;
s->transition(EngineState::connecting, std::nullopt);
}
if (s->registered)
s->host.budget().set_want(s->id, s->want_slots());
+10 -2
View File
@@ -28,7 +28,14 @@ namespace vdm::testing {
class TestServer {
public:
TestServer() {
TestServer() : TestServer(1.0) {}
// loris_seconds overrides the dribble duration slow-loris mode uses (default matches the
// no-arg ctor's long-standing 1s). A test that needs curl's stall detector
// (CURLOPT_LOW_SPEED_TIME, hardcoded to 30s in download_task.cpp) to actually fire needs a
// dribble that outlasts that threshold, not the short one every other test relies on to
// keep runtime down.
explicit TestServer(double loris_seconds) {
const char *script = VDM_TESTSERVER_PY;
if (!script || !*script || ::access(script, R_OK) != 0)
return;
@@ -50,8 +57,9 @@ class TestServer {
int devnull = ::open("/dev/null", O_WRONLY);
if (devnull >= 0)
::dup2(devnull, STDERR_FILENO);
std::string loris_str = std::to_string(loris_seconds);
::execlp("python3", "python3", script, "--port", "0", "--seed", "9", "--loris-seconds",
"1", "--throttle-bps", "131072", static_cast<char *>(nullptr));
loris_str.c_str(), "--throttle-bps", "131072", static_cast<char *>(nullptr));
::_exit(127);
}
::close(pipefd[1]);
+185
View File
@@ -124,6 +124,31 @@ std::string server_sha(TestServer &srv, const std::string &mode, const std::stri
return out.substr(open + 1, close - open - 1);
}
// Small, deliberately identical extraction to server_sha's: GET /<mode>/sign/<size>?ttl=N
// and pull the "url" field's value out of the {"url":..., "exp":...} JSON body.
std::string sign_url(TestServer &srv, const std::string &mode, const std::string &size,
int ttl_seconds) {
std::string url =
srv.url("/" + mode + "/sign/" + size + "?ttl=" + std::to_string(ttl_seconds));
std::string cmd = "curl -s '" + url + "'";
std::string out;
if (FILE *f = ::popen(cmd.c_str(), "r")) {
char buf[1024];
while (std::fgets(buf, sizeof buf, f))
out += buf;
::pclose(f);
}
auto q = out.find("\"url\"");
if (q == std::string::npos)
return {};
auto colon = out.find(':', q);
auto open = out.find('"', colon);
auto close = out.find('"', open + 1);
if (open == std::string::npos || close == std::string::npos)
return {};
return out.substr(open + 1, close - open - 1);
}
DownloadSpec spec_for(TestServer &srv, const std::string &urlpath, const std::string &save) {
DownloadSpec s;
s.url = srv.url(urlpath);
@@ -328,6 +353,30 @@ VT_TEST(engine_401_then_provide_auth_completes) {
VT_CHECK_EQ(file_size(td.file("au.bin")), 1u * 1024 * 1024);
}
VT_TEST(engine_401_digest_then_provide_auth_completes) {
// Same shape as engine_401_then_provide_auth_completes, but the challenge is HTTP
// Digest (qop=auth) rather than Basic. provide_auth() doesn't know or care which --
// http_client.cpp always asks libcurl for CURLAUTH_ANY (net::AuthScheme::any) and lets
// curl negotiate against whatever WWW-Authenticate the server actually sent -- so this
// exists purely to prove that's true end-to-end, not just at the unit level.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
DownloadHandle h;
auto cbs = rec.cbs(&h, "test", "test");
h = eng.start(spec_for(srv, "/401-digest/file/1M", td.file("dg.bin")), std::move(cbs));
rec.arm(h);
auto r = rec.wait();
VT_REQUIRE(r.has_value());
VT_CHECK(rec.auth_calls.load() >= 1);
VT_CHECK_EQ(file_size(td.file("dg.bin")), 1u * 1024 * 1024);
auto got = hash_file(td.file("dg.bin"), Checksum::Algo::sha256);
VT_CHECK_EQ(got.value(), server_sha(srv, "401-digest", "1M"));
}
// --- hostile-mode matrix: the four where a bug is silent corruption, not a visible
// failure (docs/04 §5 "ask, never silently corrupt" / §7's failure-policy table). ---
@@ -458,6 +507,142 @@ VT_TEST(engine_content_length_mismatch_fails_honestly) {
VT_CHECK_EQ(::access(td.file("clm.bin").c_str(), F_OK), -1); // never renamed into place
}
// --- remaining hostile-mode matrix (tools/testserver/README.md's mode table). ---
VT_TEST(engine_expiring_signed_url_recovers_via_refresh_url) {
// A signed URL past its ttl 403s (tools/testserver's own JSON body distinguishes
// "expired" from "bad signature", but core never parses response bodies -- CLAUDE.md
// §3 -- so both just read as a 403). The one automatic referrer retry (see
// engine_403_without_referer_retries_with_origin, below) can't fix an expired
// signature, so the second 403 asks -- via the same auto_pause_locked(..., false,
// true) "ask, don't just fail" path as wrong_status/range_bad -- rather than
// terminally failing outright, specifically so DownloadHandle::refresh_url() (its own
// contract: works "on a live or paused task", never on a terminal one) stays usable:
// the README pairs this mode with exactly that recovery.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
std::string expired = sign_url(srv, "expiring-signed-url", "64K", /*ttl=*/1);
VT_REQUIRE(!expired.empty());
std::this_thread::sleep_for(1500ms); // let the ttl actually pass before the first request
DownloadSpec s;
s.url = expired;
s.save_path = td.file("exp.bin");
auto h = eng.start(std::move(s), rec.cbs());
for (int i = 0; i < 300 && rec.decision_calls.load() == 0; ++i)
std::this_thread::sleep_for(20ms);
VT_REQUIRE(rec.decision_calls.load() >= 1);
VT_CHECK_EQ(h.state(), EngineState::paused);
std::string fresh = sign_url(srv, "expiring-signed-url", "64K", /*ttl=*/60);
VT_REQUIRE(!fresh.empty());
h.refresh_url(fresh);
auto r = rec.wait(60s);
VT_REQUIRE(r.has_value());
VT_CHECK_EQ(file_size(td.file("exp.bin")), 64u * 1024);
auto got = hash_file(td.file("exp.bin"), Checksum::Algo::sha256);
VT_CHECK_EQ(got.value(), server_sha(srv, "expiring-signed-url", "64K"));
}
VT_TEST(engine_403_without_referer_retries_with_origin) {
// docs/04 §7: "403 after redirect: retry once with the original referrer -- many CDNs
// require it." No spec.referrer is set here (the common case for anything not
// initiated from a browser page, e.g. `velox add <url>`), so the first attempt 403s;
// the engine's own retry supplies the download URL's own origin as Referer, which
// this mode accepts, and the download completes with no decision ever asked.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
auto h = eng.start(spec_for(srv, "/403-without-referer/file/128K", td.file("ref.bin")),
rec.cbs());
auto r = rec.wait(30s);
VT_REQUIRE(r.has_value());
VT_CHECK_EQ(rec.decision_calls.load(), 0); // recovered automatically, not asked
VT_CHECK_EQ(file_size(td.file("ref.bin")), 128u * 1024);
auto got = hash_file(td.file("ref.bin"), Checksum::Algo::sha256);
VT_CHECK_EQ(got.value(), server_sha(srv, "403-without-referer", "128K"));
}
VT_TEST(engine_redirect_chain_follows_to_completion) {
// 5 hops (tools/testserver's own --redirect-depth default) of a plain 302, query
// string preserved across each. No CORE-side logic needed for this one -- libcurl's
// own CURLOPT_FOLLOWLOCATION (RequestOptions::follow_redirects, already on) and
// CURLOPT_MAXREDIRS (default 20, well over 5) do the whole thing -- this is here as
// the end-to-end check that they're actually wired through both the probe and every
// segment worker's own request, not just one of the two.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
auto h = eng.start(spec_for(srv, "/redirect-chain/file/1M", td.file("rc.bin")), rec.cbs());
auto r = rec.wait(30s);
VT_REQUIRE(r.has_value());
VT_CHECK_EQ(file_size(td.file("rc.bin")), 1u * 1024 * 1024);
auto got = hash_file(td.file("rc.bin"), Checksum::Algo::sha256);
VT_CHECK_EQ(got.value(), server_sha(srv, "redirect-chain", "1M"));
}
VT_TEST(engine_slow_loris_stall_timeout_fires) {
// Status line, headers, and body dribbled out one byte at a time for --loris-seconds,
// then (if the dribble hasn't already been cut off) normal streaming -- a connection
// that's technically alive (bytes ARE arriving, just far too slowly) but must not be
// allowed to hang the task forever. http_client.cpp sets CURLOPT_LOW_SPEED_LIMIT/_TIME
// (RequestOptions::low_speed_bytes_per_sec/low_speed_secs, hardcoded in
// download_task.cpp to 1024 B/s for 30s) for exactly this.
//
// Every other test in this file uses TestServer's default 1s loris dribble to keep
// runtime down, but 1s is far shorter than curl's 30s low_speed_time: a 1s trickle
// followed by full-speed streaming never accumulates 30 CONSECUTIVE seconds under the
// floor, so curl would never actually abort it -- the download would just complete
// slightly late, which would make this test pass for the wrong reason (or not exercise
// the stall timeout at all). Explicitly ask for a dribble that outlasts the 30s
// threshold so the stall timeout is the thing actually observed firing, not assumed.
TestServer srv(40.0);
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
auto s = spec_for(srv, "/slow-loris/file/64K", td.file("sl.bin"));
s.segments = 1;
s.max_retries = 1;
auto h = eng.start(std::move(s), rec.cbs());
auto r = rec.wait(60s); // stall timeout fires ~30s in; must resolve, not hang to 60s
VT_REQUIRE(!r.has_value());
VT_CHECK(is_retryable(r.error().code) || r.error().code == Error::max_retries_exhausted);
}
VT_TEST(engine_chunked_no_length_completes_single_segment) {
// No Content-Length anywhere (HEAD gets none either, since it's the same handler path)
// -- the probe can't know total_size or prove resumability, so this should take the
// exact same "unknown size, one plain-GET segment" path as engine_non_resumable_single_
// segment, just arriving there via a chunked body instead of a server that plainly
// refuses Range. No core-side work needed if that demotion is already size-agnostic;
// this is here to prove it, since every other test's server tells the probe the size
// up front.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
auto h = eng.start(spec_for(srv, "/chunked-no-length/file/2M", td.file("ch.bin")),
rec.cbs());
auto r = rec.wait();
VT_REQUIRE(r.has_value());
VT_CHECK_EQ(rec.decision_calls.load(), 0);
VT_CHECK_EQ(file_size(td.file("ch.bin")), 2u * 1024 * 1024);
auto got = hash_file(td.file("ch.bin"), Checksum::Algo::sha256);
VT_CHECK_EQ(got.value(), server_sha(srv, "chunked-no-length", "2M"));
}
// --- DAEMON-reported bug: Progress.speed_bps reads 0 for the whole life of a live
// download while downloaded bytes visibly advance. DAEMON reads progress by polling
// DownloadHandle::progress() (engine_port_core.hpp), not the on_progress push callback --
+1
View File
@@ -75,6 +75,7 @@ target_link_libraries(veloxd_sched PUBLIC velox::proto velox::core veloxd_store
add_library(veloxd_rpc STATIC
src/rpc/runtime_dir.cpp
src/rpc/single_instance.cpp
src/rpc/systemd_activation.cpp
src/rpc/event_loop.cpp
src/rpc/event_hub.cpp
src/rpc/uds_server.cpp
+2 -1
View File
@@ -7,7 +7,7 @@ close. Kept here (not buried in commit messages) so the next pass can see them a
|---|---|---|---|---|
| ~~D7~~ | **Closed — `capture.offer` is real.** Applies `capture.enabled`/`excludedHosts`/`monitoredExtensions`/`monitoredMimeTypes`/`minSizeBytes` from settings, then the rules table (`store::Rules` + CORE's `vdm::rules::match_rules`/`glob_match` — DAEMON only converts its own stored `proto::Rule` JSON into CORE's plain `vdm::rules::Rule` vocabulary, per that header's own layering note), resolves the category folder (a rule's explicit `categoryId`/`saveDir`, else `store::Categories::guess_by_extension` — the same extension-guess `download.probe`'s `suggestedCategoryId` already used, now shared instead of duplicated), dedupes against active (non-terminal) tasks by exact URL, and on `take` calls `add_one()` — the same path `download.add` itself uses — so a captured download is a real, admitted, persisted task, not a special case. The 750 ms deadline (CLAUDE.md §4 / AGENT-DAEMON.md build step 6) is checked cooperatively between every step via a new `rpc::CaptureDataSource` seam (real impl wraps `store::*`; a test fake can jump its own clock forward to simulate "the store was slow just now" with zero real sleep) — catches the realistic failure mode (several slow steps adding up) though it can't preempt one pathologically stuck single call. Verified against real `veloxd` + `tools/testserver`: a monitored-type offer answers in ~5ms and actually creates + downloads the task; an unmonitored type, an excluded host, a rule-vetoed host, and a second offer for a still-active URL all answer `ignore` with the right `reason`; a bad category save dir surfaces its real `-32011` rather than being swallowed. New `capture_offer_test` covers all of the above plus the deadline itself (two cases, one per "slow" checkpoint), asserting real wall-clock time barely moves even though the fake clock jumped 2 simulated seconds — proof the check reads the injected clock, not a disguised sleep. | `rpc/capture_data_source.hpp`, `rpc/dispatcher.{hpp,cpp}`, `store/rules.{hpp,cpp}`, `store/categories.{hpp,cpp}`, `store/tasks.{hpp,cpp}` | — | done |
| ~~D8~~ | **Closed alongside D7** — `capture.getRules` returns the same settings-backed `enabled`/`monitoredExtensions`/`monitoredMimeTypes`/`minSizeBytes`/`excludedHosts`/`bypassModifier` capture.offer itself reads, so the two can never drift. `rulesVersion` is a constant `1` — there is no persisted revision counter yet (nothing writes `rules.*` outside this process's own lifetime to need one across a restart), and the extension already re-fetches on `event.settings.changed` regardless of what this number does; noted in case a real counter becomes worth adding later. | `rpc/dispatcher.cpp` | `rulesVersion` is a placeholder constant | — |
| D1 | Pairing prompt is `EnvAutoApprover` (needs `VELOX_PAIR_AUTO=1`) | `rpc/pairing.hpp`, `main.cpp` | A GUI dialog / `org.freedesktop.Notifications` approver is integration work | Build step 7 (systemd + notifications) |
| D1 | Pairing prompt is `EnvAutoApprover` (needs `VELOX_PAIR_AUTO=1`) | `rpc/pairing.hpp`, `main.cpp` | A GUI dialog / `org.freedesktop.Notifications` approver is integration work | the `org.freedesktop.Notifications` half of build step 7 — the systemd half closed as D11 below |
| — | **D1, checked this pass, not attempted:** `libdbus-1-dev` (or `libsystemd-dev` for `sd-bus`) has no headers installed in this build environment — only the runtime `.so`s (`dpkg -l`/`apt-cache policy` confirm `libdbus-1-3` present, `libdbus-1-dev` not, "Candidate" available but not installed). A real notification-backed approver needs one of those linked into `veloxd`, which is a new build dependency for `daemon/CMakeLists.txt` (`find_package`/`pkg_check_modules`) and — since packaging manifests need to know about it too — arguably a decision to surface rather than something to reach for silently mid-session. `PairingApprover::approve()` is also still synchronous by shape (its own doc comment already says so: "the real notification-backed approver will run async and is not this shape") — swapping it for the async pattern this session built for `download.probe` (`rpc::TaskActionPort` + the server-layer deferred-reply special-case) is the right shape once there's a real implementation to justify the churn; reshaping the interface with nothing behind it yet would just be churn. Left `EnvAutoApprover` in place rather than build a fragile hand-rolled D-Bus wire client to avoid the missing headers — a broken pairing approver is worse than an honest stub. | `rpc/pairing.hpp` | missing dev headers + an undiscussed new dependency | once `libdbus-1-dev`/`libsystemd-dev` is available and the dependency is approved |
| ~~D2~~ | **Closed** — `download.probe` is real on both transports. It's genuinely async (the engine's probe pool, up to the schema's 30s `x-deadlineMs`) and so cannot fit `VeloxDispatcher::on_download_probe`'s synchronous `HandlerResult<T>` return — `uds_server.cpp`/`ws_server.cpp` special-case `"download.probe"` before the generic `dispatch()`, exactly the way they already special-case `session.hello`/`session.subscribe`, and queue the reply whenever the callback fires. `rpc::TaskActionPort::probe_now` (kept in proto/std terms, no `vdm::net::*`, so `veloxd_rpc` never needs `core/include`'s vdm headers) is what both transports call; `sched::Scheduler::probe_now` is the implementation — builds a `vdm::net::ProbeRequest`, runs it on the engine's probe pool, maps a failure to `-32013 ProbeFailed` (with `data.httpStatus` when there was one), and fills `suggestedCategoryId`/`suggestedSaveDir` with a plain extension match against the categories table (not the real rules engine — that's still D3). Verified live: a real probe answers in ~5ms; a bad host maps to `-32013`; a connection issuing a 10s `slow-loris` probe does not block a second connection's `download.list` (answered in ~1ms) — confirms the async design actually keeps the loop free, not just compiles. | `rpc/task_action_port.hpp`, `rpc/{uds_server,ws_server}.{hpp,cpp}`, `sched/scheduler.{cpp,hpp}` | — | done |
| D3 | Stub handlers for the rest: `grabber.*`, `media.*` | `rpc/dispatcher.cpp` | HLS/DASH grabber and media-variant support don't exist anywhere in this build yet — a bigger feature than a store-wiring pass | M4 territory, per AGENT-DAEMON.md |
@@ -26,4 +26,5 @@ close. Kept here (not buried in commit messages) so the next pass can see them a
| ~~D4b~~ | **Closed** — `download.pause`/`resume`/`start`/`cancel` and `queue.start`/`stop` all drive the scheduler now, and apply *immediately* (not deferred to the next tick — pausing/resuming/cancelling a live transfer can't wait up to 1s, and per ADR 0013 §3 the governor never touches a user-owned pause on its own). New `rpc::TaskActionPort` interface (owned by `rpc/`, implemented by `sched::Scheduler`) is the seam dispatcher.hpp depends on instead of `sched/scheduler.hpp` directly — avoids a real `veloxd_rpc` <-> `veloxd_sched` circular library dependency (`veloxd_sched` already links `veloxd_rpc` for `EventHub`). `Scheduler::user_pause/resume/start/cancel` + `pause_queue` engine-call-then-eager-transition, matching `tick()`'s existing `to_pause` pattern. Fixed a real bug hit while building this: `transition()` always overwrote `pause_reason` to NULL when the engine's own delayed pause-ack callback arrived with no explicit reason, clobbering whatever the actual initiator (user or governor) had just written — now it preserves the stored reason when none is supplied. Verified against real `veloxd` + `tools/testserver`: pausing a live single-segment throttled transfer freezes `downloadedBytes`, resume continues it from that point, cancel stops it; `queue.stop(pauseRunning:true)` pauses the queue's running task immediately. NOTE: `download.start`'s contract "a task in 'queued' jumps its queue" (priority bump) is not implemented — admission is still plain FIFO by `created_at`. | `sched/scheduler.{cpp,hpp}`, `rpc/task_action_port.hpp`, `rpc/dispatcher.{hpp,cpp}`, `store/queues.{cpp,hpp}` | — | done, except the queue-jump priority bump noted above |
| ~~D5~~ | **Mostly closed** — `rpc/event_hub` fans out per-subscription; `session.subscribe` on both transports registers/updates/tears down a real subscription; `Scheduler::transition()` publishes `event.task.state` (with `previousState`) on every state change, scheduler-driven or engine-reported; `dispatcher::on_download_add` publishes `event.task.added`; a 250 ms timer batches `Scheduler::progress_snapshot()` into one `event.task.progress` array per AGENT-DAEMON.md item 5 / the schema's `x-maxRateHz: 4`. Verified live end to end. | — | `event.task.removed` has no source yet (`download.remove` is D3); `event.speed.global`, `event.notify`, `event.auth.required`, `event.settings.changed`, `event.grabber.progress` are unpublished — each lands with its owning handler | as each owning D3 handler lands |
| ~~D6~~ | **Closed** — engine numbers now reach the store: `Scheduler::tick()` probes (`EnginePort::probe`) before every `start()`, persisting `sizeBytes`/`resumable`/validators via `Tasks::set_probe_result` before a byte moves; `Scheduler::persist_progress()` (called from `progress_snapshot()` *and* once more from `on_engine_state` right before `release()`/unmap on every terminal transition) writes `downloadedBytes`/`speedBps`/`segments`/`segmentDetail` from the engine's `Progress`, so a task that finishes between two 250 ms ticks (the common case for anything small or fast) still leaves real numbers instead of the pre-persistence defaults. `TaskSummary.segments` is sourced from `segments.size()` when the task has any (matching what actually lands in `segmentDetail`, per the schema's "exactly `segments` entries"), falling back to the engine's `effective_segments` (budget slots *held*, not necessarily physical range count — see `core/include/vdm/task/download.hpp`'s `Progress` comment) only pre-segmentation. `Tasks::set_final_bytes` tops up `on_finished`'s byte count as a last-resort backstop. Migration `0002` adds `speed_bps` to both `tasks` and `segments`, and fixes `segments.state`'s CHECK to include `'pending'` (0001 omitted it, so a pre-connect snapshot could never be written). Verified against real `veloxd` + `tools/testserver` (not just unit tests): `download.list`/`download.get` correct immediately after completion and after a daemon restart. | `sched/scheduler.{cpp,hpp}`, `store/{tasks,segments}.{cpp,hpp}`, `store/migrations/0002_*.sql` | — | done |
| ~~D11~~ | **Closed — build order items 7 (the systemd half) and 9: `velox-nmhost`, socket activation, the systemd user units, and `velox(1)`.** `nmhost/src/main.cpp` (185 lines): a `poll()`-driven byte pump between Firefox's native-messaging framing on stdio (4-byte native-byte-order length prefix) and `veloxd`'s own NDJSON framing on the Unix socket — reframes each direction, no JSON parsing, no retry/backoff, exits the moment either side closes. Deliberately dependency-free (no `veloxd_*` library, no `nlohmann_json`) since it runs unconfined outside Firefox's sandbox whatever the packaging format. Two real bugs found and fixed while getting the integration test to actually pass rather than hang: (1) never set the pumped fds non-blocking, so the "drain what's available" read loop blocked on its own second `read()` instead of returning to `poll()`; (2) stdin and stdout are two different descriptors (0 and 1), not one — an early draft polled `POLLOUT` on fd 0, which is opened read-only, so EOF/writability were never both observable through the same `pollfd` entry. Both are exactly the class of bug a "trivial pump" invites and unit tests over the real binary (not just its helper functions) exist specifically to catch. `packaging/nativehost/com.velox.host.json` + its own `README.md` supersede `AGENT-DAEMON.md`'s stale "four locations" (spike S1 / ADR 0003 found only three are real — the fourth, `~/snap/firefox/common/.mozilla/...`, is not read by snap Firefox at all) and spell out the per-user-manifest / postinst implication for PKG/QA. `EnginePort`-style: `rpc/systemd_activation.cpp` is a from-scratch `sd_listen_fds()` (env vars only, no `libsystemd` link — `LISTEN_PID`/`LISTEN_FDS`, fd 3) that `UdsServer::start()` checks first, skipping its own create/bind/chmod/listen when systemd already bound the socket; `packaging/systemd/velox.socket` + `velox.service` are the unit pair, verified both by `systemd-analyze verify` and by an actual fork/dup2/execve simulation of the activation handshake (a real `session.hello` round-tripped over the handed-off fd with no `bind()` ever called inside the daemon for that run). `velox.service` deliberately skips `ProtectSystem=`/`ProtectHome=`/`ReadWritePaths=` — `saveTo.allowedRoots` is user-configurable to anywhere on the filesystem, and a sandbox here would turn a legitimately-configured save location into an opaque `EROFS`/`EACCES` instead of the daemon's own clear `-32011`. `cli/man/velox.1` documents the CLI as it actually exists today (`add`/`ls`/`pause`/`resume`/`rm`, `--json`, the three-tier `queue`/`settings` subcommands `AGENT-DAEMON.md` build step 8 originally sketched are not implemented in `cli/src/main.cpp` yet, so the page doesn't claim they are) — checked warning-free with `groff -mandoc -ww -z`. | `nmhost/{CMakeLists.txt,src/main.cpp,tests/}`, `daemon/src/rpc/{systemd_activation.{hpp,cpp},uds_server.cpp}`, `packaging/{nativehost,systemd}/`, `cli/man/velox.1` | — | done |
| — | ~~Observed, not fixed (CORE, not this lane)~~ — **routed to CORE by the user.** `vdm::task::Progress.speed_bps` reads back as `0` for the whole lifetime of a live, real (non-fake) throttled download, despite `downloadedBytes` visibly advancing between polls — `core/src/task/download_task.cpp`'s per-worker EWMA never seems to produce a nonzero aggregate in this build. DAEMON passes `EnginePort::progress()`'s `speed_bps` straight through (`Scheduler::persist_progress`); nothing in this lane drops it. Still reproduces in the D4b live checks above (0 throughout a paused/resumed/cancelled transfer whose `downloadedBytes` visibly moved) — not re-filed, since it's already CORE's. |
+37
View File
@@ -0,0 +1,37 @@
#include "rpc/systemd_activation.hpp"
#include <unistd.h>
#include <cstdlib>
#include <string>
namespace velox::daemon::rpc {
namespace {
constexpr int kListenFdsStart = 3; // SD_LISTEN_FDS_START
} // namespace
int systemd_activated_fd() {
const char* pid_env = std::getenv("LISTEN_PID");
const char* fds_env = std::getenv("LISTEN_FDS");
int fd = -1;
if (pid_env != nullptr && fds_env != nullptr) {
try {
if (std::stol(pid_env) == static_cast<long>(::getpid()) && std::stol(fds_env) == 1) {
fd = kListenFdsStart;
}
} catch (...) {
// Malformed env from something other than systemd; treat as not activated.
}
}
// Contract: consumed once, then cleared, so a value meant for veloxd is never
// mistaken for one meant for a process it might itself exec later.
::unsetenv("LISTEN_PID");
::unsetenv("LISTEN_FDS");
::unsetenv("LISTEN_FDNAMES");
return fd;
}
} // namespace velox::daemon::rpc
+20
View File
@@ -0,0 +1,20 @@
#pragma once
// Minimal sd_listen_fds(3) reimplementation — one function, no libsystemd dependency, for
// the one fd velox.socket ever hands us. See velox.socket / velox.service in
// packaging/nativehost's systemd unit pair: the socket unit binds
// $XDG_RUNTIME_DIR/velox/velox.sock itself (before veloxd ever runs, so the very first
// connection attempt after boot is queued by the kernel rather than refused) and execs
// veloxd with that listening fd already open at fd 3, LISTEN_FDS=1, LISTEN_PID=<our pid>.
namespace velox::daemon::rpc {
// The systemd-activated listening socket fd, or -1 if this process was not socket-
// activated (LISTEN_PID doesn't match our pid, or LISTEN_FDS is unset/not exactly 1 — more
// than one would mean a unit file mismatch, since veloxd only ever asks for one socket).
// Clears LISTEN_PID/LISTEN_FDS from the environment on the way out either way, per
// sd_listen_fds's own contract, so a value meant for us is never mistaken for one meant for
// a process veloxd might itself exec later.
int systemd_activated_fd();
} // namespace velox::daemon::rpc
+17
View File
@@ -12,7 +12,10 @@
#include <nlohmann/json.hpp>
#include <fcntl.h>
#include "rpc/event_loop.hpp"
#include "rpc/systemd_activation.hpp"
#include "version.hpp"
namespace velox::daemon::rpc {
@@ -74,6 +77,20 @@ UdsServer::~UdsServer() {
}
std::error_code UdsServer::start() {
// velox.socket (systemd user unit, socket activation): the unit binds this path itself
// before veloxd ever runs and hands the already-listening fd over at fd 3 — the first
// connection after boot is queued by the kernel rather than refused, and there is no
// window where a client sees ECONNREFUSED while the daemon is still starting. Skips
// create/bind/chmod/listen entirely; the socket file's lifecycle (including removal on
// stop) belongs to the unit, not to us, so bound_ stays false.
if (const int activated = systemd_activated_fd(); activated >= 0) {
::fcntl(activated, F_SETFL, O_NONBLOCK);
::fcntl(activated, F_SETFD, FD_CLOEXEC);
listen_fd_ = activated;
loop_.add_fd(listen_fd_, kRead, [this](int, unsigned) { on_listener_readable(); });
return {};
}
if (path_.size() + 1 > sizeof(sockaddr_un::sun_path)) return errc(ENAMETOOLONG);
const int fd = ::socket(AF_UNIX, SOCK_STREAM | SOCK_NONBLOCK | SOCK_CLOEXEC, 0);
+1
View File
@@ -24,6 +24,7 @@ veloxd_test(sched_scheduler LIBS veloxd_sched veloxd_rpc)
veloxd_test(event_hub LIBS veloxd_rpc)
veloxd_test(store_categories_queues LIBS veloxd_store)
veloxd_test(single_instance LIBS veloxd_rpc)
veloxd_test(systemd_activation LIBS veloxd_rpc)
veloxd_test(dispatcher_settings LIBS veloxd_rpc veloxd_store)
veloxd_test(capture_offer LIBS veloxd_rpc veloxd_store)
veloxd_test(dispatcher_misc LIBS veloxd_rpc veloxd_store)
+53
View File
@@ -0,0 +1,53 @@
// systemd_activated_fd(): the LISTEN_PID/LISTEN_FDS contract, without a real systemd.
#include <unistd.h>
#include <cstdlib>
#include <string>
#include "check.hpp"
#include "rpc/systemd_activation.hpp"
using namespace velox::daemon::rpc;
namespace {
void set_env(const char* k, const std::string& v) { ::setenv(k, v.c_str(), 1); }
} // namespace
void run() {
// Not activated: neither var set.
::unsetenv("LISTEN_PID");
::unsetenv("LISTEN_FDS");
CHECK_EQ(systemd_activated_fd(), -1);
// LISTEN_PID for a different process: not us, so not activated.
set_env("LISTEN_PID", std::to_string(::getpid() + 1));
set_env("LISTEN_FDS", "1");
CHECK_EQ(systemd_activated_fd(), -1);
// Consumed regardless of the outcome — a stale value from some other process's
// exec chain must not leak into what veloxd checks next time.
CHECK(::getenv("LISTEN_PID") == nullptr);
CHECK(::getenv("LISTEN_FDS") == nullptr);
// Our own pid, LISTEN_FDS=1: activated, fd 3 (SD_LISTEN_FDS_START).
set_env("LISTEN_PID", std::to_string(::getpid()));
set_env("LISTEN_FDS", "1");
CHECK_EQ(systemd_activated_fd(), 3);
CHECK(::getenv("LISTEN_PID") == nullptr);
// Our own pid but LISTEN_FDS=2: a unit file mismatch (veloxd only ever asks for one
// socket) — refuse rather than guess which of two fds is the right one.
set_env("LISTEN_PID", std::to_string(::getpid()));
set_env("LISTEN_FDS", "2");
CHECK_EQ(systemd_activated_fd(), -1);
// Garbage LISTEN_FDS: not activated, not a crash.
set_env("LISTEN_PID", std::to_string(::getpid()));
set_env("LISTEN_FDS", "not-a-number");
CHECK_EQ(systemd_activated_fd(), -1);
::unsetenv("LISTEN_PID");
::unsetenv("LISTEN_FDS");
}
TEST_MAIN()
+85
View File
@@ -0,0 +1,85 @@
# 08 — Porting guide (macOS first, Windows later)
Read `docs/adr/0020-cross-platform-strategy.md` first; this file is the concrete inventory
it refers to. **Rule: Linux behaviour never changes. `#ifdef` never appears in logic.**
## Directory layout
Platform backends live beside the code they serve, one directory per OS:
```
core/src/io/platform/linux/file_ops.cpp core/src/io/platform/macos/file_ops.cpp
core/src/io/platform/file_ops.hpp # the seam — no #ifdef, no OS types
daemon/src/rpc/platform/linux/{wakeup,peercred,instance_lock,runtime_dir}.cpp
daemon/src/rpc/platform/macos/{...}.cpp
daemon/src/rpc/platform/*.hpp # the seams
cmake/platform.cmake # VELOX_OS_* + source selection
```
`cmake/platform.cmake` sets exactly one of `VELOX_OS_LINUX`, `VELOX_OS_MACOS`,
`VELOX_OS_WINDOWS`, and exposes `velox_platform_sources(<target> <dir>)` which adds
`<dir>/platform/<os>/*.cpp`. Selection happens there and nowhere else.
## The seams
Five interfaces cover the whole port. Signatures are indicative, not binding — the owning
lane settles them in Phase 0.
| Seam | Interface | Why |
|---|---|---|
| File preallocation | `Result<void> preallocate(int fd, uint64_t bytes)` | `posix_fallocate` is Linux/glibc |
| Cache advice | `void advise_dontneed(int fd, uint64_t off, uint64_t len)` | `posix_fadvise` has no macOS equivalent |
| Durable flush | `Result<void> flush_durable(int fd)` | `fdatasync` vs `F_FULLFSYNC` |
| Loop wakeup | `class Wakeup { int pollfd(); void signal(); void drain(); }` | `eventfd` is Linux-only |
| Peer identity | `Result<PeerId> peer_of(int fd)` | `SO_PEERCRED` vs `LOCAL_PEERCRED` |
| Single instance | `Result<Lock> acquire(string_view runtime_dir)` | abstract sockets are Linux-only |
| Runtime dir | `string runtime_dir()`, `string data_dir()` | XDG vs `~/Library` |
## API mapping
| Linux (today) | macOS | Windows (later) | Note |
|---|---|---|---|
| `posix_fallocate(fd,0,n)` | `fcntl(F_PREALLOCATE)` then `ftruncate(n)` | `SetFileValidData` / `SetEndOfFile` | macOS needs the `ftruncate`; `F_PREALLOCATE` alone does not set size. Fall back to `ftruncate` on failure, exactly as the Linux path already does for `EOPNOTSUPP` |
| `posix_fadvise(DONTNEED)` | no equivalent — **no-op** | `FILE_FLAG_NO_BUFFERING` | Do **not** substitute `F_NOCACHE`: it changes caching for the whole descriptor, not a written range. A no-op is honest; record it |
| `fdatasync(fd)` | `fcntl(fd, F_FULLFSYNC)`, fall back to `fsync` | `FlushFileBuffers` | `fsync` on macOS does **not** guarantee the drive flushed. `.veloxpart` resume integrity depends on this — use `F_FULLFSYNC` |
| `pwrite` | same | `WriteFile` + `OVERLAPPED` | POSIX, no work |
| `O_NOFOLLOW` | same | `FILE_FLAG_OPEN_REPARSE_POINT` | POSIX, no work |
| `eventfd` | self-pipe (`pipe2`/`O_NONBLOCK|O_CLOEXEC`) | event object | `poll(2)` already used, so a pipe read-end drops straight in |
| `timerfd` | `poll()` timeout computed from the next deadline | waitable timer | Simplest port: the loop already has a deadline set |
| `SO_PEERCRED` + `struct ucred` | `getpeereid(fd,&uid,&gid)` | named-pipe token | Same-UID check is the security property; keep it |
| abstract socket `\0velox-daemon-<hash>` | socket file in the runtime dir + `flock(LOCK_EX|LOCK_NB)` | named mutex | macOS has no abstract namespace. Must unlink stale sockets on start — the abstract version got that free |
| `$XDG_RUNTIME_DIR` | `$TMPDIR` (per-user, already private) | `%LOCALAPPDATA%` | macOS has no XDG runtime dir |
| `$XDG_DATA_HOME` | `~/Library/Application Support/Velox` | `%APPDATA%` | |
| `libsecret` / Secret Service | Keychain (`Security.framework`) | DPAPI / Credential Manager | Behind the credential-store seam. CLAUDE.md §4 still applies: never SQLite, never logs |
| `systemd` user units | `launchd` plist (`~/Library/LaunchAgents`) | Service/Task Scheduler | Phase 3 |
## Build dependencies
| Ubuntu | macOS (Homebrew) |
|---|---|
| `qt6-base-dev`, `qt6-svg-dev`, `qt6-tools-dev` | `qt@6` |
| `libcurl4-openssl-dev` | system libcurl, or `curl` |
| `libsqlite3-dev` | system sqlite, or `sqlite` |
| `libssl-dev` | `openssl@3` (set `OPENSSL_ROOT_DIR`) |
| `libsecret-1-dev` | **none** — Keychain is in the SDK |
| `nlohmann-json3-dev` | `nlohmann-json` |
| `nodejs`, `npm` | `node` |
The root `CMakeLists.txt` currently does `pkg_check_modules(LIBSECRET REQUIRED ...)`
unconditionally. That must become Linux-only, or macOS cannot configure at all. It is the
single hard blocker for a first macOS build.
## What does not change
`contracts/` and both generated clients, `tools/mockd`, `tests/conformance`,
`tools/testserver`, the whole extension, and every hostile-mode expectation. If a port
tempts you to change a fixture or a schema, stop — that is a contract change and it goes
through PROTO.
## Verification gates
- **Phase 0 done:** Ubuntu suite still 57/57, and `git diff` shows only moves behind seams.
- **Phase 1 done:** `veloxd`, `velox`, `libveloxcore` build on macOS; core unit tests pass.
- **Phase 2 done:** conformance (mockd **and** live veloxd) and the engine hostile-mode
matrix pass on macOS; one real download completes with a matching SHA-256.
- **Phase 3 done:** `.dmg` or Homebrew formula installs and runs on a clean machine.
@@ -0,0 +1,107 @@
# ADR 0018 — Nullable optional fields: absent vs. explicit null
**Status:** accepted · **Date:** 2026-09-13 · **Lane:** PROTO
**Prompted by:** a DAEMON report against `download.update`: the generated C++ parser gives
`VeloxDispatcher` no way to tell "the caller left this field alone" from "the caller wants
it cleared," so `download.update` and (the moment a nullable `SettingKey` exists)
`settings.set` can set a nullable field but never clear it back to `null`.
## Context
`download.update`'s `patch` object documents the convention plainly: "Only the present
fields change. An explicit null clears a nullable field." That is a deliberate, already-
committed wire contract — not something up for redesign here. The gap is one layer down:
`contracts/codegen/gen_cpp.py`'s `emit_field_parse` collapses "key absent" and "key present
with value `null`" to the same `std::nullopt`, on purpose, and the comment says so:
> Absent and null mean the same thing: the field is not set. A client that omits a
> nullable field and one that sends null are treated identically on purpose.
That collapse is *correct* for the common case — most nullable-optional fields are on
create-style params (`DownloadSpec.saveDir`, `.categoryId`, …) where there is no existing
value to distinguish "never set" from "explicitly cleared" in the first place; either way
the daemon just uses a default. It is wrong specifically for **patch-style** params, where
a field can already hold a value and the caller needs to say which of two different things
they mean: "leave it" or "clear it."
The schema IR (`schema_ir.py`) already tracks `required` and `nullable` as two independent
booleans per `Field`, so the information needed to make this distinction exists all the way
through parsing — `emit_field_parse` just doesn't act on it. Only `download.update`'s
`patch` object is affected today (`filename`, `saveDir`, `categoryId`, `queueId`,
`description`, `segments`, `bufferBytes`, `checksum` — all eight of its fields are
nullable-and-optional with exactly this "leave vs. clear" meaning). No `SettingKey` is
nullable yet, so `settings.set` has no live instance of the bug, but the same shape
(`values` patches an existing bag) means the first nullable settings key will hit the exact
same gap.
## Decision
**A JSON-null-aware optional, opt in per field via a new `x-clearable: true` annotation —
not a blanket rule and not a companion "clear list" field.**
- New per-field schema annotation, `x-clearable: true`, valid only on a field whose type
already includes `null` (schema error otherwise — clearable implies nullable). Marks
"this field distinguishes absent from explicit null"; every other nullable-optional field
keeps today's collapse.
- The generated C++ type for a `x-clearable` field becomes `std::optional<std::optional<T>>`:
outer `nullopt` = absent (leave unchanged), outer engaged with an inner `nullopt` =
explicit `null` (clear it), outer engaged with an inner value = set it. One field, three
states, no parallel bitset to keep in sync and no second field to forget to check.
- `emit_field_parse` for such a field stops folding `is_null()` into "absent": absent skips
the assignment (outer stays `nullopt`); present-and-null assigns an engaged-but-empty
inner optional; present-and-valued parses normally into the inner optional. Every other
field's codegen (the `required`/`nullable`-but-not-`clearable` majority) is unchanged.
- TypeScript needs no generator change: `field?: T | null` already round-trips this exactly
the way JSON does — an omitted key serializes as absent, `null` serializes as `null`, and
`"field" in obj` / `obj.field === null` already distinguish the three states natively.
This gap is a C++-generator-only problem.
- Applies now to `download.update`'s eight `patch` fields. `Settings` gets no annotation
today (nothing nullable to mark); the day a nullable `SettingKey` is added, it gets
`x-clearable: true` in the same PR, not left to rediscover this ADR.
## Versioning
Per ADR 0015: this retypes a generated C++ field (`optional<T>` -> `optional<optional<T>>`)
with the wire byte-for-byte unchanged — a client sending the same JSON parses correctly
either way. **Minor bump, with a migration note** for anyone reading `patch.filename` et al.
directly (unwrap twice: check the outer, then the inner). Not major; `session.hello`'s
major-only check must not refuse a wire-compatible peer over a binding-only change.
## Consequences
- `on_download_update` (DAEMON, not this lane) can finally implement "explicit null
clears": read the outer optional for presence, the inner for clear-vs-value, exactly the
three states the schema already promised.
- The collapse comment in `emit_field_parse` stays as the default behavior and gets a
pointer to this ADR for the opt-in exception, instead of being read as an oversight.
- Implementation (schema annotation support in `schema_ir.py`, the `gen_cpp.py` emission
change above, regenerating `core/generated/`, the `x-clearable: true` annotations on
`download.update`'s eight fields, the VERSION bump and migration note) is **not** done in
this change — recorded here so DAEMON isn't blocked on relitigating the design, tracked as
its own PROTO PR per the normal contracts process (schema + regenerated code + fixtures +
VERSION bump together, CLAUDE.md §2).
## Alternatives rejected
**An explicit clear list** (e.g. `patch.clearFields: ["categoryId", …]`, plain non-nullable
`optional<T>` fields otherwise). Rejected: the wire contract "an explicit null clears a
nullable field" is already written into `download.update`'s schema description and is what
DAEMON built against — this would be a real, disruptive wire redesign to route around a
generator gap, not a fix for it. It also doesn't compose: every patch-shaped object gains a
second array to keep in sync with the first, by hand, forever.
**A parallel "which fields were present" bitset** (struct of `optional<T>` fields plus a
sibling presence-flags struct or bitset). Rejected: two things to check per field instead
of one, and nothing stops a caller from reading the optional and forgetting the presence
bit — exactly the class of bug this ADR exists to close.
**Apply the tri-state to every `nullable && !required` field automatically**, using the IR
flags already present, no annotation needed. Rejected: `emit_field_parse` only backs
`parse<T>()`, used for *params* types the daemon receives — but the conformance C++ runner
also instantiates `parse<T>()` for **result** types (round-tripping golden fixtures), and
plenty of those are nullable-optional with no patch semantics at all (`TaskSummary.effectiveUrl`,
"null until the first probe succeeds" — a plain nullable value, not a leave-or-clear
choice). Blanket application would retype those too, forcing every read site across the
daemon that already does `if (summary.effectiveUrl)` into an unwanted double-unwrap for a
distinction that field doesn't have. Opt-in keeps the blast radius at exactly the fields
that need it.
@@ -0,0 +1,45 @@
# ADR 0019 — The live-`veloxd` conformance runner is a required check, as-is
**Status:** accepted · **Date:** 2026-09-12 · **Lane:** PKG/QA
## Context
`tests/conformance/run.sh` step 3b (ADR 0014's `conformance` ctest, already required
per `.github/BRANCH_PROTECTION.md`) replays every fixture against a real, isolated
`veloxd` it builds and starts — not just mockd, which only proves the TS client and the
fixtures agree with each other. This is the runner that can catch `veloxd` disagreeing
with its own contract, and it is unconditional in `run.sh` (`set -euo pipefail`, no
skip flag): it already runs, and already blocks, inside the `conformance` job.
What was open was whether to treat that as a settled, defended gate or as something
still provisional while `daemon/docs/deferrals.md`'s D1-D4b stub handlers were excused
via `veloxd-xfail.json`. PROTO's update: 57/57 fixtures pass on `main`, and the xfail
allowlist is down to 18 entries from 34 as DAEMON lands the deferred handlers behind
them.
## Decision
The live-`veloxd` conformance runner stays required — no change to CI is needed, since
it already runs inside the already-required `conformance` job (ADR 0014). What this ADR
records is the standing: PKG/QA is not carving out an exception, a `continue-on-error`,
or a separate advisory job for it while the xfail list shrinks. A regression here fails
the same required check a schema mismatch would.
Verified, not assumed: `tests/conformance/veloxd-xfail.json` has 18 entries as of this
ADR (`python3 -c "import json; print(len(json.load(open('tests/conformance/veloxd-xfail.json'))))"`).
Each remaining entry excuses one still-stubbed handler on `deferrals.md`'s D-list, not a
real disagreement between `veloxd` and its contract — `tests/conformance/README.md`
already draws that line (an entry for anything else is a `run.sh`-detected "xfail entry
unexpectedly passed" or a straight failure, not a quiet pass).
## Consequences
- No `ci.yml` or `BRANCH_PROTECTION.md` change: `conformance` was already listed
required, and this runner was already inside it.
- The xfail list is a visible, shrinking number, not a static allowance — as DAEMON
clears more of `deferrals.md`'s D-list, entries come out of
`tests/conformance/veloxd-xfail.json`, and `replay.ts` fails loudly (per
`applyXfail`'s "unexpectedly passed" check) if one is left in after its handler ships.
- Nothing here changes who owns what: `veloxd-xfail.json` and `run.sh` stay PROTO's;
PKG/QA's role is the branch-protection policy this ADR confirms, not the runner
itself.
+72
View File
@@ -0,0 +1,72 @@
# ADR 0020 — Porting to macOS (and later Windows) without forking the tree
Status: **accepted** · Supersedes nothing · Applies to every lane
## Context
The project was built Ubuntu-first and says so in its own project description. A survey of
`main` (2026-09-15, 57/57 green) found the Linux-specific surface is far smaller than the
Ubuntu-first framing suggests — roughly ten files, all of them already isolated at the
bottom of the stack:
| Area | Linux-only thing | Files |
|---|---|---|
| `core/io` | `posix_fallocate`, `posix_fadvise`, `fdatasync` | `sparse_file.cpp` |
| `core/meta` | `fdatasync` | `veloxpart.cpp` |
| `daemon/rpc` | `eventfd` | `event_loop.{hpp,cpp}`, `main.cpp` |
| `daemon/rpc` | `timerfd` | `main.cpp`, `sched/scheduler.hpp` |
| `daemon/rpc` | `SO_PEERCRED` / `struct ucred` | `uds_server.{hpp,cpp}` |
| `daemon/rpc` | abstract-namespace socket lock | `single_instance.{hpp,cpp}` |
| `daemon/rpc` | `$XDG_RUNTIME_DIR` layout | `runtime_dir.{hpp,cpp}` |
| build | `libsecret-1` is `REQUIRED` at the root | `CMakeLists.txt` |
Everything else is already portable: the event loop is `poll(2)` (POSIX, not `epoll`), the
transfer engine is libcurl, the GUI is Qt 6, `tools/mockd` and `tests/conformance` are
Node, `tools/testserver` is stdlib Python, and the extension is a WebExtension.
## Decision
**One tree, one logic, per-OS backends behind seams.**
1. **No `#ifdef` in logic.** Platform differences live behind a narrow interface, with one
implementation file per OS. A reader of `sparse_file.cpp` must not need to know which OS
they are on. `#ifdef` is allowed only inside a `platform/<os>/` file.
2. **Linux is the reference implementation.** A port may never change Linux behaviour. The
gate is mechanical: the full suite stays green on Ubuntu, and the Linux backend keeps
the same syscalls it uses today. If a port needs a semantic change, that is an ADR of
its own, not a port commit.
3. **Seams are introduced by the owning lane, on Linux, before any port work.** CORE
introduces `core`'s seams; DAEMON introduces `daemon`'s. This is a pure refactor: move
the existing Linux code behind the interface unchanged, prove the suite is still green.
4. **A new lane, PORT, owns only the per-OS backends** — `**/platform/<os>/**`. It never
writes logic, and it does not own `cmake/`: the central OS detection module is root
build infrastructure and stays with PKG/QA, who must land it during Phase 0 because
Phase 0's own seams consume it. PORT owning it would deadlock — PORT cannot start until
Phase 0 lands, and Phase 0 cannot select platform sources without it. That keeps CLAUDE.md §1 intact: CORE
still owns `core/`'s logic, DAEMON still owns `daemon/`'s, and macOS work can proceed in
parallel without cross-lane writes.
5. **`VELOX_OS_*` is set once, centrally**, in `cmake/platform.cmake` (owned by PKG/QA),
and platform sources are selected there — not by globbing, not per-target ad hoc.
6. **Optional dependencies are gated, never removed.** `libsecret` stays `REQUIRED` on
Linux and is replaced by Keychain on macOS behind the same credential-store seam.
## Phases
- **Phase 0 — on Ubuntu, by CORE + DAEMON.** Introduce the seams and move today's Linux
code behind them. Zero behaviour change; 57/57 stays green. No macOS code exists yet.
- **Phase 1 — on macOS, by PORT.** Implement the macOS backends until `veloxd`, `velox`
and `libveloxcore` build and their unit tests pass.
- **Phase 2 — parity.** `tools/testserver`, conformance and the engine's hostile-mode
matrix pass on macOS. The GUI builds against Qt 6 for macOS.
- **Phase 3 — packaging.** `.dmg`/Homebrew, `launchd` instead of `systemd`, and the macOS
native-messaging manifest locations. Deliberately last.
## Consequences
- Windows later is the same shape: a third backend directory, no new strategy. The seams
chosen here are POSIX-flavoured but interface-level, so a Win32 backend is additive.
- CI must build both, or macOS rots silently. Until a macOS runner exists, Phase 1 is
verified by hand on the porting machine and the Linux gate stays authoritative.
- The cost is one indirection at the bottom of the I/O and RPC stacks. Measured against
the alternative — `#ifdef` drift, or a forked repo that diverges in a month — it is
cheap.
+5 -1
View File
@@ -43,7 +43,11 @@ Read `contracts/`, `core/include/`, `docs/`. Never write in `core/`, `gui/`, or
output. Build this early: it is how you test the daemon before the GUI exists.
9. **`velox-nmhost`** — 4-byte-length-prefixed stdio ⇄ Unix socket pump. **Under 300 lines,
zero business logic**, and it must exit cleanly when Firefox closes the pipe. Install
manifests to all four locations listed in `docs/05` §4.
manifests to the three real locations in `docs/05` §4 / `docs/adr/0003` — not four:
spike S1 found `~/snap/firefox/common/.mozilla/native-messaging-hosts/` (the intuitive
"inside the snap" path) is not actually read by snap Firefox, and corrected `docs/05` §4
down from its original four-location list. `packaging/nativehost/README.md` has the
current table.
## Definition of done (M1)
- Passes the full conformance suite as a server, over **both** transports.
+51
View File
@@ -0,0 +1,51 @@
# Agent brief — PORT (per-OS backends)
**Starts after Phase 0: CORE and DAEMON have landed the seams on Ubuntu.**
## You own
```
core/src/**/platform/<os>/** daemon/src/**/platform/<os>/**
packaging/macos/** packaging/windows/**
```
You may read everything. **You never write logic** — not in `core/src/io/*.cpp`, not in
`daemon/src/rpc/*.cpp`, not in `gui/`, never in `contracts/`.
## Read first
`docs/adr/0020-cross-platform-strategy.md`, then `docs/08-porting.md` — it has the seam
list, the API mapping and the dependency table. Then `CLAUDE.md`.
## The one rule that matters
**Linux behaviour never changes.** If your port needs a seam that doesn't exist, or a seam
whose shape is wrong, you do not widen it yourself — you file a request with the owning
lane (CORE for `core/`, DAEMON for `daemon/`) exactly as every other lane does. A port that
quietly edits shared logic is how one tree becomes two.
## Build order (macOS)
1. **Configure at all.** The root `CMakeLists.txt` requires `libsecret-1` unconditionally;
that must be Linux-gated before anything else compiles. PKG/QA owns that, and also owns
`cmake/platform.cmake` (the `VELOX_OS_*` detection and `velox_platform_sources()`); both
land in Phase 0, before you start. File anything else you need with them.
2. `core/` — `preallocate`, `advise_dontneed`, `flush_durable`. Use `F_FULLFSYNC`, not
`fsync`: `.veloxpart` resume correctness depends on a real flush.
3. `daemon/` — `Wakeup` (self-pipe), `peer_of` (`getpeereid`), `instance_lock` (socket file
+ `flock`, with stale-socket cleanup), `runtime_dir` (`$TMPDIR`, `~/Library/Application
Support/Velox`).
4. Unit tests green for both lanes.
5. `tools/testserver` (stdlib Python) and conformance (Node) — expected to run unchanged.
If they don't, that's a bug worth reporting, not patching around.
6. GUI against Qt 6 for macOS.
7. Packaging — `.dmg`/Homebrew, `launchd`, native-messaging manifest locations. Last.
## Definition of done (Phase 1–2)
- macOS builds `libveloxcore`, `veloxd` and `velox` with `-Wall -Wextra -Werror`.
- Core and daemon unit tests pass on macOS.
- One real download completes on macOS against `tools/testserver` with a SHA-256 that
matches the server's reference — the same gate the Linux vertical slice passed.
- `kill -9` mid-download, restart, resume completes and the checksum still matches.
- The Ubuntu suite is **still** green from the same commit.
## Do not
- Do not add `#ifdef` outside a `platform/<os>/` file.
- Do not change a fixture, a schema or a hostile-mode expectation to make macOS pass.
- Do not substitute `F_NOCACHE` for `posix_fadvise(DONTNEED)` — see `docs/08` for why.
- Do not start with packaging.
+28 -2
View File
@@ -19,7 +19,12 @@ import {
} from './context-menus.js';
import { MediaBridge, notifyTab } from './media-bridge.js';
import { createTransport, transportStorage, type TransportStatus, type VeloxTransport } from './transport/index.js';
import type { CaptureOfferParams, CaptureRules, DownloadSpec } from '../shared/protocol/index.js';
import {
SESSION_SUBSCRIBE_PARAMS_EVENTS_ITEM_VALUES,
type CaptureOfferParams,
type CaptureRules,
type DownloadSpec,
} from '../shared/protocol/index.js';
let transport: VeloxTransport | undefined;
let rules: CaptureRules = DEFAULT_CAPTURE_RULES;
@@ -75,10 +80,31 @@ async function refreshRules(): Promise<void> {
}
}
/**
* "Nothing is delivered until this is called" (session.subscribe's own description) —
* without it, event.task.progress et al. never reach this connection at all, no matter
* how many listeners bridge.ts registers locally. Requests the whole set every time
* because any popup/options document could open at any moment and none of them narrow
* per-tab; a fresh connection (first connect, or after a drop) starts with nothing
* subscribed until this runs again.
*/
async function subscribeToEvents(): Promise<void> {
try {
await mustTransport().call('session.subscribe', {
events: [...SESSION_SUBSCRIBE_PARAMS_EVENTS_ITEM_VALUES],
});
} catch {
// Best-effort; a reconnect (or the next event.settings.changed-driven refresh) retries.
}
}
function onTransportState(status: TransportStatus): void {
const detail = status.fatal ?? (status.needsPairing ? 'needs pairing' : '');
console.debug(`[velox] transport ${status.state}${detail ? ` — ${detail}` : ''}`);
if (status.state === 'connected') void refreshRules();
if (status.state === 'connected') {
void refreshRules();
void subscribeToEvents();
}
}
async function setOverride(override: 'auto' | 'ws' | 'uds'): Promise<void> {
+389
View File
@@ -0,0 +1,389 @@
// Runs the transport, the capture hook, and the popup event path against a REAL veloxd
// — not FakeDaemon. Everything else in this suite is faithful to the documented wire
// protocol, but "faithful" isn't "real"; this is what actually proves it.
//
// Requires VELOXD_BIN (path to a built veloxd) in the environment. Skips itself with a
// clear message otherwise, so `npm test` and CI (no daemon binary lying around) are
// unaffected. Run it like:
//
// VELOXD_BIN=/path/to/build/dev/bin/veloxd npx vitest run tests/live
//
// Each veloxd instance gets its own scratch XDG_RUNTIME_DIR/XDG_DATA_HOME/
// XDG_CONFIG_HOME/HOME (main.cpp's single-instance lock is keyed to the runtime dir, so
// this can run alongside another developer's or CI's own veloxd on the same machine).
// VELOX_PAIR_AUTO=1 stands in for the GUI's Allow-prompt approver during dev/test
// (rpc/pairing.hpp's EnvAutoApprover) — pairing itself is exercised for real, only the
// human click is stubbed.
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process';
import { mkdtempSync, mkdirSync, rmSync } from 'node:fs';
import { readFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath } from 'node:url';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { WebSocket as WsClient } from 'ws';
import { RpcError, TransportClosedError } from '../../src/background/transport/types.js';
import { WebSocketTransport, type WebSocketCtor, type WebSocketTransportDeps } from '../../src/background/transport/websocket.js';
import { CaptureHook } from '../../src/background/capture/index.js';
import type { OnHeadersReceivedDetails } from '../../src/background/capture/index.js';
import type { CaptureRules, DownloadSpec, TaskProgressEvent, TaskStateEvent } from '../../src/shared/protocol/index.js';
const VELOXD_BIN = process.env.VELOXD_BIN;
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '../../..');
const TESTSERVER_PY = join(REPO_ROOT, 'tools/testserver/testserver.py');
// A fixed moz-extension origin, used both as session.pair's extensionId and as the WS
// upgrade's Origin header — real Firefox sets the latter itself; ws's client needs it
// spelled out (docs/05 §4: the daemon refuses the upgrade without a moz-extension:// Origin).
const EXTENSION_ID = '11111111-2222-3333-4444-555555555555';
const ORIGIN = `moz-extension://${EXTENSION_ID}`;
class OriginWebSocket extends WsClient {
constructor(url: string) {
super(url, { origin: ORIGIN });
}
}
const CTOR = OriginWebSocket as unknown as WebSocketCtor;
function sleep(ms: number): Promise<void> {
return new Promise((r) => setTimeout(r, ms));
}
async function waitFor(cond: () => Promise<boolean> | boolean, timeoutMs: number, what: string): Promise<void> {
const deadline = Date.now() + timeoutMs;
for (;;) {
if (await cond()) return;
if (Date.now() > deadline) throw new Error(`timed out waiting for ${what}`);
await sleep(50);
}
}
interface VeloxdInstance {
proc: ChildProcessWithoutNullStreams;
scratch: string;
wsPort: number;
/** True once the process has actually exited, by signal or otherwise. Node only sets
* `proc.exitCode` for a normal exit — a signal-killed process reports its death via
* `signalCode` and an `exit` event instead, never a non-null `exitCode`. */
hasExited(): boolean;
kill(signal?: NodeJS.Signals): void;
}
async function startVeloxd(bin: string): Promise<VeloxdInstance> {
const scratch = mkdtempSync(join(tmpdir(), 'velox-live-'));
const runtime = join(scratch, 'rt');
const data = join(scratch, 'data');
const config = join(scratch, 'cfg');
const home = join(scratch, 'home');
mkdirSync(runtime, { mode: 0o700 });
mkdirSync(data, { recursive: true });
mkdirSync(config, { recursive: true });
mkdirSync(join(home, 'Downloads'), { recursive: true });
const proc = spawn(bin, [], {
env: {
...process.env,
VELOX_PAIR_AUTO: '1',
XDG_RUNTIME_DIR: runtime,
XDG_DATA_HOME: data,
XDG_CONFIG_HOME: config,
HOME: home,
},
});
let exited = false;
proc.on('exit', () => {
exited = true;
});
let stderr = '';
proc.stderr.on('data', (d) => {
stderr += String(d);
});
const portFile = join(runtime, 'velox', 'ws.port');
try {
await waitFor(async () => {
if (exited) throw new Error(`veloxd exited early (code ${proc.exitCode}, signal ${proc.signalCode}): ${stderr}`);
try {
await readFile(portFile);
return true;
} catch {
return false;
}
}, 10_000, 'veloxd to write ws.port');
} catch (e) {
proc.kill('SIGKILL');
rmSync(scratch, { recursive: true, force: true });
throw e;
}
const wsPort = Number((await readFile(portFile, 'utf8')).trim());
return {
proc,
scratch,
wsPort,
hasExited: () => exited,
kill(signal: NodeJS.Signals = 'SIGTERM') {
proc.kill(signal);
},
};
}
interface TestServerInstance {
proc: ChildProcessWithoutNullStreams;
baseUrl: string;
}
async function startTestServer(): Promise<TestServerInstance> {
const proc = spawn('python3', [TESTSERVER_PY, '--port', '0'], {});
let stdout = '';
let port: number | null = null;
proc.stdout.on('data', (d) => {
stdout += String(d);
const m = /^(\d+)\s*$/m.exec(stdout);
if (m) port = Number(m[1]);
});
await waitFor(() => port !== null, 5_000, 'testserver to print its port');
const baseUrl = `http://127.0.0.1:${port}`;
await waitFor(async () => {
try {
const res = await fetch(`${baseUrl}/__health`);
return res.ok;
} catch {
return false;
}
}, 5_000, 'testserver /__health');
return { proc, baseUrl };
}
function memDeps(init: { token?: string | null } = {}): { deps: WebSocketTransportDeps; store: { token: string | null } } {
const store = { token: init.token ?? null };
return {
store,
deps: {
getToken: async () => store.token,
setToken: async (t) => {
store.token = t;
},
getCachedPort: async () => null,
setCachedPort: async () => undefined,
extensionId: EXTENSION_ID,
},
};
}
function makeTransport(port: number, deps: WebSocketTransportDeps, extra: Partial<WebSocketTransportDeps> = {}): WebSocketTransport {
return new WebSocketTransport({
...deps,
...extra,
webSocketCtor: CTOR,
portRange: { start: port, end: port },
openTimeoutMs: 2000,
});
}
const maybeDescribe = VELOXD_BIN ? describe : describe.skip;
if (!VELOXD_BIN) {
console.warn('tests/live/real-veloxd.test.ts: VELOXD_BIN not set — skipping (see file header).');
}
maybeDescribe('WebSocketTransport against a real veloxd', () => {
let daemon: VeloxdInstance;
let testserver: TestServerInstance;
// The mid-test fail-open case kills `daemon` and a later test starts a replacement —
// every scratch dir that ever existed gets cleaned up here, not just the last one.
const allScratchDirs: string[] = [];
async function freshVeloxd(): Promise<VeloxdInstance> {
const d = await startVeloxd(VELOXD_BIN!);
allScratchDirs.push(d.scratch);
return d;
}
beforeAll(async () => {
daemon = await freshVeloxd();
testserver = await startTestServer();
}, 20_000);
afterAll(() => {
daemon?.kill('SIGKILL');
testserver?.proc.kill('SIGKILL');
for (const dir of allScratchDirs) rmSync(dir, { recursive: true, force: true });
});
it('session.hello without a token surfaces NotPaired / needsPairing', async () => {
const { deps } = memDeps();
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
await expect(t.connect()).rejects.toBeInstanceOf(RpcError);
expect(t.status.needsPairing).toBe(true);
t.disconnect();
});
let pairedToken: string;
it('pairs (VELOX_PAIR_AUTO=1 stands in for the human Allow click) and hellos with the issued token', async () => {
const { deps, store } = memDeps();
const t = makeTransport(daemon.wsPort, deps); // autoPair: true (default)
await t.connect();
expect(t.state).toBe('connected');
expect(t.status.daemonVersion).toBeTruthy();
expect(store.token).toBeTruthy();
pairedToken = store.token!;
t.disconnect();
});
it('the pairing token survives a reconnect: a fresh transport reuses it with no fresh pairing', async () => {
const { deps } = memDeps({ token: pairedToken });
// autoPair: false — if this succeeds at all, it can only be because the stored
// token from the previous test was accepted outright, not because this transport
// silently re-paired.
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
await t.connect();
expect(t.state).toBe('connected');
t.disconnect();
});
it('a wrong token is rejected, and repeating it rate-limits the next pairing attempt', async () => {
// Five failed session.hello attempts from this origin (ws_server.cpp records a
// rate-limiter failure on every not-paired hello, not only on a failed session.pair)
// exhausts the window; the sixth thing this origin tries — a pairing attempt — gets
// RateLimited rather than a fresh token.
for (let i = 0; i < 5; i += 1) {
const { deps } = memDeps({ token: 'not-the-real-token' });
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
await expect(t.connect()).rejects.toBeInstanceOf(RpcError);
t.disconnect();
}
const { deps } = memDeps(); // no token -> autoPair kicks in -> session.pair
const t = makeTransport(daemon.wsPort, deps);
await expect(t.connect()).rejects.toBeInstanceOf(RpcError);
expect(t.status.needsPairing).toBe(true);
expect(t.status.retryAfterSec).toBeGreaterThan(0);
t.disconnect();
});
it('download.add creates a real task the engine picks up', async () => {
const { deps } = memDeps({ token: pairedToken });
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
await t.connect();
try {
const spec: DownloadSpec = { url: `${testserver.baseUrl}/plain/file/64K`, filename: 'plain-download.bin' };
const added = await t.call('download.add', spec);
expect(added.taskId).toBeTruthy();
await waitFor(async () => {
const detail = await t.call('download.get', { taskId: added.taskId });
const state = detail.summary.state;
return state === 'complete' || state === 'downloading' || state === 'verifying';
}, 10_000, 'the task to leave the queued state');
} finally {
t.disconnect();
}
}, 15_000);
it('capture.offer end to end: the real capture path takes a monitored download, ignores its own duplicate, and fails open when the daemon dies mid-offer', async () => {
const { deps } = memDeps({ token: pairedToken });
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
await t.connect();
const rules: CaptureRules = await t.call('capture.getRules', {});
expect(rules.monitoredExtensions).toContain('zip'); // seeded default (0001_initial.sql)
const hook = new CaptureHook({
offer: (params, opts) => t.call('capture.offer', params, opts),
stash: { take: () => undefined, peek: () => undefined },
getCookies: async () => [],
getRules: () => rules,
origin: ORIGIN,
});
// A throttled URL so the task the first offer creates is still active (not yet
// complete) when the dedupe offer for the same URL follows immediately after.
const url = `${testserver.baseUrl}/throttled/file/512K`;
const details: OnHeadersReceivedDetails = {
requestId: 'live-1',
url,
method: 'GET',
type: 'other',
statusCode: 200,
tabId: 1,
responseHeaders: [{ name: 'content-disposition', value: 'attachment; filename="live-capture.zip"' }],
};
const first = await hook.handle(details);
expect(first).toEqual({ cancel: true }); // the daemon took it — Firefox never starts its own download
const list = await t.call('download.list', { filter: { query: 'live-capture' } });
expect(list.items.length).toBeGreaterThan(0);
const task = list.items[0]!;
expect(task.categoryId).toBe('programs'); // "zip" routes to the built-in Programs category
expect(task.saveDir).toContain('Downloads/Programs');
// Same URL again, task still active: the daemon's own dedupe (has_active_duplicate)
// says Ignore, so the hook proceeds instead of cancelling a second time.
const dup = await hook.handle({ ...details, requestId: 'live-2' });
expect(dup).toEqual({});
// Now kill the daemon mid-offer and prove fail-open holds against the REAL binary,
// not just FakeDaemon: the hook must still resolve to {} (Firefox downloads
// normally) well inside its own 750 ms budget.
daemon.kill('SIGKILL');
await waitFor(() => daemon.hasExited(), 5_000, 'veloxd to actually die');
const started = Date.now();
const afterDeath = await hook.handle({ ...details, requestId: 'live-3', url: `${url}?after-death=1` });
const elapsedMs = Date.now() - started;
expect(afterDeath).toEqual({}); // fail open — never {cancel: true} with a dead daemon
expect(elapsedMs).toBeLessThan(900); // budget is 750ms; the hook's own timer bounds this
t.disconnect();
}, 20_000);
it('event.task.progress reaches a subscribed client (the popup\'s own path)', async () => {
if (daemon.hasExited()) {
// The previous test kills the daemon on purpose to prove fail-open; start a fresh
// one so this test still exercises the real event path end to end.
daemon = await freshVeloxd();
}
const { deps } = memDeps();
const t = makeTransport(daemon.wsPort, deps); // fresh daemon instance -> fresh pairing
await t.connect();
try {
await t.call('session.subscribe', {
events: ['event.task.added', 'event.task.state', 'event.task.progress'],
});
const progressEvents: TaskProgressEvent[] = [];
const stateEvents: TaskStateEvent[] = [];
t.on('event.task.progress', (p) => progressEvents.push(p as TaskProgressEvent));
t.on('event.task.state', (p) => stateEvents.push(p as TaskStateEvent));
const spec: DownloadSpec = { url: `${testserver.baseUrl}/throttled/file/1M`, filename: 'progress-check.bin' };
const added = await t.call('download.add', spec);
// /throttled defaults to 1 MiB/s, so a 1 MiB file takes ~1s — long enough that at
// least one 4 Hz progress tick (event.task.progress's documented cap) lands before
// it completes, exactly the path popup/store.ts consumes in the real extension.
await waitFor(
() => progressEvents.some((e) => e.tasks.some((row) => row.taskId === added.taskId)),
8_000,
'a live event.task.progress tick for our task',
);
expect(stateEvents.some((e) => e.taskId === added.taskId)).toBe(true);
} finally {
t.disconnect();
}
}, 15_000);
it('fail-open also holds through the transport itself: a call against a dead socket rejects, never hangs past its deadline', async () => {
const { deps } = memDeps();
const t = makeTransport(daemon.wsPort, deps);
await t.connect();
t.disconnect(); // closes the socket without telling the daemon anything is wrong
await expect(t.call('capture.offer', { url: 'https://example.com/x.zip', method: 'GET', tabUrl: '' }, { timeoutMs: 200 })).rejects.toBeInstanceOf(
TransportClosedError,
);
});
});
+14
View File
@@ -0,0 +1,14 @@
# nmhost/ — velox-nmhost, the Firefox native-messaging host. Owned by lane DAEMON.
#
# Deliberately dependency-free: no veloxd_* library, no nlohmann_json, no SQLite. It is a
# byte-level pump between two framings (see src/main.cpp's own header comment) and runs
# unconfined outside Firefox's sandbox (ADR 0003) — the less it links, the less there is to
# go wrong running from wherever a snap/deb/flatpak install puts it.
add_executable(velox-nmhost src/main.cpp)
target_compile_features(velox-nmhost PRIVATE cxx_std_23)
target_compile_options(velox-nmhost PRIVATE -Wall -Wextra -Wpedantic -Werror)
if(VELOX_BUILD_TESTS AND EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/tests/CMakeLists.txt)
add_subdirectory(tests)
endif()
View File
+203
View File
@@ -0,0 +1,203 @@
// velox-nmhost — Firefox native-messaging host. A dumb pump between two framings, nothing
// else: stdin/stdout speak Firefox's own protocol (a 4-byte native-byte-order length
// prefix, then that many bytes of UTF-8 JSON); $XDG_RUNTIME_DIR/velox/velox.sock speaks
// veloxd's own NDJSON (one '\n'-terminated JSON value per line, daemon/src/rpc/ndjson.hpp).
// Reframing between the two is the entire job.
//
// Runs unconfined outside Firefox's snap sandbox with the real $HOME and
// $XDG_RUNTIME_DIR (ADR 0003 §Q2) — see packaging/nativehost/ for the manifest this is
// installed as, and which native-messaging-hosts directory actually gets read by which
// Firefox flavour.
//
// No business logic: no JSON parsing (frames are pure byte spans; only the length prefix
// and the line boundary matter here), no retry/backoff (the extension re-launches a fresh
// host on its own reconnect), no protocol version check (veloxd and the extension settle
// that between themselves once the pump hands their bytes through). Exits the moment
// either side closes.
#include <fcntl.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <cerrno>
#include <cstdint>
#include <cstdlib>
#include <cstring>
#include <poll.h>
#include <string>
namespace {
// Firefox's own cap (host -> browser) is 1 MiB; this is just a sanity backstop against a
// runaway peer so a malformed stream can't grow a buffer without bound.
constexpr std::size_t kMaxFrameBytes = 8 * 1024 * 1024;
std::string socket_path() {
const char* xdg = std::getenv("XDG_RUNTIME_DIR");
std::string base = (xdg != nullptr && xdg[0] != '\0') ? xdg
: ("/run/user/" + std::to_string(::getuid()));
if (!base.empty() && base.back() == '/') base.pop_back();
return base + "/velox/velox.sock";
}
int connect_socket(const std::string& path) {
const int fd = ::socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
if (fd < 0) return -1;
sockaddr_un addr{};
addr.sun_family = AF_UNIX;
if (path.size() + 1 > sizeof(addr.sun_path)) {
::close(fd);
return -1;
}
std::memcpy(addr.sun_path, path.c_str(), path.size());
if (::connect(fd, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) != 0) {
::close(fd);
return -1;
}
return fd;
}
// Reads everything currently available into `buf`. true on EOF, false otherwise (short
// reads / EAGAIN just return with whatever was appended).
bool read_available(int fd, std::string& buf) {
char chunk[65536];
for (;;) {
const ssize_t n = ::read(fd, chunk, sizeof(chunk));
if (n > 0) {
buf.append(chunk, static_cast<std::size_t>(n));
continue;
}
if (n == 0) return true; // EOF
if (errno == EAGAIN || errno == EWOULDBLOCK) return false;
if (errno == EINTR) continue;
return true; // treat any other error as if the peer hung up
}
}
// Writes as much of `buf` as the fd accepts right now, trimming what was sent. Returns
// false on a hard error (peer gone); EAGAIN is not an error, just "try again later".
bool flush_some(int fd, std::string& buf) {
while (!buf.empty()) {
const ssize_t n = ::write(fd, buf.data(), buf.size());
if (n > 0) {
buf.erase(0, static_cast<std::size_t>(n));
continue;
}
if (n < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) return true;
if (n < 0 && errno == EINTR) continue;
return false;
}
return true;
}
// stdin frames (4-byte length + payload) -> NDJSON lines appended to `to_socket`.
// Malformed (oversized) length is a hard stop.
bool drain_stdin_frames(std::string& in, std::string& to_socket) {
for (;;) {
if (in.size() < 4) return true;
std::uint32_t len;
std::memcpy(&len, in.data(), 4);
if (len > kMaxFrameBytes) return false;
if (in.size() < 4 + len) return true;
to_socket.append(in, 4, len);
to_socket.push_back('\n');
in.erase(0, 4 + len);
}
}
// NDJSON lines from the socket -> stdout frames (4-byte length + payload) appended to
// `to_stdout`.
bool drain_socket_lines(std::string& in, std::string& to_stdout) {
for (;;) {
const auto nl = in.find('\n');
if (nl == std::string::npos) {
if (in.size() > kMaxFrameBytes) return false;
return true;
}
std::string_view line(in.data(), nl);
if (!line.empty() && line.back() == '\r') line.remove_suffix(1);
if (line.size() > kMaxFrameBytes) return false;
const auto len = static_cast<std::uint32_t>(line.size());
to_stdout.append(reinterpret_cast<const char*>(&len), 4);
to_stdout.append(line);
in.erase(0, nl + 1);
}
}
} // namespace
int main() {
const int sock = connect_socket(socket_path());
if (sock < 0) return 1; // daemon not running / socket missing: nothing to pump
// Every fd this pumps must be non-blocking: read_available()'s own drain loop keeps
// calling read() until it actually sees EAGAIN, and a blocking fd never returns that —
// it just blocks inside the "drain what's available" loop instead of going back to
// poll(), which stalls the whole pump the moment one side has more to send than fits
// in a single read().
::fcntl(0, F_SETFL, ::fcntl(0, F_GETFL) | O_NONBLOCK);
::fcntl(1, F_SETFL, ::fcntl(1, F_GETFL) | O_NONBLOCK);
::fcntl(sock, F_SETFL, ::fcntl(sock, F_GETFL) | O_NONBLOCK);
std::string stdin_buf, stdout_buf, socket_in_buf, socket_out_buf;
bool stdin_eof = false;
// Three distinct descriptors, not two: stdin (0) and stdout (1) are separate pipes
// (read-only and write-only respectively — never the same fd, even though they sit
// next to each other in a shell's mental model of "the process's stdio"), plus the
// bidirectional socket.
enum { kStdin, kStdout, kSock };
for (;;) {
pollfd fds[3] = {
{0, 0, 0},
{1, 0, 0},
{sock, 0, 0},
};
if (!stdin_eof) fds[kStdin].events |= POLLIN;
if (!stdout_buf.empty()) fds[kStdout].events |= POLLOUT;
fds[kSock].events |= POLLIN;
if (!socket_out_buf.empty()) fds[kSock].events |= POLLOUT;
// Nothing left to wait for: both directions exhausted.
if (fds[kStdin].events == 0 && fds[kStdout].events == 0 && fds[kSock].events == 0) break;
const int n = ::poll(fds, 3, -1);
if (n < 0) {
if (errno == EINTR) continue;
break;
}
if (fds[kStdout].revents & POLLOUT) {
if (!flush_some(1, stdout_buf)) break; // Firefox closed our stdout
}
if (fds[kSock].revents & POLLOUT) {
if (!flush_some(sock, socket_out_buf)) break;
}
if (fds[kStdin].revents & (POLLIN | POLLHUP)) {
if (read_available(0, stdin_buf)) stdin_eof = true;
if (!drain_stdin_frames(stdin_buf, socket_out_buf)) break;
}
if (fds[kSock].revents & (POLLIN | POLLHUP)) {
const bool socket_eof = read_available(sock, socket_in_buf);
if (!drain_socket_lines(socket_in_buf, stdout_buf)) break;
if (socket_eof) {
// The daemon is gone. Flush whatever we already turned into stdout
// frames, then stop — there is nothing left to relay either direction.
(void)flush_some(1, stdout_buf);
break;
}
}
if ((fds[kStdin].revents | fds[kStdout].revents | fds[kSock].revents) &
(POLLERR | POLLNVAL))
break;
// Firefox closed the pipe: nothing more will ever arrive on stdin, and once our
// own outbound backlog drains there is nothing left to send it either. Stop
// rather than idle forever relaying replies nobody reads.
if (stdin_eof && socket_out_buf.empty()) break;
}
::close(sock);
return 0;
}
+13
View File
@@ -0,0 +1,13 @@
# Integration test only — velox-nmhost has no internal functions worth unit-testing in
# isolation (it's ~15 lines of byte-shuffling helpers around one poll() loop); what matters
# is the real binary's observable behaviour over real pipes and a real socket.
add_executable(velox_nmhost_pump_test pump_test.cpp)
target_compile_features(velox_nmhost_pump_test PRIVATE cxx_std_23)
target_compile_options(velox_nmhost_pump_test PRIVATE -Wall -Wextra -Wpedantic -Werror)
target_compile_definitions(velox_nmhost_pump_test PRIVATE
VELOX_NMHOST_BIN="$<TARGET_FILE:velox-nmhost>")
add_dependencies(velox_nmhost_pump_test velox-nmhost)
add_test(NAME nmhost.pump COMMAND velox_nmhost_pump_test)
set_tests_properties(nmhost.pump PROPERTIES TIMEOUT 30)
+54
View File
@@ -0,0 +1,54 @@
#pragma once
// Minimal test harness: CHECK accumulates failures, TEST_MAIN reports and sets the exit
// code. Copied from daemon/tests/check.hpp rather than shared across a build-dependency —
// nmhost is deliberately dependency-free, tests included.
#include <cstdio>
#include <string>
#include <vector>
namespace veloxd_test {
inline std::vector<std::string>& failures() {
static std::vector<std::string> f;
return f;
}
inline int& checks() {
static int n = 0;
return n;
}
} // namespace veloxd_test
#define CHECK(cond) \
do { \
++::veloxd_test::checks(); \
if (!(cond)) { \
::veloxd_test::failures().push_back(std::string(__FILE__) + ":" + \
std::to_string(__LINE__) + ": " + #cond); \
} \
} while (0)
#define CHECK_EQ(a, b) \
do { \
++::veloxd_test::checks(); \
auto _va = (a); \
auto _vb = (b); \
if (!(_va == _vb)) { \
::veloxd_test::failures().push_back(std::string(__FILE__) + ":" + \
std::to_string(__LINE__) + ": " + #a + \
" == " + #b); \
} \
} while (0)
#define TEST_MAIN() \
int main() { \
run(); \
for (const auto& f : ::veloxd_test::failures()) std::printf("FAIL %s\n", f.c_str()); \
std::printf("%d/%d checks passed\n", \
::veloxd_test::checks() - \
static_cast<int>(::veloxd_test::failures().size()), \
::veloxd_test::checks()); \
return ::veloxd_test::failures().empty() ? 0 : 1; \
}
+148
View File
@@ -0,0 +1,148 @@
// Integration test for velox-nmhost: spawns the real binary, feeds it a framed stdin
// message, answers over a fake Unix socket standing in for veloxd, and checks what comes
// back out on stdout — plus that it exits promptly once stdin closes.
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/un.h>
#include <sys/wait.h>
#include <unistd.h>
#include <cstdint>
#include <cstdlib>
#include <cstring>
#include <ctime>
#include <string>
#include "check.hpp"
namespace {
std::string make_temp_dir() {
char tmpl[] = "/tmp/velox-nmhost-test-XXXXXX";
const char* dir = ::mkdtemp(tmpl);
return dir ? dir : "/tmp";
}
// Firefox's own framing: 4-byte native-byte-order length, then that many bytes.
std::string frame(const std::string& payload) {
std::uint32_t len = static_cast<std::uint32_t>(payload.size());
std::string out(reinterpret_cast<char*>(&len), 4);
out += payload;
return out;
}
// Reads exactly one framed message off `fd`, blocking. Empty on EOF/short read.
std::string read_frame(int fd) {
std::uint32_t len = 0;
std::size_t got = 0;
while (got < 4) {
const ssize_t n = ::read(fd, reinterpret_cast<char*>(&len) + got, 4 - got);
if (n <= 0) return {};
got += static_cast<std::size_t>(n);
}
std::string payload(len, '\0');
got = 0;
while (got < len) {
const ssize_t n = ::read(fd, payload.data() + got, len - got);
if (n <= 0) return {};
got += static_cast<std::size_t>(n);
}
return payload;
}
bool write_all(int fd, const std::string& s) {
std::size_t off = 0;
while (off < s.size()) {
const ssize_t n = ::write(fd, s.data() + off, s.size() - off);
if (n <= 0) return false;
off += static_cast<std::size_t>(n);
}
return true;
}
} // namespace
void run() {
const std::string dir = make_temp_dir();
const std::string velox_dir = dir + "/velox";
CHECK(::mkdir(velox_dir.c_str(), 0700) == 0);
const std::string sock_path = velox_dir + "/velox.sock";
// A bare listening socket standing in for veloxd.
const int listen_fd = ::socket(AF_UNIX, SOCK_STREAM, 0);
CHECK(listen_fd >= 0);
sockaddr_un addr{};
addr.sun_family = AF_UNIX;
std::memcpy(addr.sun_path, sock_path.c_str(), sock_path.size());
CHECK(::bind(listen_fd, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) == 0);
CHECK(::listen(listen_fd, 1) == 0);
int child_stdin[2]; // [0] read (child), [1] write (parent)
int child_stdout[2]; // [0] read (parent), [1] write (child)
CHECK(::pipe(child_stdin) == 0);
CHECK(::pipe(child_stdout) == 0);
::setenv("XDG_RUNTIME_DIR", dir.c_str(), 1);
const pid_t pid = ::fork();
CHECK(pid >= 0);
if (pid == 0) {
::dup2(child_stdin[0], 0);
::dup2(child_stdout[1], 1);
::close(child_stdin[0]);
::close(child_stdin[1]);
::close(child_stdout[0]);
::close(child_stdout[1]);
::close(listen_fd);
::execl(VELOX_NMHOST_BIN, "velox-nmhost", nullptr);
::_exit(127);
}
::close(child_stdin[0]);
::close(child_stdout[1]);
// Accept nmhost's connection.
const int conn = ::accept(listen_fd, nullptr, nullptr);
CHECK(conn >= 0);
// stdin (framed) -> socket (NDJSON line).
CHECK(write_all(child_stdin[1], frame(R"({"hello":1})")));
char line[256] = {};
ssize_t n = ::read(conn, line, sizeof(line) - 1);
CHECK(n > 0);
CHECK_EQ(std::string(line, static_cast<std::size_t>(n)), std::string("{\"hello\":1}\n"));
// socket (NDJSON line) -> stdout (framed).
CHECK(write_all(conn, "{\"world\":2}\n"));
const std::string got = read_frame(child_stdout[0]);
CHECK_EQ(got, std::string(R"({"world":2})"));
// A second round trip on the same connection, to prove buffering across calls works
// (not just "the first message happens to line up with one read()").
CHECK(write_all(child_stdin[1], frame(R"({"again":3})")));
n = ::read(conn, line, sizeof(line) - 1);
CHECK(n > 0);
CHECK_EQ(std::string(line, static_cast<std::size_t>(n)), std::string("{\"again\":3}\n"));
// Firefox closes the pipe: nmhost must exit promptly rather than hang.
::close(child_stdin[1]);
int status = 0;
pid_t waited = -1;
for (int i = 0; i < 50 && waited != pid; ++i) {
waited = ::waitpid(pid, &status, WNOHANG);
if (waited == pid) break;
struct timespec ts{0, 20'000'000}; // 20ms
::nanosleep(&ts, nullptr);
}
CHECK_EQ(waited, pid);
if (waited == pid) CHECK(WIFEXITED(status));
::close(conn);
::close(child_stdout[0]);
::close(listen_fd);
::unlink(sock_path.c_str());
::rmdir(velox_dir.c_str());
::rmdir(dir.c_str());
}
TEST_MAIN()
View File
+97
View File
@@ -0,0 +1,97 @@
# packaging/nativehost — the Firefox native-messaging manifest
Owner: lane DAEMON (`packaging/nativehost/**` is explicitly DAEMON's per `CLAUDE.md`'s
lane table, unlike the rest of `packaging/`, which is PKG/QA's). This directory holds the
manifest `velox-nmhost` (built from `nmhost/`) is registered under, and this file is the
one place that says exactly which on-disk locations need it and why — the
"install manifests to all four locations" line in `docs/agents/AGENT-DAEMON.md`'s build
order predates `docs/adr/0003-native-messaging-under-snap.md`, which cut that down to
three *real* ones. Read the ADR before touching this list; it is the empirical spike, not
a guess.
## The manifest
`com.velox.host.json` in this directory is the template:
```json
{
"name": "com.velox.host",
"description": "Velox download manager — native messaging bridge to veloxd",
"path": "/usr/libexec/velox/velox-nmhost",
"type": "stdio",
"allowed_extensions": ["velox@velox.download"]
}
```
- `name` is what the extension calls `browser.runtime.connectNative("com.velox.host")`
with (`extension/src/background/transport/native.ts`'s `HOST_NAME`) — do not rename one
without the other.
- `allowed_extensions` must match `extension/manifest.json`'s
`browser_specific_settings.gecko.id` exactly (`[email protected]`). A mismatch here is
a silent failure: Firefox reports "no such native application" as if the manifest didn't
exist at all, which reads exactly like a missing-file bug and is easy to chase in the
wrong place.
- `path` is `/usr/libexec/velox/velox-nmhost` — the `.deb`'s install layout
(`docs/07-packaging.md`). **Every other packaging format needs a different `path`** —
the binary doesn't live at that absolute path inside a Flatpak sandbox or an AppImage
mount, so whatever installs the manifest for those formats must rewrite this field to
wherever it actually put the binary, not ship this file verbatim. That substitution is
the packaging step's job, not this template's.
## Where it has to be written, and why
Per ADR 0003 (spike S1, run against the real snap Firefox on the target machine — not a
guess about how confinement "should" work):
| Location | Covers | Why |
|---|---|---|
| `~/.mozilla/native-messaging-hosts/com.velox.host.json` | **deb/tarball Firefox, and snap Firefox** | The one location snap Firefox 154 actually reads. Firefox's snap launches native-messaging hosts *outside* the sandbox with the user's real `$HOME` — so this is not "the deb location that happens to also work"; it is *the* snap location, full stop. `~/snap/firefox/common/.mozilla/native-messaging-hosts/` (the intuitive "inside the snap" path) is **not** read by Firefox 154 snap — confirmed empirically, not inferred. |
| `/usr/lib/mozilla/native-messaging-hosts/com.velox.host.json` | **deb/tarball Firefox only** | System-wide, so it covers every user on the box for a real (non-snap) Firefox install — but confirmed **not** read by snap Firefox. Do not treat this path as "covers snap too"; that was the wrong assumption `docs/05` §4 corrected in the same commit as the ADR. |
| `~/.var/app/org.mozilla.firefox/.mozilla/native-messaging-hosts/com.velox.host.json` | **Flatpak Firefox** | Flatpak's own sandboxed home. Untested on this machine (Firefox here is the snap, not flatpak) — carried over from `docs/05` §4's original four-location list, which this table otherwise supersedes. Verify before relying on it in a release checklist. |
That is three real locations, not four — the fourth
(`~/snap/firefox/common/.mozilla/native-messaging-hosts/`) was the pre-ADR guess the spike
disproved. If `docs/agents/AGENT-DAEMON.md`'s build order still says "four locations" when
you read this, it is stale; this table is the current source of truth alongside the ADR
itself.
## What this means for `postinst` (PKG/QA's file, not this one)
This directory ships the manifest template and documents the target paths; it does not
install anything itself (`packaging/**` outside this one directory is PKG/QA's — see
`CLAUDE.md`'s lane table — and `postinst` specifically is a `.deb`-packaging concern this
lane doesn't own the file for). For whoever writes it:
- **The two `~/`-relative locations are per-user.** `postinst` runs as root, once, at
install time — it does not run once per user session. It needs either a real-user
enumeration at install time (every UID with a home directory and no login shell of
`/usr/sbin/nologin`-style exclusions, roughly what `deluser --remove-home` scripts already
have to reason about) or a first-run hook that runs as the logged-in user (a systemd user
unit's `ExecStartPre`, or the GUI's own first-run wizard) and writes its own manifest the
first time it starts. `docs/adr/0003`'s own follow-up section flagged this as open; it
still is.
- **`/usr/lib/mozilla/native-messaging-hosts/` is the only one of the three that's a plain
root-owned, install-time write** — no per-user enumeration needed for that one.
- **Detect snap Firefox and say so.** `docs/07-packaging.md` already commits to this
("detects whether Firefox is a snap... prints... a one-line note that the extension will
pair over loopback") — the detection matters here specifically because if Firefox turns
out to be neither deb/tarball nor a snap this spike covers (a genuinely unknown or future
packaging of Firefox), silently trusting `NativeTransport` to work is exactly the failure
mode ADR 0003 exists to prevent. `WebSocketTransport` is the guaranteed fallback either
way (ADR 0003's own decision) — nothing breaks if the manifest doesn't land correctly, it
just means the extension pairs over loopback instead of the (opportunistic, not
required) native path.
- **`docs/07-packaging.md`'s own install layout line currently lists only the
system-wide `/usr/lib/mozilla/...` path.** That line is correct as far as it goes (it *is*
one of the three locations, and the only pure root-owned one) but reads as if it were the
whole story for native messaging; it predates this file and the ADR. Worth a line
pointing here so the two documents don't quietly disagree — PKG/QA's call, not edited
here since `docs/07` is PKG/QA's own file.
## The binary
`nmhost/` builds `velox-nmhost` — see that directory's own `src/main.cpp` for what it does
(a byte-level pump, no protocol logic) and `docs/05-extension-spec.md` §4 for the two
transports it sits behind. It is deliberately dependency-free (not linked against any
`veloxd_*` library) so wherever a packaging format's sandbox puts it, it has nothing else
to go looking for at runtime.
+7
View File
@@ -0,0 +1,7 @@
{
"name": "com.velox.host",
"description": "Velox download manager — native messaging bridge to veloxd",
"path": "/usr/libexec/velox/velox-nmhost",
"type": "stdio",
"allowed_extensions": ["[email protected]"]
}
+65
View File
@@ -0,0 +1,65 @@
# packaging/systemd — the user unit pair for veloxd
Provided by lane DAEMON (`daemon/docs/AGENT-DAEMON.md` build step 7: "systemd user units
— `velox.service` + `velox.socket` for socket activation"), for PKG/QA to install per
`docs/07-packaging.md`'s layout:
```
/usr/lib/systemd/user/velox.service
/usr/lib/systemd/user/velox.socket
```
`packaging/` outside `nativehost/` is PKG/QA's per `CLAUDE.md`'s lane table; these two
files are here because they are inputs to that packaging step, not a claim on the rest of
the directory — same relationship `packaging/nativehost/` already has.
## Why both files, and what `RuntimeDirectory=` is doing
`velox.socket` binds `$XDG_RUNTIME_DIR/velox/velox.sock` **before `veloxd` ever runs** and
hands the daemon the already-listening fd at startup (`daemon/src/rpc/systemd_activation.cpp`
implements the receiving half — `LISTEN_PID`/`LISTEN_FDS`, fd 3 — without a `libsystemd`
link). Two things this buys over the daemon binding its own socket on every start:
- **No window where a client gets `ECONNREFUSED`.** The socket exists and queues
connections from the moment `velox.socket` is active, not from whenever `veloxd`
finishes starting up — this is the actual point of socket activation, not just "start
on demand."
- **Cold-boot ordering is free.** Nothing has to wait for `veloxd` to be ready before the
GUI, the CLI, or a native-messaging host can attempt a connection; the kernel queues it.
`RuntimeDirectory=velox` on the socket unit creates `%t/velox` (mode 0700) before the
`ListenStream=` bind — without it, binding fails outright the first time (nothing has
created the parent directory yet). `veloxd` itself creates that same directory
(`ensure_private_dir` in `runtime_dir.cpp`) for the case where it's started directly,
outside systemd (`./veloxd` in a terminal, still supported and how most of this daemon's
own testing runs) — the two paths converge on the same directory with the same mode
either way.
`UdsServer::start()` (`daemon/src/rpc/uds_server.cpp`) checks for the activated fd first
and, if present, skips create/bind/chmod/listen entirely — the socket file's lifecycle then
belongs to the unit (including `RemoveOnStop=yes` on stop), not to the daemon. Falls back
to binding its own socket exactly as before when not socket-activated (a manual run, or a
distro that ships the daemon without the unit files).
## What was deliberately left out
`velox.service` does **not** set `ProtectSystem=`, `ProtectHome=`, or `ReadWritePaths=`.
`saveTo.allowedRoots` is user-configurable to anywhere on the filesystem — an external
drive, a second mount, anywhere `fs/safepath.hpp`'s own canonicalize-and-check accepts —
not a fixed set of directories a unit file could enumerate ahead of time. A filesystem-level
sandbox here would turn a legitimately-configured save location into an opaque
`EROFS`/`EACCES` the daemon can't explain, in place of its own clear `-32011` — worse than
no sandbox, specifically for a download manager. `NoNewPrivileges=yes` is kept: it has no
such trade-off.
## Verifying socket activation without a real install
`systemd-analyze verify --user velox.service velox.socket` checks unit-file syntax (it
will complain that `/usr/bin/veloxd` and the `velox(1)` man page don't exist on a dev
box that hasn't installed the package — expected, not a unit bug). To exercise the actual
activation handshake without installing anything: bind a Unix socket, `dup2` it onto fd 3,
fork, set `LISTEN_PID=<child pid>` and `LISTEN_FDS=1` in the child's environment, clear
`FD_CLOEXEC` on fd 3, and `execve` `veloxd` — a real `session.hello` round-trips over that
fd with no `bind()`/`listen()` call ever happening inside the daemon for that run. This is
exactly what `velox.socket`'s `Requires=`/`ExecStart` sequence does in production; systemd
supplies the fd, `veloxd` doesn't know the difference.
+38
View File
@@ -0,0 +1,38 @@
[Unit]
Description=Velox download manager daemon
Documentation=man:velox(1)
# Socket activation (velox.socket) means this unit does not need to be enabled or started
# directly for the RPC transport to come up on demand — the first connection attempt after
# boot starts veloxd with the listening socket already bound (see velox.socket's own
# comment). Requires=/After= still matter for a manual `systemctl --user start velox`.
Requires=velox.socket
After=velox.socket
# Never more than one real instance for this user regardless of how it was started — the
# abstract-socket single-instance lock (main.cpp, keyed off the resolved runtime dir) is
# the actual enforcement; this just keeps systemd itself from racing two starts.
StartLimitIntervalSec=60
StartLimitBurst=5
[Service]
Type=simple
ExecStart=/usr/bin/veloxd
# main.cpp's SIGTERM handler stops the event loop and falls through to a clean shutdown
# (flushes buffers, closes the store, releases the single-instance lock) — the default
# KillSignal=SIGTERM and TimeoutStopSec are already the right shape for that; no
# ExecStop/KillMode override needed.
Restart=on-failure
RestartSec=2
# Hardening deliberately stops here, not at ProtectSystem=/ProtectHome=/ReadWritePaths=:
# saveTo.allowedRoots is user-configurable to anywhere (an external drive, a second
# mount — fs/safepath.hpp is the daemon's own validation boundary, not a fixed set of
# directories a unit file could enumerate up front). A filesystem-level sandbox here would
# silently turn a legitimately-configured save location into an opaque EROFS/EACCES the
# daemon can't explain, instead of its own clear -32011 — worse than no sandbox, for a
# download manager specifically. NoNewPrivileges is free of that trade-off.
NoNewPrivileges=yes
[Install]
WantedBy=default.target
Also=velox.socket
+28
View File
@@ -0,0 +1,28 @@
[Unit]
Description=Velox download manager — RPC socket
[Socket]
# %t is $XDG_RUNTIME_DIR for a user unit — the exact path veloxd itself resolves
# (daemon/src/rpc/runtime_dir.cpp's resolve_runtime_dir), and the exact path velox(1)
# resolves too (default_socket_path() in cli/src/client.cpp). All three must agree; this
# is the one line that has to.
ListenStream=%t/velox/velox.sock
# RuntimeDirectory creates %t/velox (mode 0700, this user's own) before binding, so the
# ListenStream= path above always has somewhere to land — veloxd itself does the same
# thing (ensure_private_dir) when it creates the directory unassisted (the non-activated
# path, e.g. a manual `veloxd` run outside systemd).
RuntimeDirectory=velox
RuntimeDirectoryMode=0700
# Same-UID-only, matching the socket's own authorization once a connection is accepted
# (SO_PEERCRED, checked again in uds_server.cpp regardless of this mode bit — belt and
# braces, not a substitute for it).
SocketMode=0600
# The socket file belongs to systemd's socket-activation state, not to whatever's on disk
# from a previous boot; remove it on stop so a stale entry never shadows the next start.
RemoveOnStop=yes
[Install]
WantedBy=sockets.target
+17 -5
View File
@@ -160,11 +160,18 @@ if [ -z "$EXTERNAL_UDS" ] && [ -z "$EXTERNAL_WS" ]; then
for _ in $(seq 1 50); do [ -S "$VUDS" ] && break; sleep 0.2; done
[ -S "$VUDS" ] || { echo "veloxd did not start:"; cat "$WORK/veloxd.log"; exit 1; }
# saveTo.allowedRoots defaults to ["~/Downloads"]; download.add.json (fixture) asks
# for a saveDir under $HOME/Downloads, so both that and download.add's own isolated
# downloads dir need to be allowed roots, or every download.add fixture fails -32011
# before the point of this runner is even reached. settings.set is itself a D3 stub,
# so this is written straight into the isolated velox.db rather than over the wire.
# saveTo.allowedRoots defaults to ["~/Downloads"]; download.add's own isolated
# downloads dir needs to be an allowed root too, or every download.add fixture fails
# -32011 before the point of this runner is even reached. $HOME/Downloads stays in
# the list alongside it: a few fixtures still set an explicit saveDir there
# (category.upsert.json, download.update.json) rather than take the default.
# capture.minSizeBytes defaults to 0 (nothing is ever "too small"), which makes
# errors/capture.offer.ignore.json's below-minimum-size case impossible to reach
# against a fresh daemon; raised here so that fixture's scenario is actually
# reachable. Written straight into the isolated velox.db, before veloxd has any RPC
# session to write it through: settings.set is real now (D9), but this has to be in
# place before the very first fixture runs, and setup happens before any connection
# exists.
python3 - "$VXDG/data/velox/velox.db" "$VXDG/downloads" "$HOME/Downloads" <<'PY'
import json, sqlite3, sys
db_path, isolated_downloads, home_downloads = sys.argv[1:4]
@@ -174,6 +181,11 @@ db.execute(
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
("saveTo.allowedRoots", json.dumps([isolated_downloads, home_downloads])),
)
db.execute(
"INSERT INTO settings(key, value) VALUES(?, ?) "
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
("capture.minSizeBytes", json.dumps(1000000)),
)
db.execute(
"INSERT INTO settings(key, value) VALUES(?, ?) "
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
+13 -27
View File
@@ -1,40 +1,26 @@
[
{ "fixture": "contracts/fixtures/download.refreshUrl.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/download.update.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/grabber.harvest.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
{ "fixture": "contracts/fixtures/grabber.start.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
{ "fixture": "contracts/fixtures/grabber.status.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
{ "fixture": "contracts/fixtures/rules.list.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/rules.upsert.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/settings.get.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/settings.set.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/limiter.get.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/limiter.set.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/schedule.get.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/schedule.set.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/queue.reorder.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/grabber.harvest.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/grabber.start.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/grabber.status.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/media.addVariant.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/media.listVariants.json", "reason": "D3: stub handler, -32603" },
{ "fixture": "contracts/fixtures/capture.getRules.json", "reason": "D3: stub handler, -32603 -- DAEMON is filing capture.offer next" },
{ "fixture": "contracts/fixtures/capture.offer.take.json", "reason": "D3: stub handler, -32603 -- DAEMON is filing capture.offer next" },
{ "fixture": "contracts/fixtures/errors/capture.offer.ignore.json", "reason": "D3: stub handler, -32603 -- DAEMON is filing capture.offer next" },
{ "fixture": "contracts/fixtures/media.addVariant.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
{ "fixture": "contracts/fixtures/media.listVariants.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
{ "fixture": "contracts/fixtures/errors/download.provideAuth.not-found.json", "reason": "real bug: on_download_provideAuth (dispatcher.cpp) never checks the task exists -- TaskActionPort::provide_auth returns false for an unknown id, which the handler folds into a normal {ok:false} result instead of -32010" },
{ "fixture": "contracts/fixtures/category.list.json", "reason": "documented gap (deferrals.md D3a note): categories table has no mimeTypes/sortOrder columns, so category.upsert accepts them but category.list never echoes mimeTypes back" },
{ "fixture": "contracts/fixtures/capture.getRules.json", "reason": "not a bug: capture.monitoredMimeTypes defaults to [] (store/settings.cpp's kDefaults) on a fresh daemon; the golden's non-empty example illustrates a configured one" },
{ "fixture": "contracts/fixtures/download.probe.json", "reason": "not a bug: requiresAuth is optional-and-omitted-when-false (schema doesn't require it); the golden shows it because that fixture's probe hit a 401, this run's doesn't" },
{ "fixture": "contracts/fixtures/download.get.json", "reason": "not a bug: effectiveUrl is 'null until the first probe succeeds' (schema) and omitted rather than sent as null; our bound $taskId is a fresh, never-started task, so it's never been probed -- the golden depicts an in-progress download instead" },
{ "fixture": "contracts/fixtures/download.list.json", "reason": "same as download.get.json: effectiveUrl omitted for our never-started bound tasks, golden depicts an in-progress download" },
{ "fixture": "contracts/fixtures/session.hello.json", "reason": "not a bug: capabilities is genuinely empty because media/grabber/Secret Service aren't implemented yet; the golden's ['media','grabber','secretservice'] illustrates a future daemon, not this one" },
{ "fixture": "contracts/fixtures/download.update.json", "reason": "not a bug: etaSeconds is only known for a task the engine has probed/is running; our bound $taskId is a fresh, never-started task, so it's absent -- same class as download.get.json's effectiveUrl" },
{ "fixture": "contracts/fixtures/session.hello.json", "reason": "not a bug: capabilities is genuinely empty because media/grabber aren't implemented yet (capture/Secret Service's parts of it now are); the golden's example list illustrates a future daemon, not this one" },
{ "fixture": "contracts/fixtures/queue.start.json", "reason": "not a bug: startedTaskIds is empty because nothing is a member of queue 'main' in this isolated run; the golden depicts a queue with real membership" },
{ "fixture": "contracts/fixtures/queue.reorder.json", "reason": "not a bug: the fixture's taskIds are two literal ids that only ever existed in a seeded mock; queue 'main' has no members at all in this isolated run, so any non-empty list is correctly rejected as not a permutation of (empty) membership" },
{ "fixture": "contracts/fixtures/rules.list.json", "reason": "not a bug: no rule is ever seeded in a fresh daemon; the golden depicts a configured rule set" },
{ "fixture": "contracts/fixtures/schedule.set.json", "reason": "documented gap (deferrals.md D3f note): nextRunAt is deliberately left unset -- computing it needs DST-aware next-transition logic sched/schedule_window.hpp doesn't have yet" },
{ "fixture": "contracts/fixtures/category.remove.json", "reason": "not a bug: reassignedTaskIds is empty because nothing was ever filed under the 'firmware' category this run creates; the golden depicts a category with real membership" }
]
+46 -72
View File
@@ -88,88 +88,62 @@ manual testing.
nowhere to run. GUI owns the harness; PKG/QA owns the CI job. This is the wiring contract
so the two halves meet without another round trip.
### What PKG/QA needs from GUI
### What GUI built, and the invocation contract
A driver invoked as `gui/tests/dod/run.sh <gate> [--json <path>]` (exact path TBD by GUI),
headless-capable (Xvfb or offscreen `QT_QPA_PLATFORM`), with:
`gui/tests/dod/run.sh <gate> [--json <path>]` (`gate` one of `scroll-60fps` / `rss-flat`
/ `unhappy-path`), backed by `gui/tests/dod/dod_harness.cpp` (target `gui-dod-harness`).
Headless by default — `run.sh` sets `QT_QPA_PLATFORM=offscreen` itself, so the CI jobs
below need no Xvfb.
| `<gate>` | Pass / fail condition | Budget |
|---|---|---|
| `scroll-60fps` | `mockd --tasks 10000`, scripted fling scroll; **fail** if p99 frame > 16.6 ms | per-PR |
| `rss-flat` | `mockd --tasks 10000` + progress events, 10 min; **fail** if RSS growth > a fixed slack (GUI picks the number, states it) | nightly |
| `unhappy-path` | `mockd --slow` / `--flaky <f>` / `--drop-connection <s>`; **fail** on crash, on watchdog-detected hang, or if connection state never returns to `Connected` | per-PR |
| `scroll-60fps` | `mockd --tasks 10000`, scripted fling scroll; **fail** if p99 frame > 16.6 ms (×4 under an ASan/UBSan build — `VELOX_DOD_FRAME_BUDGET_MS` overrides outright) | per-PR |
| `rss-flat` | `mockd --tasks 10000` + progress events, 10 min; **fail** if RSS growth past warm-up exceeds a 20 MiB slack (`VELOX_DOD_RSS_SLACK_KIB` overrides) | nightly |
| `unhappy-path` | Three phases (`--slow` / `--flaky <f>` / `--drop-connection <s>`), one `mockd` restart each; **fail** on crash, on the harness's own watchdog firing (75 s), or if connection state never (re)reaches `Connected` | per-PR |
Contract:
Contract, as built: exit `0` pass, non-zero fail; a hang is the harness's own watchdog
converting itself into a non-zero exit (`3`), never something CI needs `timeout(1)`
around; `--json` writes one result object per gate (`unhappy-path` merges its three
phases into one file); no network, no writes outside a tempdir, no leaked child process
on any exit path (`run.sh`'s `trap cleanup EXIT INT TERM`).
* exit `0` pass, non-zero fail; a hang is the harness's own watchdog to catch and turn
into a non-zero exit, not something CI should have to `timeout(1)` around.
* `--json` writes one machine-readable result file (measured p99, RSS series, recovery
time) so the job can upload it as an artifact and a regression is a diff, not a re-run.
* no network, no writes outside a tempdir, no leaked child processes on failure.
### CI jobs — wired
### CI job — pre-drafted, add once the harness path is fixed
`gui-dod` (per-PR: `scroll-60fps` + `unhappy-path`) and `gui-dod-nightly` (`rss-flat`,
`schedule`/`workflow_dispatch` only) are live in `ci.yml`. Both bootstrap, build
`gui-dod-harness`, `npm ci` in `tools/mockd`, run `gui/tests/dod/run.sh`, and upload the
`--json` output as an artifact — no Xvfb step, since `run.sh` already runs offscreen.
```yaml
gui-dod:
# Per-PR GUI gates. The 10-minute rss-flat gate is in gui-dod-nightly, not here.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Bootstrap toolchain
run: sudo ./tools/bootstrap.sh
- uses: actions/setup-node@v4
with:
node-version: '22' # tools/mockd
- name: Configure + build
run: |
cmake --preset dev
cmake --build --preset dev --target velox-gui
- name: Install mockd
run: cd tools/mockd && npm ci
- name: Xvfb + gates
run: |
sudo apt-get install -y --no-install-recommends xvfb
xvfb-run -a gui/tests/dod/run.sh scroll-60fps --json scroll.json # TODO(GUI): path
xvfb-run -a gui/tests/dod/run.sh unhappy-path --json unhappy.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: gui-dod-${{ github.run_id }}
path: "*.json"
### Forced red, once per gate, before wiring it required
gui-dod-nightly:
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Bootstrap toolchain
run: sudo ./tools/bootstrap.sh
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Configure + build
run: |
cmake --preset dev
cmake --build --preset dev --target velox-gui
- run: cd tools/mockd && npm ci
- name: RSS soak (10 min)
run: |
sudo apt-get install -y --no-install-recommends xvfb
xvfb-run -a gui/tests/dod/run.sh rss-flat --json rss.json
- uses: actions/upload-artifact@v4
if: always()
with:
name: gui-dod-rss-${{ github.run_id }}
path: rss.json
```
| Gate | Forced via | Observed |
|---|---|---|
| `scroll-60fps` | `VELOX_DOD_FRAME_BUDGET_MS=0.01` | `FAIL: p99=50.73 ms mean=31.34 ms max=52.88 ms budget=0.01 ms over 240 steps, 10000 rows`, exit 1 |
| `rss-flat` | `VELOX_DOD_RSS_SLACK_KIB=-999999999` (guarantees `growthKiB > slack` regardless of actual RSS behavior that run) | `FAIL: growth=13644 KiB slack=-999999999 KiB over 15s (warmup 1s), 10000 rows`, exit 1 |
| `unhappy-path` | ran `gui-dod-harness` directly (bypassing `run.sh`, which always starts a working `mockd`) against a `--sock` path with nothing listening | 75 s of `Connecting`/`Reconnecting`, then `FAIL: dod_harness watchdog fired — hung past 75s`, exit 3 |
`gui-dod-nightly` needs `if: github.event_name == 'schedule'` (the `nightly-integration`
job below already added that trigger to `ci.yml` — `cron: '17 3 * * *'` — so this no
longer needs its own).
The real (non-forced) runs all pass live: `scroll-60fps` p99 51.28 ms against a 66.4 ms
budget (ASan/UBSan build), `rss-flat` growth 13504 KiB against the 20480 KiB slack over a
15 s smoke duration, `unhappy-path` all three phases `PASS` with `finalConnected=true`.
### Known gap: `unhappy-path`'s drop-connection phase doesn't drop anything
Filed by GUI in `gui/docs/proto-requests-m1.md`: `mockd --drop-connection` only works
over WebSocket — `startUds()` never wires the periodic-drop timer `startWs()` has, so the
UDS transport (GUI/CLI/nmhost's only one) never sees a connection actually die. GUI's own
live verification: 45 s observing a `--drop-connection 5` mockd over UDS, `stateChanged`
never fires. The phase still runs and its JSON honestly records `sawDisruption: false`
rather than silently passing as if it proved something — that's what the real run above
shows, `pass: true` alongside `sawDisruption: false`, so a reviewer reading the JSON sees
exactly how much this phase currently covers.
This is PROTO's fix, not PKG/QA's or GUI's to route around locally — coordinating on it
rather than patching mockd from this lane. Once PROTO lands `dropEverySec` on `startUds`,
`unhappy-path`'s drop-connection phase starts exercising a real drop and this note comes
out; until then `gui-dod` stays required as specified (crash/hang/never-reconnects still
catch real regressions), just not yet catching a swallowed real disconnect.
### Status
Blocked on GUI's harness. Not urgent (GUI M1 DoD, not M0). When GUI files the follow-up
with the real `run.sh` path and the `rss-flat` slack number, PKG/QA drops the `TODO(GUI)`
markers and marks `gui-dod` required. The `schedule:` trigger `gui-dod-nightly` needs is
already in `ci.yml`.
Wired and required: `gui-dod` runs per-PR, `gui-dod-nightly` on schedule. Revisit once
PROTO's UDS `--drop-connection` fix lands (see above).
+4 -1
View File
@@ -288,7 +288,10 @@ export class Dispatcher {
}
case 'limiter.get':
return state.limiter;
// applyToRunning is a write-only instruction on limiter.set ("retune already-
// running transfers now"), not a persisted setting, so get never echoes it back
// (contracts/fixtures/limiter.get.json).
return { enabled: state.limiter.enabled, globalBps: state.limiter.globalBps };
case 'limiter.set': {
state.limiter = {