Compare commits
13
Commits
lane/gui
...
4e87730ce9
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4e87730ce9 | ||
|
|
facd851824 | ||
|
|
eb72aa522c | ||
|
|
3e553f08c0 | ||
|
|
1d359af5a3 | ||
|
|
75536304bb | ||
|
|
27865888be | ||
|
|
939a19b7ea | ||
|
|
79c29b47e8 | ||
|
|
8e7d14ba7e | ||
|
|
c1c5c82f8b | ||
|
|
4e177ec809 | ||
|
|
b5c6e1f47d |
@@ -18,8 +18,9 @@ policy so it can be re-applied or audited.
|
||||
| `bootstrap-script-2604` | now — real `--with-clang` install in a 26.04 container; the release the project ships on |
|
||||
| `build (gcc)` / `build (clang)` | now — core, daemon and gui have merged |
|
||||
| `sanitizers (dev)` / `sanitizers (tsan)` | now — core, daemon and gui have merged |
|
||||
| `conformance` | **now — `tests/conformance/` has landed; this is the M0 exit gate** |
|
||||
| `conformance` | **now — `tests/conformance/` has landed; this is the M0 exit gate.** Includes the live-`veloxd` runner (step 3b of `run.sh`), unconditional in the script — see `docs/adr/0019-live-veloxd-conformance-is-required.md`. |
|
||||
| `extension-lint` | now — `extension/` has merged (MV3 manifest + esbuild build) |
|
||||
| `gui-dod` | now — `gui/tests/dod/` has landed (GUI M1 DoD gates R3: `scroll-60fps`, `unhappy-path`); see `tests/integration/README.md#gui-m1-definition-of-done-gates-r3`. `gui-dod-nightly` (`rss-flat`) is schedule-only and cannot be a required PR check. |
|
||||
|
||||
`clang-tidy` is intentionally **not** required through M1 (`continue-on-error: true`,
|
||||
`.clang-tidy` has `WarningsAsErrors: ''`). Make it required at M2.
|
||||
|
||||
@@ -248,3 +248,56 @@ jobs:
|
||||
run: cmake --build --preset dev --target veloxd
|
||||
- name: Nightly integration run
|
||||
run: python3 tests/integration/nightly_run.py --veloxd build/dev/bin/veloxd --tasks 50 --timeout 180
|
||||
|
||||
gui-dod:
|
||||
# Per-PR GUI M1 DoD gates (gui/docs/pkg-qa-requests-m1.md R3): scroll-60fps and
|
||||
# unhappy-path. The 10-minute rss-flat gate is gui-dod-nightly, not here. GUI's
|
||||
# harness defaults QT_QPA_PLATFORM=offscreen itself, so no Xvfb/compositor needed.
|
||||
# See tests/integration/README.md#gui-m1-definition-of-done-gates-r3 for what each
|
||||
# gate catches and the forced-failure transcript proving it isn't vacuous.
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Bootstrap toolchain
|
||||
run: sudo ./tools/bootstrap.sh
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22' # tools/mockd
|
||||
- name: Configure + build
|
||||
run: |
|
||||
cmake --preset dev
|
||||
cmake --build --preset dev --target gui-dod-harness
|
||||
- name: Install mockd
|
||||
run: cd tools/mockd && npm ci
|
||||
- name: Gates
|
||||
run: |
|
||||
gui/tests/dod/run.sh scroll-60fps --json scroll.json
|
||||
gui/tests/dod/run.sh unhappy-path --json unhappy.json
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: gui-dod-${{ github.run_id }}
|
||||
path: "*.json"
|
||||
|
||||
gui-dod-nightly:
|
||||
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Bootstrap toolchain
|
||||
run: sudo ./tools/bootstrap.sh
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
- name: Configure + build
|
||||
run: |
|
||||
cmake --preset dev
|
||||
cmake --build --preset dev --target gui-dod-harness
|
||||
- run: cd tools/mockd && npm ci
|
||||
- name: RSS soak (10 min)
|
||||
run: gui/tests/dod/run.sh rss-flat --json rss.json
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: gui-dod-rss-${{ github.run_id }}
|
||||
path: rss.json
|
||||
|
||||
@@ -12,6 +12,7 @@ Read this before touching anything. Then read your lane brief in [docs/agents/](
|
||||
| GUI | `gui/` | everything else |
|
||||
| EXT | `extension/` | everything else |
|
||||
| PKG/QA | `packaging/`, `.github/`, `tools/testserver/`, `tests/integration/`, `tests/e2e/`, root build files | any lane's feature code |
|
||||
| PORT | `**/platform/<os>/**`, `cmake/platform*.cmake`, `packaging/macos/`, `packaging/windows/` | all logic — see [ADR 0020](docs/adr/0020-cross-platform-strategy.md) |
|
||||
|
||||
If your task seems to require editing another lane's files, that is a signal the interface
|
||||
is wrong. **File the request; don't reach across.**
|
||||
|
||||
+129
@@ -0,0 +1,129 @@
|
||||
.TH VELOX 1 "2026-09-12" "Velox" "User Commands"
|
||||
.SH NAME
|
||||
velox \- command-line client for veloxd, the Velox download manager daemon
|
||||
.SH SYNOPSIS
|
||||
.B velox
|
||||
.I command
|
||||
.RI [ options ]
|
||||
.SH DESCRIPTION
|
||||
.B velox
|
||||
talks to
|
||||
\fBveloxd\fR
|
||||
over its Unix-domain socket and gives a scriptable, terminal-first view of the same
|
||||
downloads the GUI and the Firefox extension see. It does no downloading itself \(em every
|
||||
command is a thin JSON-RPC call; the daemon owns all state.
|
||||
.PP
|
||||
.B veloxd
|
||||
must already be running (see
|
||||
.B ENVIRONMENT
|
||||
below for how
|
||||
.B velox
|
||||
finds it, and
|
||||
.BR systemctl (1)
|
||||
.RI ( "systemctl --user status velox" )
|
||||
for whether it's up). If nothing is listening on the socket,
|
||||
.B velox
|
||||
exits with status 3 rather than hanging.
|
||||
.SH COMMANDS
|
||||
.TP
|
||||
.BI "add " url " [\-\-dir " dir "] [\-\-out " name "] [\-\-segments " n ]
|
||||
Add a new download. Prints the new task's id and initial state.
|
||||
.RS
|
||||
.TP
|
||||
.BI "\-\-dir " dir
|
||||
Destination directory. Must resolve inside one of the daemon's configured
|
||||
.B saveTo.allowedRoots
|
||||
or the call fails; omit to use the configured default download directory.
|
||||
.TP
|
||||
.BI "\-\-out " name
|
||||
Filename to save as. Omit to derive one from the URL (or, once the daemon has probed it,
|
||||
from the server's own
|
||||
.BR Content-Disposition ).
|
||||
.TP
|
||||
.BI "\-\-segments " n
|
||||
Requested connection count for this download, 1\(en32. The daemon may use fewer \(em a
|
||||
per-host cap or a source that turns out not to support resuming both lower it. The
|
||||
.B ls
|
||||
table (and
|
||||
.RI "\-\-json's " segments
|
||||
field) show what was actually granted, not what was asked for.
|
||||
.RE
|
||||
.TP
|
||||
.B ls
|
||||
List every download the daemon knows about: id, state, progress, and filename. With
|
||||
.BR \-\-json ", the raw " download.list " result instead of the table."
|
||||
.TP
|
||||
.BI "pause " id " [" id " ...]"
|
||||
Pause one or more downloads by id. A download already paused, or already finished, is
|
||||
left alone \(em not an error.
|
||||
.TP
|
||||
.BI "resume " id " [" id " ...]"
|
||||
Resume one or more paused downloads.
|
||||
.TP
|
||||
.BI "rm " id " [" id " ...] " "[\-\-delete\-file]"
|
||||
Remove one or more downloads from the list. Without
|
||||
.BR \-\-delete\-file ,
|
||||
any partial data on disk
|
||||
.RI ( .veloxpart / .veloxpart.meta )
|
||||
is discarded but a
|
||||
.B completed
|
||||
file is left in place. With
|
||||
.BR \-\-delete\-file ,
|
||||
the finished file is deleted too \(em there is deliberately no default for this flag; you
|
||||
must say which you mean every time.
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
.B \-\-json
|
||||
Print the raw JSON-RPC result instead of a formatted table. Works with every command;
|
||||
combine with
|
||||
.BR jq (1)
|
||||
for scripting. On error, the JSON form is an
|
||||
.B {"error": {...}}
|
||||
object on stdout rather than a message on stderr.
|
||||
.TP
|
||||
.B \-h ", " \-\-help
|
||||
Print usage and exit 0.
|
||||
.SH EXIT STATUS
|
||||
.TP
|
||||
.B 0
|
||||
Success.
|
||||
.TP
|
||||
.B 1
|
||||
The daemon reached the call but returned a JSON-RPC error (bad task id, path outside the
|
||||
allowed roots, and so on). The message is on stderr, or in the JSON error object with
|
||||
.BR \-\-json .
|
||||
.TP
|
||||
.B 2
|
||||
Usage error \(em missing argument, unknown option, or unknown command.
|
||||
.TP
|
||||
.B 3
|
||||
Could not reach
|
||||
.B veloxd
|
||||
at all: not running, or its socket is missing or unreachable.
|
||||
.SH ENVIRONMENT
|
||||
.TP
|
||||
.B XDG_RUNTIME_DIR
|
||||
.B velox
|
||||
connects to
|
||||
.IR "$XDG_RUNTIME_DIR/velox/velox.sock" .
|
||||
If unset, it falls back to
|
||||
.IR /run/user/ <uid> ,
|
||||
matching
|
||||
\fBveloxd\fR's
|
||||
own resolution \(em the two must agree for the client to find the daemon, so this is
|
||||
normally left to the desktop session's default rather than set by hand.
|
||||
.SH FILES
|
||||
.TP
|
||||
.I $XDG_RUNTIME_DIR/velox/velox.sock
|
||||
The daemon's Unix-domain socket, mode 0600, same-UID only (\fBSO_PEERCRED\fR checked on
|
||||
every connection \(em this is the transport's authorization, not an extra login).
|
||||
.SH SEE ALSO
|
||||
.BR systemctl (1),
|
||||
.BR jq (1)
|
||||
.PP
|
||||
.I docs/01-architecture.md
|
||||
and
|
||||
.I docs/agents/AGENT-DAEMON.md
|
||||
in the Velox source tree for the daemon's own build order and the wire protocol
|
||||
.B velox
|
||||
speaks.
|
||||
@@ -506,7 +506,12 @@ def emit_field_parse(f: Field, indent: str) -> list[str]:
|
||||
f'{i} const auto it = j.find("{f.name}");']
|
||||
if f.optional:
|
||||
# Absent and null mean the same thing: the field is not set. A client that omits
|
||||
# a nullable field and one that sends null are treated identically on purpose.
|
||||
# a nullable field and one that sends null are treated identically on purpose --
|
||||
# correct for create-style params, where there is no existing value to distinguish
|
||||
# "never set" from "explicitly cleared". Patch-style fields need the distinction
|
||||
# (download.update's patch: "an explicit null clears a nullable field") and get an
|
||||
# opt-in exception via x-clearable per ADR 0018 (not implemented yet: this is the
|
||||
# decision record, not the generator change).
|
||||
o.append(f"{i} if (it != j.end() && !it->is_null()) {{")
|
||||
o += emit_value_parse(f.type, "(*it)", "val", "fp", i + " ")
|
||||
o.append(f"{i} out.{m} = std::move(val);")
|
||||
|
||||
@@ -21,7 +21,7 @@ fixtures/
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"name": "download.add — start an ISO now, into the Programs category",
|
||||
"name": "download.add — add an ISO for later, into the Programs category",
|
||||
"description": "Why this case is worth pinning.",
|
||||
"transport": "uds", // optional: replay only on this transport
|
||||
"requires": "...", // optional: a condition a plain server cannot produce
|
||||
@@ -81,3 +81,27 @@ cases in `tests/integration/`.
|
||||
correct response is *no response*: past 750 ms the extension must abandon the offer and let
|
||||
Firefox download normally. A download manager that eats downloads when its daemon is down
|
||||
is worse than no download manager.
|
||||
|
||||
## No fixture may pair a real external URL with `startMode: "now"`
|
||||
|
||||
This suite replays every fixture against a real, live `veloxd` (`tests/conformance/run.sh`),
|
||||
not just `mockd`. `mockd` never actually fetches anything, so it hid this for a while: a
|
||||
fixture with `startMode: "now"` (or `"queue"` into a running queue — anything that gets
|
||||
admitted to the scheduler right away) and a real, resolvable URL makes a **real** daemon
|
||||
actually start downloading it, for real, onto whatever machine runs the suite. This
|
||||
happened — twice, with `download.add.json` pointed at a ~6 GB Ubuntu ISO, straight into the
|
||||
developer's real `~/Downloads`.
|
||||
|
||||
The fix in each case is one of:
|
||||
- `startMode: "later"` — exercises the add path (validation, category assignment, the
|
||||
event) without ever handing the task to the engine;
|
||||
- a URL under `example.org`/`example.com` (IANA-reserved for exactly this, RFC 2606) —
|
||||
resolvable enough to validate as a URL, never a real download source;
|
||||
- `requires`, if the fixture's entire point needs a real transfer to fail in a specific way
|
||||
(see `errors/download.add.disk-full.json`) — skipped by default, so it only ever runs
|
||||
where the condition has actually been arranged.
|
||||
|
||||
A real `saveDir` gets the same treatment for the same reason: an absolute path like
|
||||
`/home/sami/Downloads/...` only means anything on the machine that fixture was written on.
|
||||
Omit `saveDir` and let `saveTo.defaultDir` apply, or use a relative-feeling path under a
|
||||
root the runner controls.
|
||||
|
||||
@@ -1,22 +1,23 @@
|
||||
{
|
||||
"name": "capture.offer — attachment on a monitored type is taken",
|
||||
"description": "Golden fixture. tests/conformance replays this against the real daemon AND the TS client. If either side drifts, this goes red before the lanes ever integrate.",
|
||||
"description": "Golden fixture. tests/conformance replays this against the real daemon AND the TS client. If either side drifts, this goes red before the lanes ever integrate. url is example.org (RFC 2606), not a real download source: 'take' against a real veloxd (tests/conformance/run.sh) admits a real task and hands it to the engine for real, and no fixture may do that against a real external URL. contentLength is a plausible-but-small 5 MiB rather than a real ISO's size: the 'Programs' category's saveDir is a migration-seeded builtin (~/Downloads/Programs, daemon/src/store/migrations/0001_initial.sql), not something an isolated test run's settings can redirect, so 'take' always sparse-preallocates into that real path on whatever machine runs this suite -- keeping the declared size small keeps that footprint trivial instead of a real ISO's worth of disk. transport is uds only: a real 'take' persists an active task, so replaying this same fixture again on a second live transport against the same daemon would correctly dedupe against it (capture.offer dedupes by exact URL) and get 'ignore' instead -- an artifact of replaying one fixture against one shared daemon over two transports, not a behaviour to golden.",
|
||||
"transport": "uds",
|
||||
"request": {
|
||||
"jsonrpc": "2.0",
|
||||
"id": 42,
|
||||
"method": "capture.offer",
|
||||
"params": {
|
||||
"url": "https://releases.ubuntu.com/26.04/ubuntu-26.04-desktop-amd64.iso",
|
||||
"url": "https://example.org/dl/ubuntu-26.04-desktop-amd64.iso",
|
||||
"method": "GET",
|
||||
"tabUrl": "https://releases.ubuntu.com/26.04/",
|
||||
"tabUrl": "https://example.org/26.04/",
|
||||
"headers": {
|
||||
"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:154.0) Gecko/20100101 Firefox/154.0",
|
||||
"Referer": "https://releases.ubuntu.com/26.04/",
|
||||
"Referer": "https://example.org/26.04/",
|
||||
"Accept": "*/*"
|
||||
},
|
||||
"cookies": [],
|
||||
"contentType": "application/octet-stream",
|
||||
"contentLength": 6228541440,
|
||||
"contentLength": 5242880,
|
||||
"contentDisposition": "attachment; filename=\"ubuntu-26.04-desktop-amd64.iso\"",
|
||||
"filename": "ubuntu-26.04-desktop-amd64.iso",
|
||||
"origin": "moz-extension://11111111-2222-3333-4444-555555555555"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "download.add \u2014 start an ISO now, into the Programs category",
|
||||
"description": "The ordinary add path. saveDir is canonicalized and checked against the allowed roots before anything is written.",
|
||||
"name": "download.add — add an ISO for later, into the Programs category",
|
||||
"description": "The ordinary add path. saveDir is canonicalized and checked against the allowed roots before anything is written. startMode is 'later' deliberately: this suite replays against a real veloxd (tests/conformance/run.sh), and a real daemon given startMode 'now' would actually start fetching url for real. No fixture may pair a real external URL with startMode 'now' -- see contracts/fixtures/README.md.",
|
||||
"request": {
|
||||
"jsonrpc": "2.0",
|
||||
"id": 11,
|
||||
@@ -8,10 +8,9 @@
|
||||
"params": {
|
||||
"url": "https://releases.ubuntu.com/26.04/ubuntu-26.04-desktop-amd64.iso",
|
||||
"filename": "ubuntu-26.04-desktop-amd64.iso",
|
||||
"saveDir": "/home/sami/Downloads/Programs",
|
||||
"categoryId": "programs",
|
||||
"segments": 8,
|
||||
"startMode": "now"
|
||||
"startMode": "later"
|
||||
}
|
||||
},
|
||||
"response": {
|
||||
@@ -19,13 +18,13 @@
|
||||
"id": 11,
|
||||
"result": {
|
||||
"taskId": "$uuid",
|
||||
"state": "connecting",
|
||||
"state": "paused",
|
||||
"duplicate": null
|
||||
}
|
||||
},
|
||||
"assertions": [
|
||||
"the .veloxpart file is created sparse and preallocated at the final size",
|
||||
"saveDir resolves inside saveTo.allowedRoots, or the call fails -32011 having written nothing",
|
||||
"saveDir is omitted here on purpose: it resolves to saveTo.defaultDir, which is itself checked against saveTo.allowedRoots the same way an explicit saveDir would be -- see errors/download.add.invalid-path.json for the -32011 case",
|
||||
"startMode 'later' lands the task in 'paused' and never hands it to the engine, so nothing is fetched and no .veloxpart is created yet -- that only happens once the task is actually started (download.start.json, or startMode 'now'/'queue' against a source this suite controls)",
|
||||
"event.task.added is emitted to every subscriber before this reply is sent"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "limiter.get \u2014 the limiter is off",
|
||||
"description": "globalBps still carries the last configured value so the GUI can restore it when the user re-enables the limit.",
|
||||
"description": "globalBps still carries the last configured value so the GUI can restore it when the user re-enables the limit. applyToRunning is a write-only instruction on limiter.set (\"retune already-running transfers now\", not a persisted setting), so it never comes back from get.",
|
||||
"request": {
|
||||
"jsonrpc": "2.0",
|
||||
"id": 52,
|
||||
@@ -12,11 +12,11 @@
|
||||
"id": 52,
|
||||
"result": {
|
||||
"enabled": false,
|
||||
"globalBps": 2097152,
|
||||
"applyToRunning": false
|
||||
"globalBps": 2097152
|
||||
}
|
||||
},
|
||||
"assertions": [
|
||||
"enabled false means no throttling regardless of globalBps"
|
||||
"enabled false means no throttling regardless of globalBps",
|
||||
"applyToRunning is absent, not false: it's meaningless outside a limiter.set call"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -64,6 +64,19 @@ std::string lower(std::string s) {
|
||||
return s;
|
||||
}
|
||||
|
||||
// scheme://host[:port] of `url`, with no path/query/fragment -- what docs/04 §7's "403
|
||||
// after redirect: retry once with the original referrer" retries with as the Referer
|
||||
// header. Empty on an unparseable URL (the caller just won't get a referrer retry).
|
||||
std::string origin_of(std::string_view url) {
|
||||
auto s = net::split_url(url);
|
||||
if (!s.valid)
|
||||
return {};
|
||||
std::string out = s.scheme + "://" + s.host;
|
||||
if (s.port)
|
||||
out += ":" + std::to_string(*s.port);
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
// What to do once every worker has drained (see DownloadTaskState::begin_drain_locked).
|
||||
@@ -88,6 +101,7 @@ struct SegWorker {
|
||||
bool needs_auth = false;
|
||||
bool wrong_status = false;
|
||||
bool range_bad = false;
|
||||
bool forbidden = false; // 403 -- docs/04 §7's "retry once with the original referrer"
|
||||
bool auth_handshake = false; // saw a 401/407 and let libcurl resend with credentials
|
||||
std::string resp_etag, resp_last_modified; // captured on a wrong_status 200, for demote
|
||||
std::optional<ErrorInfo> flush_error;
|
||||
@@ -135,6 +149,16 @@ struct DownloadTaskState : std::enable_shared_from_this<DownloadTaskState> {
|
||||
std::optional<std::uint64_t> total_size;
|
||||
std::string origin_host;
|
||||
|
||||
// docs/04 §7's "403 after redirect: retry once with the original referrer" -- many
|
||||
// CDNs 403 a bare/foreign Referer. Starts as spec.referrer (the browser's, verbatim);
|
||||
// start_worker_locked() sends this, not spec.referrer directly, so a 403 retry can
|
||||
// override it (to the download URL's own origin) without touching what the caller
|
||||
// actually asked for. referrer_retried bounds it to exactly once per task -- a second
|
||||
// 403 with a same-origin Referer already set is a real, honest failure
|
||||
// (Error::forbidden), not something a referrer swap can fix.
|
||||
std::string effective_referrer;
|
||||
bool referrer_retried = false;
|
||||
|
||||
std::unique_ptr<segment::Segmenter> seg;
|
||||
std::unique_ptr<io::SparseFile> file;
|
||||
std::unordered_map<std::uint32_t, std::unique_ptr<SegWorker>> workers;
|
||||
@@ -164,7 +188,7 @@ struct DownloadTaskState : std::enable_shared_from_this<DownloadTaskState> {
|
||||
std::vector<std::function<void()>> deferred;
|
||||
|
||||
DownloadTaskState(TaskHost &h, TaskId i, DownloadSpec s, DownloadCallbacks c)
|
||||
: host(h), id(i), spec(std::move(s)), cbs(std::move(c)) {}
|
||||
: host(h), id(i), spec(std::move(s)), cbs(std::move(c)), effective_referrer(spec.referrer) {}
|
||||
|
||||
// --- deferred callbacks -------------------------------------------------------------
|
||||
void defer(std::function<void()> fn) {
|
||||
@@ -286,7 +310,7 @@ void DownloadTaskState::restart_probe(bool with_auth) {
|
||||
pr.url = spec.url;
|
||||
pr.headers = spec.headers;
|
||||
pr.cookies = spec.cookies;
|
||||
pr.referrer = spec.referrer;
|
||||
pr.referrer = effective_referrer;
|
||||
pr.user_agent = spec.user_agent;
|
||||
pr.proxy = spec.proxy;
|
||||
if (with_auth)
|
||||
@@ -299,12 +323,37 @@ void DownloadTaskState::restart_probe(bool with_auth) {
|
||||
}
|
||||
|
||||
void DownloadTaskState::on_probe_result(Result<net::ProbeResult> r) {
|
||||
bool retry_probe_with_referrer = false;
|
||||
{
|
||||
std::unique_lock lk(mu);
|
||||
if (retired.load() || is_terminal(state))
|
||||
return;
|
||||
if (!r.has_value()) {
|
||||
fail_locked(std::move(r).error());
|
||||
ErrorInfo e = std::move(r).error();
|
||||
// docs/04 §7's referrer retry applies here too: a probe (HEAD, or the
|
||||
// ranged-GET fallback when HEAD is refused -- probe.cpp) can be the request
|
||||
// that actually gets 403'd, before any segment worker exists to retry it
|
||||
// (net::Prober builds its own request from ProbeRequest::referrer, not
|
||||
// through start_worker_locked() -- see restart_probe()'s use of
|
||||
// effective_referrer below). Same one-shot bound via referrer_retried as the
|
||||
// worker-level retry (seg_finished's w->forbidden branch) shares.
|
||||
if (e.code == Error::forbidden && !referrer_retried) {
|
||||
referrer_retried = true;
|
||||
effective_referrer = origin_of(spec.url);
|
||||
retry_probe_with_referrer = true;
|
||||
} else if (e.code == Error::forbidden) {
|
||||
// Already retried with the origin referrer and still 403 -- not something
|
||||
// another blind retry fixes (an expired signed URL, a private resource).
|
||||
// Ask rather than fail outright, the same "ask, don't just fail" shape as
|
||||
// wrong_status/range_bad/the worker-level 403 branch: refresh_url() is a
|
||||
// no-op once the task is terminal, and tools/testserver's expiring-signed-
|
||||
// url mode (also a bare 403, indistinguishable from any other without
|
||||
// parsing the body -- CLAUDE.md §3, core never does) is meant to be
|
||||
// recovered exactly that way.
|
||||
auto_pause_locked(std::move(e), false, true);
|
||||
} else {
|
||||
fail_locked(std::move(e));
|
||||
}
|
||||
} else {
|
||||
probe = std::move(r).value();
|
||||
have_probe = true;
|
||||
@@ -319,6 +368,8 @@ void DownloadTaskState::on_probe_result(Result<net::ProbeResult> r) {
|
||||
}
|
||||
}
|
||||
flush_deferred();
|
||||
if (retry_probe_with_referrer)
|
||||
restart_probe(false);
|
||||
}
|
||||
|
||||
void DownloadTaskState::finish_probe_locked() {
|
||||
@@ -472,7 +523,7 @@ void DownloadTaskState::start_worker_locked(std::uint32_t seg_idx) {
|
||||
req.url = current_url();
|
||||
req.headers = spec.headers;
|
||||
req.cookies = spec.cookies;
|
||||
req.referrer = spec.referrer;
|
||||
req.referrer = effective_referrer;
|
||||
req.user_agent = spec.user_agent;
|
||||
req.proxy = spec.proxy;
|
||||
req.auth = spec.auth;
|
||||
@@ -552,6 +603,10 @@ net::DataAction DownloadTaskState::seg_head(std::uint32_t seg_idx, const net::Re
|
||||
w->range_bad = true;
|
||||
return net::DataAction::abort;
|
||||
}
|
||||
if (h.status == 403) {
|
||||
w->forbidden = true;
|
||||
return net::DataAction::abort;
|
||||
}
|
||||
if (h.status >= 400)
|
||||
return net::DataAction::abort;
|
||||
seg->set_segment_state(seg_idx, segment::SegState::downloading);
|
||||
@@ -637,7 +692,7 @@ void DownloadTaskState::seg_finished(std::uint32_t seg_idx, Result<net::Transfer
|
||||
// (content-length-mismatch's honest-length lie, flaky-reset's tail, a proxy RST after
|
||||
// the last byte). If the segment is fully covered, that's a success.
|
||||
if (seg && !cancel_requested && !pause_requested && !w->needs_auth && !w->wrong_status &&
|
||||
!w->flush_error && !w->range_bad) {
|
||||
!w->flush_error && !w->range_bad && !w->forbidden) {
|
||||
const std::uint64_t len = seg->segment_end(seg_idx) - seg->segment_start(seg_idx) + 1;
|
||||
if (len != 0 && seg->segment_completed(seg_idx) >= len) {
|
||||
r = Result<net::TransferStats>(net::TransferStats{});
|
||||
@@ -756,6 +811,35 @@ void DownloadTaskState::seg_finished(std::uint32_t seg_idx, Result<net::Transfer
|
||||
auto_pause_locked(ErrorInfo(Error::range_not_satisfiable, "416"), false, true);
|
||||
return done();
|
||||
}
|
||||
if (w->forbidden) {
|
||||
// docs/04 §7: "403 after redirect: retry once with the original referrer -- many
|
||||
// CDNs require it." Bare/foreign Referer is the common cause; origin_of() rebuilds
|
||||
// it from the (possibly redirected) URL the response actually came from. Exactly
|
||||
// once per task, not a backoff series -- a second 403 with a same-origin Referer
|
||||
// already set isn't something another blind retry can fix (a private/expired
|
||||
// resource, an expiring signed URL past its window, ...). That's not necessarily
|
||||
// terminal, though: ask (same "ask, don't just fail outright" shape as
|
||||
// wrong_status/range_bad above) rather than fail_locked() outright, specifically
|
||||
// so DownloadHandle::refresh_url() -- do_refresh_url() is a no-op once the task is
|
||||
// terminal -- stays usable for the case tools/testserver's README pairs it with:
|
||||
// a caller that gets a fresh signed URL and hands it back.
|
||||
release_slot();
|
||||
if (!referrer_retried) {
|
||||
referrer_retried = true;
|
||||
effective_referrer = origin_of(current_url());
|
||||
seg->set_segment_state(seg_idx, segment::SegState::stalled);
|
||||
auto wp = weak_from_this();
|
||||
host.schedule(std::chrono::steady_clock::now(), [wp, seg_idx] {
|
||||
if (auto s = wp.lock())
|
||||
s->retry_worker(seg_idx);
|
||||
});
|
||||
if (workers.empty())
|
||||
transition(EngineState::retry_wait, std::nullopt);
|
||||
} else {
|
||||
auto_pause_locked(ErrorInfo(Error::forbidden, "403", w->http_status), false, true);
|
||||
}
|
||||
return done();
|
||||
}
|
||||
|
||||
if (!r.has_value()) {
|
||||
ErrorInfo e = std::move(r).error();
|
||||
@@ -1215,6 +1299,7 @@ void DownloadTaskState::do_refresh_url(std::string url, std::vector<net::HeaderF
|
||||
net::ProbeRequest pr;
|
||||
pr.url = spec.url;
|
||||
pr.headers = spec.headers;
|
||||
pr.referrer = effective_referrer;
|
||||
pr.auth = spec.auth;
|
||||
pr.proxy = spec.proxy;
|
||||
host.probe(std::move(pr), [wp](Result<net::ProbeResult> r) {
|
||||
@@ -1224,10 +1309,43 @@ void DownloadTaskState::do_refresh_url(std::string url, std::vector<net::HeaderF
|
||||
std::unique_lock lk(s->mu);
|
||||
if (s->retired.load() || is_terminal(s->state))
|
||||
return;
|
||||
if (r.has_value()) {
|
||||
s->probe.effective_url = r.value().effective_url;
|
||||
s->probe.etag = r.value().etag;
|
||||
s->probe.last_modified = r.value().last_modified;
|
||||
if (!r.has_value()) {
|
||||
lk.unlock();
|
||||
s->flush_deferred();
|
||||
return; // still paused; the caller can retry refresh_url() or decide()
|
||||
}
|
||||
if (!s->have_probe) {
|
||||
// The task's *first* probe never succeeded (e.g. this session's own
|
||||
// expiring-signed-url path: 403, one referrer retry, still 403 -> ask rather
|
||||
// than fail outright -- see on_probe_result() -- specifically so this branch
|
||||
// exists to recover it). finish_probe_locked() is what actually registers the
|
||||
// task with the budget and builds its Segmenter; nothing downstream of a
|
||||
// partial field copy would ever start a worker without it.
|
||||
s->probe = std::move(r).value();
|
||||
s->have_probe = true;
|
||||
s->awaiting_auth = false;
|
||||
s->awaiting_decision = false;
|
||||
s->finish_probe_locked();
|
||||
lk.unlock();
|
||||
s->flush_deferred();
|
||||
return;
|
||||
}
|
||||
s->probe.effective_url = r.value().effective_url;
|
||||
s->probe.etag = r.value().etag;
|
||||
s->probe.last_modified = r.value().last_modified;
|
||||
// refresh_url()'s own contract is "on a live OR PAUSED task, without losing
|
||||
// progress" -- distinct from do_decide(restart), which discards progress. A task
|
||||
// can be paused here for any of three reasons (a plain user pause, awaiting_auth,
|
||||
// or awaiting_decision -- e.g. this session's own 403-after-referrer-retry path,
|
||||
// or the pre-existing wrong_status/range_bad ones); apply_slot_target()'s guard
|
||||
// blocks on awaiting_auth/awaiting_decision specifically, so leaving either set
|
||||
// would have set_want() below recompute a target that nothing ever acts on --
|
||||
// the caller's new URL re-probed successfully and then the task just sat there.
|
||||
// Clear both and leave `paused` the same way do_decide(restart) does.
|
||||
if (s->state == EngineState::paused) {
|
||||
s->awaiting_auth = false;
|
||||
s->awaiting_decision = false;
|
||||
s->transition(EngineState::connecting, std::nullopt);
|
||||
}
|
||||
if (s->registered)
|
||||
s->host.budget().set_want(s->id, s->want_slots());
|
||||
|
||||
@@ -28,7 +28,14 @@ namespace vdm::testing {
|
||||
|
||||
class TestServer {
|
||||
public:
|
||||
TestServer() {
|
||||
TestServer() : TestServer(1.0) {}
|
||||
|
||||
// loris_seconds overrides the dribble duration slow-loris mode uses (default matches the
|
||||
// no-arg ctor's long-standing 1s). A test that needs curl's stall detector
|
||||
// (CURLOPT_LOW_SPEED_TIME, hardcoded to 30s in download_task.cpp) to actually fire needs a
|
||||
// dribble that outlasts that threshold, not the short one every other test relies on to
|
||||
// keep runtime down.
|
||||
explicit TestServer(double loris_seconds) {
|
||||
const char *script = VDM_TESTSERVER_PY;
|
||||
if (!script || !*script || ::access(script, R_OK) != 0)
|
||||
return;
|
||||
@@ -50,8 +57,9 @@ class TestServer {
|
||||
int devnull = ::open("/dev/null", O_WRONLY);
|
||||
if (devnull >= 0)
|
||||
::dup2(devnull, STDERR_FILENO);
|
||||
std::string loris_str = std::to_string(loris_seconds);
|
||||
::execlp("python3", "python3", script, "--port", "0", "--seed", "9", "--loris-seconds",
|
||||
"1", "--throttle-bps", "131072", static_cast<char *>(nullptr));
|
||||
loris_str.c_str(), "--throttle-bps", "131072", static_cast<char *>(nullptr));
|
||||
::_exit(127);
|
||||
}
|
||||
::close(pipefd[1]);
|
||||
|
||||
@@ -124,6 +124,31 @@ std::string server_sha(TestServer &srv, const std::string &mode, const std::stri
|
||||
return out.substr(open + 1, close - open - 1);
|
||||
}
|
||||
|
||||
// Small, deliberately identical extraction to server_sha's: GET /<mode>/sign/<size>?ttl=N
|
||||
// and pull the "url" field's value out of the {"url":..., "exp":...} JSON body.
|
||||
std::string sign_url(TestServer &srv, const std::string &mode, const std::string &size,
|
||||
int ttl_seconds) {
|
||||
std::string url =
|
||||
srv.url("/" + mode + "/sign/" + size + "?ttl=" + std::to_string(ttl_seconds));
|
||||
std::string cmd = "curl -s '" + url + "'";
|
||||
std::string out;
|
||||
if (FILE *f = ::popen(cmd.c_str(), "r")) {
|
||||
char buf[1024];
|
||||
while (std::fgets(buf, sizeof buf, f))
|
||||
out += buf;
|
||||
::pclose(f);
|
||||
}
|
||||
auto q = out.find("\"url\"");
|
||||
if (q == std::string::npos)
|
||||
return {};
|
||||
auto colon = out.find(':', q);
|
||||
auto open = out.find('"', colon);
|
||||
auto close = out.find('"', open + 1);
|
||||
if (open == std::string::npos || close == std::string::npos)
|
||||
return {};
|
||||
return out.substr(open + 1, close - open - 1);
|
||||
}
|
||||
|
||||
DownloadSpec spec_for(TestServer &srv, const std::string &urlpath, const std::string &save) {
|
||||
DownloadSpec s;
|
||||
s.url = srv.url(urlpath);
|
||||
@@ -328,6 +353,30 @@ VT_TEST(engine_401_then_provide_auth_completes) {
|
||||
VT_CHECK_EQ(file_size(td.file("au.bin")), 1u * 1024 * 1024);
|
||||
}
|
||||
|
||||
VT_TEST(engine_401_digest_then_provide_auth_completes) {
|
||||
// Same shape as engine_401_then_provide_auth_completes, but the challenge is HTTP
|
||||
// Digest (qop=auth) rather than Basic. provide_auth() doesn't know or care which --
|
||||
// http_client.cpp always asks libcurl for CURLAUTH_ANY (net::AuthScheme::any) and lets
|
||||
// curl negotiate against whatever WWW-Authenticate the server actually sent -- so this
|
||||
// exists purely to prove that's true end-to-end, not just at the unit level.
|
||||
TestServer srv;
|
||||
VT_REQUIRE(srv.available());
|
||||
TmpDir td;
|
||||
Recorder rec;
|
||||
Engine eng;
|
||||
DownloadHandle h;
|
||||
auto cbs = rec.cbs(&h, "test", "test");
|
||||
h = eng.start(spec_for(srv, "/401-digest/file/1M", td.file("dg.bin")), std::move(cbs));
|
||||
rec.arm(h);
|
||||
|
||||
auto r = rec.wait();
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK(rec.auth_calls.load() >= 1);
|
||||
VT_CHECK_EQ(file_size(td.file("dg.bin")), 1u * 1024 * 1024);
|
||||
auto got = hash_file(td.file("dg.bin"), Checksum::Algo::sha256);
|
||||
VT_CHECK_EQ(got.value(), server_sha(srv, "401-digest", "1M"));
|
||||
}
|
||||
|
||||
// --- hostile-mode matrix: the four where a bug is silent corruption, not a visible
|
||||
// failure (docs/04 §5 "ask, never silently corrupt" / §7's failure-policy table). ---
|
||||
|
||||
@@ -458,6 +507,142 @@ VT_TEST(engine_content_length_mismatch_fails_honestly) {
|
||||
VT_CHECK_EQ(::access(td.file("clm.bin").c_str(), F_OK), -1); // never renamed into place
|
||||
}
|
||||
|
||||
// --- remaining hostile-mode matrix (tools/testserver/README.md's mode table). ---
|
||||
|
||||
VT_TEST(engine_expiring_signed_url_recovers_via_refresh_url) {
|
||||
// A signed URL past its ttl 403s (tools/testserver's own JSON body distinguishes
|
||||
// "expired" from "bad signature", but core never parses response bodies -- CLAUDE.md
|
||||
// §3 -- so both just read as a 403). The one automatic referrer retry (see
|
||||
// engine_403_without_referer_retries_with_origin, below) can't fix an expired
|
||||
// signature, so the second 403 asks -- via the same auto_pause_locked(..., false,
|
||||
// true) "ask, don't just fail" path as wrong_status/range_bad -- rather than
|
||||
// terminally failing outright, specifically so DownloadHandle::refresh_url() (its own
|
||||
// contract: works "on a live or paused task", never on a terminal one) stays usable:
|
||||
// the README pairs this mode with exactly that recovery.
|
||||
TestServer srv;
|
||||
VT_REQUIRE(srv.available());
|
||||
TmpDir td;
|
||||
Recorder rec;
|
||||
Engine eng;
|
||||
|
||||
std::string expired = sign_url(srv, "expiring-signed-url", "64K", /*ttl=*/1);
|
||||
VT_REQUIRE(!expired.empty());
|
||||
std::this_thread::sleep_for(1500ms); // let the ttl actually pass before the first request
|
||||
|
||||
DownloadSpec s;
|
||||
s.url = expired;
|
||||
s.save_path = td.file("exp.bin");
|
||||
auto h = eng.start(std::move(s), rec.cbs());
|
||||
|
||||
for (int i = 0; i < 300 && rec.decision_calls.load() == 0; ++i)
|
||||
std::this_thread::sleep_for(20ms);
|
||||
VT_REQUIRE(rec.decision_calls.load() >= 1);
|
||||
VT_CHECK_EQ(h.state(), EngineState::paused);
|
||||
|
||||
std::string fresh = sign_url(srv, "expiring-signed-url", "64K", /*ttl=*/60);
|
||||
VT_REQUIRE(!fresh.empty());
|
||||
h.refresh_url(fresh);
|
||||
|
||||
auto r = rec.wait(60s);
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK_EQ(file_size(td.file("exp.bin")), 64u * 1024);
|
||||
auto got = hash_file(td.file("exp.bin"), Checksum::Algo::sha256);
|
||||
VT_CHECK_EQ(got.value(), server_sha(srv, "expiring-signed-url", "64K"));
|
||||
}
|
||||
|
||||
VT_TEST(engine_403_without_referer_retries_with_origin) {
|
||||
// docs/04 §7: "403 after redirect: retry once with the original referrer -- many CDNs
|
||||
// require it." No spec.referrer is set here (the common case for anything not
|
||||
// initiated from a browser page, e.g. `velox add <url>`), so the first attempt 403s;
|
||||
// the engine's own retry supplies the download URL's own origin as Referer, which
|
||||
// this mode accepts, and the download completes with no decision ever asked.
|
||||
TestServer srv;
|
||||
VT_REQUIRE(srv.available());
|
||||
TmpDir td;
|
||||
Recorder rec;
|
||||
Engine eng;
|
||||
auto h = eng.start(spec_for(srv, "/403-without-referer/file/128K", td.file("ref.bin")),
|
||||
rec.cbs());
|
||||
auto r = rec.wait(30s);
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK_EQ(rec.decision_calls.load(), 0); // recovered automatically, not asked
|
||||
VT_CHECK_EQ(file_size(td.file("ref.bin")), 128u * 1024);
|
||||
auto got = hash_file(td.file("ref.bin"), Checksum::Algo::sha256);
|
||||
VT_CHECK_EQ(got.value(), server_sha(srv, "403-without-referer", "128K"));
|
||||
}
|
||||
|
||||
VT_TEST(engine_redirect_chain_follows_to_completion) {
|
||||
// 5 hops (tools/testserver's own --redirect-depth default) of a plain 302, query
|
||||
// string preserved across each. No CORE-side logic needed for this one -- libcurl's
|
||||
// own CURLOPT_FOLLOWLOCATION (RequestOptions::follow_redirects, already on) and
|
||||
// CURLOPT_MAXREDIRS (default 20, well over 5) do the whole thing -- this is here as
|
||||
// the end-to-end check that they're actually wired through both the probe and every
|
||||
// segment worker's own request, not just one of the two.
|
||||
TestServer srv;
|
||||
VT_REQUIRE(srv.available());
|
||||
TmpDir td;
|
||||
Recorder rec;
|
||||
Engine eng;
|
||||
auto h = eng.start(spec_for(srv, "/redirect-chain/file/1M", td.file("rc.bin")), rec.cbs());
|
||||
auto r = rec.wait(30s);
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK_EQ(file_size(td.file("rc.bin")), 1u * 1024 * 1024);
|
||||
auto got = hash_file(td.file("rc.bin"), Checksum::Algo::sha256);
|
||||
VT_CHECK_EQ(got.value(), server_sha(srv, "redirect-chain", "1M"));
|
||||
}
|
||||
|
||||
VT_TEST(engine_slow_loris_stall_timeout_fires) {
|
||||
// Status line, headers, and body dribbled out one byte at a time for --loris-seconds,
|
||||
// then (if the dribble hasn't already been cut off) normal streaming -- a connection
|
||||
// that's technically alive (bytes ARE arriving, just far too slowly) but must not be
|
||||
// allowed to hang the task forever. http_client.cpp sets CURLOPT_LOW_SPEED_LIMIT/_TIME
|
||||
// (RequestOptions::low_speed_bytes_per_sec/low_speed_secs, hardcoded in
|
||||
// download_task.cpp to 1024 B/s for 30s) for exactly this.
|
||||
//
|
||||
// Every other test in this file uses TestServer's default 1s loris dribble to keep
|
||||
// runtime down, but 1s is far shorter than curl's 30s low_speed_time: a 1s trickle
|
||||
// followed by full-speed streaming never accumulates 30 CONSECUTIVE seconds under the
|
||||
// floor, so curl would never actually abort it -- the download would just complete
|
||||
// slightly late, which would make this test pass for the wrong reason (or not exercise
|
||||
// the stall timeout at all). Explicitly ask for a dribble that outlasts the 30s
|
||||
// threshold so the stall timeout is the thing actually observed firing, not assumed.
|
||||
TestServer srv(40.0);
|
||||
VT_REQUIRE(srv.available());
|
||||
TmpDir td;
|
||||
Recorder rec;
|
||||
Engine eng;
|
||||
auto s = spec_for(srv, "/slow-loris/file/64K", td.file("sl.bin"));
|
||||
s.segments = 1;
|
||||
s.max_retries = 1;
|
||||
auto h = eng.start(std::move(s), rec.cbs());
|
||||
auto r = rec.wait(60s); // stall timeout fires ~30s in; must resolve, not hang to 60s
|
||||
VT_REQUIRE(!r.has_value());
|
||||
VT_CHECK(is_retryable(r.error().code) || r.error().code == Error::max_retries_exhausted);
|
||||
}
|
||||
|
||||
VT_TEST(engine_chunked_no_length_completes_single_segment) {
|
||||
// No Content-Length anywhere (HEAD gets none either, since it's the same handler path)
|
||||
// -- the probe can't know total_size or prove resumability, so this should take the
|
||||
// exact same "unknown size, one plain-GET segment" path as engine_non_resumable_single_
|
||||
// segment, just arriving there via a chunked body instead of a server that plainly
|
||||
// refuses Range. No core-side work needed if that demotion is already size-agnostic;
|
||||
// this is here to prove it, since every other test's server tells the probe the size
|
||||
// up front.
|
||||
TestServer srv;
|
||||
VT_REQUIRE(srv.available());
|
||||
TmpDir td;
|
||||
Recorder rec;
|
||||
Engine eng;
|
||||
auto h = eng.start(spec_for(srv, "/chunked-no-length/file/2M", td.file("ch.bin")),
|
||||
rec.cbs());
|
||||
auto r = rec.wait();
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK_EQ(rec.decision_calls.load(), 0);
|
||||
VT_CHECK_EQ(file_size(td.file("ch.bin")), 2u * 1024 * 1024);
|
||||
auto got = hash_file(td.file("ch.bin"), Checksum::Algo::sha256);
|
||||
VT_CHECK_EQ(got.value(), server_sha(srv, "chunked-no-length", "2M"));
|
||||
}
|
||||
|
||||
// --- DAEMON-reported bug: Progress.speed_bps reads 0 for the whole life of a live
|
||||
// download while downloaded bytes visibly advance. DAEMON reads progress by polling
|
||||
// DownloadHandle::progress() (engine_port_core.hpp), not the on_progress push callback --
|
||||
|
||||
@@ -75,6 +75,7 @@ target_link_libraries(veloxd_sched PUBLIC velox::proto velox::core veloxd_store
|
||||
add_library(veloxd_rpc STATIC
|
||||
src/rpc/runtime_dir.cpp
|
||||
src/rpc/single_instance.cpp
|
||||
src/rpc/systemd_activation.cpp
|
||||
src/rpc/event_loop.cpp
|
||||
src/rpc/event_hub.cpp
|
||||
src/rpc/uds_server.cpp
|
||||
|
||||
@@ -7,7 +7,7 @@ close. Kept here (not buried in commit messages) so the next pass can see them a
|
||||
|---|---|---|---|---|
|
||||
| ~~D7~~ | **Closed — `capture.offer` is real.** Applies `capture.enabled`/`excludedHosts`/`monitoredExtensions`/`monitoredMimeTypes`/`minSizeBytes` from settings, then the rules table (`store::Rules` + CORE's `vdm::rules::match_rules`/`glob_match` — DAEMON only converts its own stored `proto::Rule` JSON into CORE's plain `vdm::rules::Rule` vocabulary, per that header's own layering note), resolves the category folder (a rule's explicit `categoryId`/`saveDir`, else `store::Categories::guess_by_extension` — the same extension-guess `download.probe`'s `suggestedCategoryId` already used, now shared instead of duplicated), dedupes against active (non-terminal) tasks by exact URL, and on `take` calls `add_one()` — the same path `download.add` itself uses — so a captured download is a real, admitted, persisted task, not a special case. The 750 ms deadline (CLAUDE.md §4 / AGENT-DAEMON.md build step 6) is checked cooperatively between every step via a new `rpc::CaptureDataSource` seam (real impl wraps `store::*`; a test fake can jump its own clock forward to simulate "the store was slow just now" with zero real sleep) — catches the realistic failure mode (several slow steps adding up) though it can't preempt one pathologically stuck single call. Verified against real `veloxd` + `tools/testserver`: a monitored-type offer answers in ~5ms and actually creates + downloads the task; an unmonitored type, an excluded host, a rule-vetoed host, and a second offer for a still-active URL all answer `ignore` with the right `reason`; a bad category save dir surfaces its real `-32011` rather than being swallowed. New `capture_offer_test` covers all of the above plus the deadline itself (two cases, one per "slow" checkpoint), asserting real wall-clock time barely moves even though the fake clock jumped 2 simulated seconds — proof the check reads the injected clock, not a disguised sleep. | `rpc/capture_data_source.hpp`, `rpc/dispatcher.{hpp,cpp}`, `store/rules.{hpp,cpp}`, `store/categories.{hpp,cpp}`, `store/tasks.{hpp,cpp}` | — | done |
|
||||
| ~~D8~~ | **Closed alongside D7** — `capture.getRules` returns the same settings-backed `enabled`/`monitoredExtensions`/`monitoredMimeTypes`/`minSizeBytes`/`excludedHosts`/`bypassModifier` capture.offer itself reads, so the two can never drift. `rulesVersion` is a constant `1` — there is no persisted revision counter yet (nothing writes `rules.*` outside this process's own lifetime to need one across a restart), and the extension already re-fetches on `event.settings.changed` regardless of what this number does; noted in case a real counter becomes worth adding later. | `rpc/dispatcher.cpp` | `rulesVersion` is a placeholder constant | — |
|
||||
| D1 | Pairing prompt is `EnvAutoApprover` (needs `VELOX_PAIR_AUTO=1`) | `rpc/pairing.hpp`, `main.cpp` | A GUI dialog / `org.freedesktop.Notifications` approver is integration work | Build step 7 (systemd + notifications) |
|
||||
| D1 | Pairing prompt is `EnvAutoApprover` (needs `VELOX_PAIR_AUTO=1`) | `rpc/pairing.hpp`, `main.cpp` | A GUI dialog / `org.freedesktop.Notifications` approver is integration work | the `org.freedesktop.Notifications` half of build step 7 — the systemd half closed as D11 below |
|
||||
| — | **D1, checked this pass, not attempted:** `libdbus-1-dev` (or `libsystemd-dev` for `sd-bus`) has no headers installed in this build environment — only the runtime `.so`s (`dpkg -l`/`apt-cache policy` confirm `libdbus-1-3` present, `libdbus-1-dev` not, "Candidate" available but not installed). A real notification-backed approver needs one of those linked into `veloxd`, which is a new build dependency for `daemon/CMakeLists.txt` (`find_package`/`pkg_check_modules`) and — since packaging manifests need to know about it too — arguably a decision to surface rather than something to reach for silently mid-session. `PairingApprover::approve()` is also still synchronous by shape (its own doc comment already says so: "the real notification-backed approver will run async and is not this shape") — swapping it for the async pattern this session built for `download.probe` (`rpc::TaskActionPort` + the server-layer deferred-reply special-case) is the right shape once there's a real implementation to justify the churn; reshaping the interface with nothing behind it yet would just be churn. Left `EnvAutoApprover` in place rather than build a fragile hand-rolled D-Bus wire client to avoid the missing headers — a broken pairing approver is worse than an honest stub. | `rpc/pairing.hpp` | missing dev headers + an undiscussed new dependency | once `libdbus-1-dev`/`libsystemd-dev` is available and the dependency is approved |
|
||||
| ~~D2~~ | **Closed** — `download.probe` is real on both transports. It's genuinely async (the engine's probe pool, up to the schema's 30s `x-deadlineMs`) and so cannot fit `VeloxDispatcher::on_download_probe`'s synchronous `HandlerResult<T>` return — `uds_server.cpp`/`ws_server.cpp` special-case `"download.probe"` before the generic `dispatch()`, exactly the way they already special-case `session.hello`/`session.subscribe`, and queue the reply whenever the callback fires. `rpc::TaskActionPort::probe_now` (kept in proto/std terms, no `vdm::net::*`, so `veloxd_rpc` never needs `core/include`'s vdm headers) is what both transports call; `sched::Scheduler::probe_now` is the implementation — builds a `vdm::net::ProbeRequest`, runs it on the engine's probe pool, maps a failure to `-32013 ProbeFailed` (with `data.httpStatus` when there was one), and fills `suggestedCategoryId`/`suggestedSaveDir` with a plain extension match against the categories table (not the real rules engine — that's still D3). Verified live: a real probe answers in ~5ms; a bad host maps to `-32013`; a connection issuing a 10s `slow-loris` probe does not block a second connection's `download.list` (answered in ~1ms) — confirms the async design actually keeps the loop free, not just compiles. | `rpc/task_action_port.hpp`, `rpc/{uds_server,ws_server}.{hpp,cpp}`, `sched/scheduler.{cpp,hpp}` | — | done |
|
||||
| D3 | Stub handlers for the rest: `grabber.*`, `media.*` | `rpc/dispatcher.cpp` | HLS/DASH grabber and media-variant support don't exist anywhere in this build yet — a bigger feature than a store-wiring pass | M4 territory, per AGENT-DAEMON.md |
|
||||
@@ -26,4 +26,5 @@ close. Kept here (not buried in commit messages) so the next pass can see them a
|
||||
| ~~D4b~~ | **Closed** — `download.pause`/`resume`/`start`/`cancel` and `queue.start`/`stop` all drive the scheduler now, and apply *immediately* (not deferred to the next tick — pausing/resuming/cancelling a live transfer can't wait up to 1s, and per ADR 0013 §3 the governor never touches a user-owned pause on its own). New `rpc::TaskActionPort` interface (owned by `rpc/`, implemented by `sched::Scheduler`) is the seam dispatcher.hpp depends on instead of `sched/scheduler.hpp` directly — avoids a real `veloxd_rpc` <-> `veloxd_sched` circular library dependency (`veloxd_sched` already links `veloxd_rpc` for `EventHub`). `Scheduler::user_pause/resume/start/cancel` + `pause_queue` engine-call-then-eager-transition, matching `tick()`'s existing `to_pause` pattern. Fixed a real bug hit while building this: `transition()` always overwrote `pause_reason` to NULL when the engine's own delayed pause-ack callback arrived with no explicit reason, clobbering whatever the actual initiator (user or governor) had just written — now it preserves the stored reason when none is supplied. Verified against real `veloxd` + `tools/testserver`: pausing a live single-segment throttled transfer freezes `downloadedBytes`, resume continues it from that point, cancel stops it; `queue.stop(pauseRunning:true)` pauses the queue's running task immediately. NOTE: `download.start`'s contract "a task in 'queued' jumps its queue" (priority bump) is not implemented — admission is still plain FIFO by `created_at`. | `sched/scheduler.{cpp,hpp}`, `rpc/task_action_port.hpp`, `rpc/dispatcher.{hpp,cpp}`, `store/queues.{cpp,hpp}` | — | done, except the queue-jump priority bump noted above |
|
||||
| ~~D5~~ | **Mostly closed** — `rpc/event_hub` fans out per-subscription; `session.subscribe` on both transports registers/updates/tears down a real subscription; `Scheduler::transition()` publishes `event.task.state` (with `previousState`) on every state change, scheduler-driven or engine-reported; `dispatcher::on_download_add` publishes `event.task.added`; a 250 ms timer batches `Scheduler::progress_snapshot()` into one `event.task.progress` array per AGENT-DAEMON.md item 5 / the schema's `x-maxRateHz: 4`. Verified live end to end. | — | `event.task.removed` has no source yet (`download.remove` is D3); `event.speed.global`, `event.notify`, `event.auth.required`, `event.settings.changed`, `event.grabber.progress` are unpublished — each lands with its owning handler | as each owning D3 handler lands |
|
||||
| ~~D6~~ | **Closed** — engine numbers now reach the store: `Scheduler::tick()` probes (`EnginePort::probe`) before every `start()`, persisting `sizeBytes`/`resumable`/validators via `Tasks::set_probe_result` before a byte moves; `Scheduler::persist_progress()` (called from `progress_snapshot()` *and* once more from `on_engine_state` right before `release()`/unmap on every terminal transition) writes `downloadedBytes`/`speedBps`/`segments`/`segmentDetail` from the engine's `Progress`, so a task that finishes between two 250 ms ticks (the common case for anything small or fast) still leaves real numbers instead of the pre-persistence defaults. `TaskSummary.segments` is sourced from `segments.size()` when the task has any (matching what actually lands in `segmentDetail`, per the schema's "exactly `segments` entries"), falling back to the engine's `effective_segments` (budget slots *held*, not necessarily physical range count — see `core/include/vdm/task/download.hpp`'s `Progress` comment) only pre-segmentation. `Tasks::set_final_bytes` tops up `on_finished`'s byte count as a last-resort backstop. Migration `0002` adds `speed_bps` to both `tasks` and `segments`, and fixes `segments.state`'s CHECK to include `'pending'` (0001 omitted it, so a pre-connect snapshot could never be written). Verified against real `veloxd` + `tools/testserver` (not just unit tests): `download.list`/`download.get` correct immediately after completion and after a daemon restart. | `sched/scheduler.{cpp,hpp}`, `store/{tasks,segments}.{cpp,hpp}`, `store/migrations/0002_*.sql` | — | done |
|
||||
| ~~D11~~ | **Closed — build order items 7 (the systemd half) and 9: `velox-nmhost`, socket activation, the systemd user units, and `velox(1)`.** `nmhost/src/main.cpp` (185 lines): a `poll()`-driven byte pump between Firefox's native-messaging framing on stdio (4-byte native-byte-order length prefix) and `veloxd`'s own NDJSON framing on the Unix socket — reframes each direction, no JSON parsing, no retry/backoff, exits the moment either side closes. Deliberately dependency-free (no `veloxd_*` library, no `nlohmann_json`) since it runs unconfined outside Firefox's sandbox whatever the packaging format. Two real bugs found and fixed while getting the integration test to actually pass rather than hang: (1) never set the pumped fds non-blocking, so the "drain what's available" read loop blocked on its own second `read()` instead of returning to `poll()`; (2) stdin and stdout are two different descriptors (0 and 1), not one — an early draft polled `POLLOUT` on fd 0, which is opened read-only, so EOF/writability were never both observable through the same `pollfd` entry. Both are exactly the class of bug a "trivial pump" invites and unit tests over the real binary (not just its helper functions) exist specifically to catch. `packaging/nativehost/com.velox.host.json` + its own `README.md` supersede `AGENT-DAEMON.md`'s stale "four locations" (spike S1 / ADR 0003 found only three are real — the fourth, `~/snap/firefox/common/.mozilla/...`, is not read by snap Firefox at all) and spell out the per-user-manifest / postinst implication for PKG/QA. `EnginePort`-style: `rpc/systemd_activation.cpp` is a from-scratch `sd_listen_fds()` (env vars only, no `libsystemd` link — `LISTEN_PID`/`LISTEN_FDS`, fd 3) that `UdsServer::start()` checks first, skipping its own create/bind/chmod/listen when systemd already bound the socket; `packaging/systemd/velox.socket` + `velox.service` are the unit pair, verified both by `systemd-analyze verify` and by an actual fork/dup2/execve simulation of the activation handshake (a real `session.hello` round-tripped over the handed-off fd with no `bind()` ever called inside the daemon for that run). `velox.service` deliberately skips `ProtectSystem=`/`ProtectHome=`/`ReadWritePaths=` — `saveTo.allowedRoots` is user-configurable to anywhere on the filesystem, and a sandbox here would turn a legitimately-configured save location into an opaque `EROFS`/`EACCES` instead of the daemon's own clear `-32011`. `cli/man/velox.1` documents the CLI as it actually exists today (`add`/`ls`/`pause`/`resume`/`rm`, `--json`, the three-tier `queue`/`settings` subcommands `AGENT-DAEMON.md` build step 8 originally sketched are not implemented in `cli/src/main.cpp` yet, so the page doesn't claim they are) — checked warning-free with `groff -mandoc -ww -z`. | `nmhost/{CMakeLists.txt,src/main.cpp,tests/}`, `daemon/src/rpc/{systemd_activation.{hpp,cpp},uds_server.cpp}`, `packaging/{nativehost,systemd}/`, `cli/man/velox.1` | — | done |
|
||||
| — | ~~Observed, not fixed (CORE, not this lane)~~ — **routed to CORE by the user.** `vdm::task::Progress.speed_bps` reads back as `0` for the whole lifetime of a live, real (non-fake) throttled download, despite `downloadedBytes` visibly advancing between polls — `core/src/task/download_task.cpp`'s per-worker EWMA never seems to produce a nonzero aggregate in this build. DAEMON passes `EnginePort::progress()`'s `speed_bps` straight through (`Scheduler::persist_progress`); nothing in this lane drops it. Still reproduces in the D4b live checks above (0 throughout a paused/resumed/cancelled transfer whose `downloadedBytes` visibly moved) — not re-filed, since it's already CORE's. |
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
#include "rpc/systemd_activation.hpp"
|
||||
|
||||
#include <unistd.h>
|
||||
|
||||
#include <cstdlib>
|
||||
#include <string>
|
||||
|
||||
namespace velox::daemon::rpc {
|
||||
|
||||
namespace {
|
||||
constexpr int kListenFdsStart = 3; // SD_LISTEN_FDS_START
|
||||
} // namespace
|
||||
|
||||
int systemd_activated_fd() {
|
||||
const char* pid_env = std::getenv("LISTEN_PID");
|
||||
const char* fds_env = std::getenv("LISTEN_FDS");
|
||||
int fd = -1;
|
||||
|
||||
if (pid_env != nullptr && fds_env != nullptr) {
|
||||
try {
|
||||
if (std::stol(pid_env) == static_cast<long>(::getpid()) && std::stol(fds_env) == 1) {
|
||||
fd = kListenFdsStart;
|
||||
}
|
||||
} catch (...) {
|
||||
// Malformed env from something other than systemd; treat as not activated.
|
||||
}
|
||||
}
|
||||
|
||||
// Contract: consumed once, then cleared, so a value meant for veloxd is never
|
||||
// mistaken for one meant for a process it might itself exec later.
|
||||
::unsetenv("LISTEN_PID");
|
||||
::unsetenv("LISTEN_FDS");
|
||||
::unsetenv("LISTEN_FDNAMES");
|
||||
return fd;
|
||||
}
|
||||
|
||||
} // namespace velox::daemon::rpc
|
||||
@@ -0,0 +1,20 @@
|
||||
#pragma once
|
||||
|
||||
// Minimal sd_listen_fds(3) reimplementation — one function, no libsystemd dependency, for
|
||||
// the one fd velox.socket ever hands us. See velox.socket / velox.service in
|
||||
// packaging/nativehost's systemd unit pair: the socket unit binds
|
||||
// $XDG_RUNTIME_DIR/velox/velox.sock itself (before veloxd ever runs, so the very first
|
||||
// connection attempt after boot is queued by the kernel rather than refused) and execs
|
||||
// veloxd with that listening fd already open at fd 3, LISTEN_FDS=1, LISTEN_PID=<our pid>.
|
||||
|
||||
namespace velox::daemon::rpc {
|
||||
|
||||
// The systemd-activated listening socket fd, or -1 if this process was not socket-
|
||||
// activated (LISTEN_PID doesn't match our pid, or LISTEN_FDS is unset/not exactly 1 — more
|
||||
// than one would mean a unit file mismatch, since veloxd only ever asks for one socket).
|
||||
// Clears LISTEN_PID/LISTEN_FDS from the environment on the way out either way, per
|
||||
// sd_listen_fds's own contract, so a value meant for us is never mistaken for one meant for
|
||||
// a process veloxd might itself exec later.
|
||||
int systemd_activated_fd();
|
||||
|
||||
} // namespace velox::daemon::rpc
|
||||
@@ -12,7 +12,10 @@
|
||||
|
||||
#include <nlohmann/json.hpp>
|
||||
|
||||
#include <fcntl.h>
|
||||
|
||||
#include "rpc/event_loop.hpp"
|
||||
#include "rpc/systemd_activation.hpp"
|
||||
#include "version.hpp"
|
||||
|
||||
namespace velox::daemon::rpc {
|
||||
@@ -74,6 +77,20 @@ UdsServer::~UdsServer() {
|
||||
}
|
||||
|
||||
std::error_code UdsServer::start() {
|
||||
// velox.socket (systemd user unit, socket activation): the unit binds this path itself
|
||||
// before veloxd ever runs and hands the already-listening fd over at fd 3 — the first
|
||||
// connection after boot is queued by the kernel rather than refused, and there is no
|
||||
// window where a client sees ECONNREFUSED while the daemon is still starting. Skips
|
||||
// create/bind/chmod/listen entirely; the socket file's lifecycle (including removal on
|
||||
// stop) belongs to the unit, not to us, so bound_ stays false.
|
||||
if (const int activated = systemd_activated_fd(); activated >= 0) {
|
||||
::fcntl(activated, F_SETFL, O_NONBLOCK);
|
||||
::fcntl(activated, F_SETFD, FD_CLOEXEC);
|
||||
listen_fd_ = activated;
|
||||
loop_.add_fd(listen_fd_, kRead, [this](int, unsigned) { on_listener_readable(); });
|
||||
return {};
|
||||
}
|
||||
|
||||
if (path_.size() + 1 > sizeof(sockaddr_un::sun_path)) return errc(ENAMETOOLONG);
|
||||
|
||||
const int fd = ::socket(AF_UNIX, SOCK_STREAM | SOCK_NONBLOCK | SOCK_CLOEXEC, 0);
|
||||
|
||||
@@ -24,6 +24,7 @@ veloxd_test(sched_scheduler LIBS veloxd_sched veloxd_rpc)
|
||||
veloxd_test(event_hub LIBS veloxd_rpc)
|
||||
veloxd_test(store_categories_queues LIBS veloxd_store)
|
||||
veloxd_test(single_instance LIBS veloxd_rpc)
|
||||
veloxd_test(systemd_activation LIBS veloxd_rpc)
|
||||
veloxd_test(dispatcher_settings LIBS veloxd_rpc veloxd_store)
|
||||
veloxd_test(capture_offer LIBS veloxd_rpc veloxd_store)
|
||||
veloxd_test(dispatcher_misc LIBS veloxd_rpc veloxd_store)
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
// systemd_activated_fd(): the LISTEN_PID/LISTEN_FDS contract, without a real systemd.
|
||||
|
||||
#include <unistd.h>
|
||||
|
||||
#include <cstdlib>
|
||||
#include <string>
|
||||
|
||||
#include "check.hpp"
|
||||
#include "rpc/systemd_activation.hpp"
|
||||
|
||||
using namespace velox::daemon::rpc;
|
||||
|
||||
namespace {
|
||||
void set_env(const char* k, const std::string& v) { ::setenv(k, v.c_str(), 1); }
|
||||
} // namespace
|
||||
|
||||
void run() {
|
||||
// Not activated: neither var set.
|
||||
::unsetenv("LISTEN_PID");
|
||||
::unsetenv("LISTEN_FDS");
|
||||
CHECK_EQ(systemd_activated_fd(), -1);
|
||||
|
||||
// LISTEN_PID for a different process: not us, so not activated.
|
||||
set_env("LISTEN_PID", std::to_string(::getpid() + 1));
|
||||
set_env("LISTEN_FDS", "1");
|
||||
CHECK_EQ(systemd_activated_fd(), -1);
|
||||
// Consumed regardless of the outcome — a stale value from some other process's
|
||||
// exec chain must not leak into what veloxd checks next time.
|
||||
CHECK(::getenv("LISTEN_PID") == nullptr);
|
||||
CHECK(::getenv("LISTEN_FDS") == nullptr);
|
||||
|
||||
// Our own pid, LISTEN_FDS=1: activated, fd 3 (SD_LISTEN_FDS_START).
|
||||
set_env("LISTEN_PID", std::to_string(::getpid()));
|
||||
set_env("LISTEN_FDS", "1");
|
||||
CHECK_EQ(systemd_activated_fd(), 3);
|
||||
CHECK(::getenv("LISTEN_PID") == nullptr);
|
||||
|
||||
// Our own pid but LISTEN_FDS=2: a unit file mismatch (veloxd only ever asks for one
|
||||
// socket) — refuse rather than guess which of two fds is the right one.
|
||||
set_env("LISTEN_PID", std::to_string(::getpid()));
|
||||
set_env("LISTEN_FDS", "2");
|
||||
CHECK_EQ(systemd_activated_fd(), -1);
|
||||
|
||||
// Garbage LISTEN_FDS: not activated, not a crash.
|
||||
set_env("LISTEN_PID", std::to_string(::getpid()));
|
||||
set_env("LISTEN_FDS", "not-a-number");
|
||||
CHECK_EQ(systemd_activated_fd(), -1);
|
||||
|
||||
::unsetenv("LISTEN_PID");
|
||||
::unsetenv("LISTEN_FDS");
|
||||
}
|
||||
|
||||
TEST_MAIN()
|
||||
@@ -0,0 +1,85 @@
|
||||
# 08 — Porting guide (macOS first, Windows later)
|
||||
|
||||
Read `docs/adr/0020-cross-platform-strategy.md` first; this file is the concrete inventory
|
||||
it refers to. **Rule: Linux behaviour never changes. `#ifdef` never appears in logic.**
|
||||
|
||||
## Directory layout
|
||||
|
||||
Platform backends live beside the code they serve, one directory per OS:
|
||||
|
||||
```
|
||||
core/src/io/platform/linux/file_ops.cpp core/src/io/platform/macos/file_ops.cpp
|
||||
core/src/io/platform/file_ops.hpp # the seam — no #ifdef, no OS types
|
||||
daemon/src/rpc/platform/linux/{wakeup,peercred,instance_lock,runtime_dir}.cpp
|
||||
daemon/src/rpc/platform/macos/{...}.cpp
|
||||
daemon/src/rpc/platform/*.hpp # the seams
|
||||
cmake/platform.cmake # VELOX_OS_* + source selection
|
||||
```
|
||||
|
||||
`cmake/platform.cmake` sets exactly one of `VELOX_OS_LINUX`, `VELOX_OS_MACOS`,
|
||||
`VELOX_OS_WINDOWS`, and exposes `velox_platform_sources(<target> <dir>)` which adds
|
||||
`<dir>/platform/<os>/*.cpp`. Selection happens there and nowhere else.
|
||||
|
||||
## The seams
|
||||
|
||||
Five interfaces cover the whole port. Signatures are indicative, not binding — the owning
|
||||
lane settles them in Phase 0.
|
||||
|
||||
| Seam | Interface | Why |
|
||||
|---|---|---|
|
||||
| File preallocation | `Result<void> preallocate(int fd, uint64_t bytes)` | `posix_fallocate` is Linux/glibc |
|
||||
| Cache advice | `void advise_dontneed(int fd, uint64_t off, uint64_t len)` | `posix_fadvise` has no macOS equivalent |
|
||||
| Durable flush | `Result<void> flush_durable(int fd)` | `fdatasync` vs `F_FULLFSYNC` |
|
||||
| Loop wakeup | `class Wakeup { int pollfd(); void signal(); void drain(); }` | `eventfd` is Linux-only |
|
||||
| Peer identity | `Result<PeerId> peer_of(int fd)` | `SO_PEERCRED` vs `LOCAL_PEERCRED` |
|
||||
| Single instance | `Result<Lock> acquire(string_view runtime_dir)` | abstract sockets are Linux-only |
|
||||
| Runtime dir | `string runtime_dir()`, `string data_dir()` | XDG vs `~/Library` |
|
||||
|
||||
## API mapping
|
||||
|
||||
| Linux (today) | macOS | Windows (later) | Note |
|
||||
|---|---|---|---|
|
||||
| `posix_fallocate(fd,0,n)` | `fcntl(F_PREALLOCATE)` then `ftruncate(n)` | `SetFileValidData` / `SetEndOfFile` | macOS needs the `ftruncate`; `F_PREALLOCATE` alone does not set size. Fall back to `ftruncate` on failure, exactly as the Linux path already does for `EOPNOTSUPP` |
|
||||
| `posix_fadvise(DONTNEED)` | no equivalent — **no-op** | `FILE_FLAG_NO_BUFFERING` | Do **not** substitute `F_NOCACHE`: it changes caching for the whole descriptor, not a written range. A no-op is honest; record it |
|
||||
| `fdatasync(fd)` | `fcntl(fd, F_FULLFSYNC)`, fall back to `fsync` | `FlushFileBuffers` | `fsync` on macOS does **not** guarantee the drive flushed. `.veloxpart` resume integrity depends on this — use `F_FULLFSYNC` |
|
||||
| `pwrite` | same | `WriteFile` + `OVERLAPPED` | POSIX, no work |
|
||||
| `O_NOFOLLOW` | same | `FILE_FLAG_OPEN_REPARSE_POINT` | POSIX, no work |
|
||||
| `eventfd` | self-pipe (`pipe2`/`O_NONBLOCK|O_CLOEXEC`) | event object | `poll(2)` already used, so a pipe read-end drops straight in |
|
||||
| `timerfd` | `poll()` timeout computed from the next deadline | waitable timer | Simplest port: the loop already has a deadline set |
|
||||
| `SO_PEERCRED` + `struct ucred` | `getpeereid(fd,&uid,&gid)` | named-pipe token | Same-UID check is the security property; keep it |
|
||||
| abstract socket `\0velox-daemon-<hash>` | socket file in the runtime dir + `flock(LOCK_EX|LOCK_NB)` | named mutex | macOS has no abstract namespace. Must unlink stale sockets on start — the abstract version got that free |
|
||||
| `$XDG_RUNTIME_DIR` | `$TMPDIR` (per-user, already private) | `%LOCALAPPDATA%` | macOS has no XDG runtime dir |
|
||||
| `$XDG_DATA_HOME` | `~/Library/Application Support/Velox` | `%APPDATA%` | |
|
||||
| `libsecret` / Secret Service | Keychain (`Security.framework`) | DPAPI / Credential Manager | Behind the credential-store seam. CLAUDE.md §4 still applies: never SQLite, never logs |
|
||||
| `systemd` user units | `launchd` plist (`~/Library/LaunchAgents`) | Service/Task Scheduler | Phase 3 |
|
||||
|
||||
## Build dependencies
|
||||
|
||||
| Ubuntu | macOS (Homebrew) |
|
||||
|---|---|
|
||||
| `qt6-base-dev`, `qt6-svg-dev`, `qt6-tools-dev` | `qt@6` |
|
||||
| `libcurl4-openssl-dev` | system libcurl, or `curl` |
|
||||
| `libsqlite3-dev` | system sqlite, or `sqlite` |
|
||||
| `libssl-dev` | `openssl@3` (set `OPENSSL_ROOT_DIR`) |
|
||||
| `libsecret-1-dev` | **none** — Keychain is in the SDK |
|
||||
| `nlohmann-json3-dev` | `nlohmann-json` |
|
||||
| `nodejs`, `npm` | `node` |
|
||||
|
||||
The root `CMakeLists.txt` currently does `pkg_check_modules(LIBSECRET REQUIRED ...)`
|
||||
unconditionally. That must become Linux-only, or macOS cannot configure at all. It is the
|
||||
single hard blocker for a first macOS build.
|
||||
|
||||
## What does not change
|
||||
|
||||
`contracts/` and both generated clients, `tools/mockd`, `tests/conformance`,
|
||||
`tools/testserver`, the whole extension, and every hostile-mode expectation. If a port
|
||||
tempts you to change a fixture or a schema, stop — that is a contract change and it goes
|
||||
through PROTO.
|
||||
|
||||
## Verification gates
|
||||
|
||||
- **Phase 0 done:** Ubuntu suite still 57/57, and `git diff` shows only moves behind seams.
|
||||
- **Phase 1 done:** `veloxd`, `velox`, `libveloxcore` build on macOS; core unit tests pass.
|
||||
- **Phase 2 done:** conformance (mockd **and** live veloxd) and the engine hostile-mode
|
||||
matrix pass on macOS; one real download completes with a matching SHA-256.
|
||||
- **Phase 3 done:** `.dmg` or Homebrew formula installs and runs on a clean machine.
|
||||
@@ -0,0 +1,107 @@
|
||||
# ADR 0018 — Nullable optional fields: absent vs. explicit null
|
||||
|
||||
**Status:** accepted · **Date:** 2026-09-13 · **Lane:** PROTO
|
||||
**Prompted by:** a DAEMON report against `download.update`: the generated C++ parser gives
|
||||
`VeloxDispatcher` no way to tell "the caller left this field alone" from "the caller wants
|
||||
it cleared," so `download.update` and (the moment a nullable `SettingKey` exists)
|
||||
`settings.set` can set a nullable field but never clear it back to `null`.
|
||||
|
||||
## Context
|
||||
|
||||
`download.update`'s `patch` object documents the convention plainly: "Only the present
|
||||
fields change. An explicit null clears a nullable field." That is a deliberate, already-
|
||||
committed wire contract — not something up for redesign here. The gap is one layer down:
|
||||
`contracts/codegen/gen_cpp.py`'s `emit_field_parse` collapses "key absent" and "key present
|
||||
with value `null`" to the same `std::nullopt`, on purpose, and the comment says so:
|
||||
|
||||
> Absent and null mean the same thing: the field is not set. A client that omits a
|
||||
> nullable field and one that sends null are treated identically on purpose.
|
||||
|
||||
That collapse is *correct* for the common case — most nullable-optional fields are on
|
||||
create-style params (`DownloadSpec.saveDir`, `.categoryId`, …) where there is no existing
|
||||
value to distinguish "never set" from "explicitly cleared" in the first place; either way
|
||||
the daemon just uses a default. It is wrong specifically for **patch-style** params, where
|
||||
a field can already hold a value and the caller needs to say which of two different things
|
||||
they mean: "leave it" or "clear it."
|
||||
|
||||
The schema IR (`schema_ir.py`) already tracks `required` and `nullable` as two independent
|
||||
booleans per `Field`, so the information needed to make this distinction exists all the way
|
||||
through parsing — `emit_field_parse` just doesn't act on it. Only `download.update`'s
|
||||
`patch` object is affected today (`filename`, `saveDir`, `categoryId`, `queueId`,
|
||||
`description`, `segments`, `bufferBytes`, `checksum` — all eight of its fields are
|
||||
nullable-and-optional with exactly this "leave vs. clear" meaning). No `SettingKey` is
|
||||
nullable yet, so `settings.set` has no live instance of the bug, but the same shape
|
||||
(`values` patches an existing bag) means the first nullable settings key will hit the exact
|
||||
same gap.
|
||||
|
||||
## Decision
|
||||
|
||||
**A JSON-null-aware optional, opt in per field via a new `x-clearable: true` annotation —
|
||||
not a blanket rule and not a companion "clear list" field.**
|
||||
|
||||
- New per-field schema annotation, `x-clearable: true`, valid only on a field whose type
|
||||
already includes `null` (schema error otherwise — clearable implies nullable). Marks
|
||||
"this field distinguishes absent from explicit null"; every other nullable-optional field
|
||||
keeps today's collapse.
|
||||
- The generated C++ type for a `x-clearable` field becomes `std::optional<std::optional<T>>`:
|
||||
outer `nullopt` = absent (leave unchanged), outer engaged with an inner `nullopt` =
|
||||
explicit `null` (clear it), outer engaged with an inner value = set it. One field, three
|
||||
states, no parallel bitset to keep in sync and no second field to forget to check.
|
||||
- `emit_field_parse` for such a field stops folding `is_null()` into "absent": absent skips
|
||||
the assignment (outer stays `nullopt`); present-and-null assigns an engaged-but-empty
|
||||
inner optional; present-and-valued parses normally into the inner optional. Every other
|
||||
field's codegen (the `required`/`nullable`-but-not-`clearable` majority) is unchanged.
|
||||
- TypeScript needs no generator change: `field?: T | null` already round-trips this exactly
|
||||
the way JSON does — an omitted key serializes as absent, `null` serializes as `null`, and
|
||||
`"field" in obj` / `obj.field === null` already distinguish the three states natively.
|
||||
This gap is a C++-generator-only problem.
|
||||
- Applies now to `download.update`'s eight `patch` fields. `Settings` gets no annotation
|
||||
today (nothing nullable to mark); the day a nullable `SettingKey` is added, it gets
|
||||
`x-clearable: true` in the same PR, not left to rediscover this ADR.
|
||||
|
||||
## Versioning
|
||||
|
||||
Per ADR 0015: this retypes a generated C++ field (`optional<T>` -> `optional<optional<T>>`)
|
||||
with the wire byte-for-byte unchanged — a client sending the same JSON parses correctly
|
||||
either way. **Minor bump, with a migration note** for anyone reading `patch.filename` et al.
|
||||
directly (unwrap twice: check the outer, then the inner). Not major; `session.hello`'s
|
||||
major-only check must not refuse a wire-compatible peer over a binding-only change.
|
||||
|
||||
## Consequences
|
||||
|
||||
- `on_download_update` (DAEMON, not this lane) can finally implement "explicit null
|
||||
clears": read the outer optional for presence, the inner for clear-vs-value, exactly the
|
||||
three states the schema already promised.
|
||||
- The collapse comment in `emit_field_parse` stays as the default behavior and gets a
|
||||
pointer to this ADR for the opt-in exception, instead of being read as an oversight.
|
||||
- Implementation (schema annotation support in `schema_ir.py`, the `gen_cpp.py` emission
|
||||
change above, regenerating `core/generated/`, the `x-clearable: true` annotations on
|
||||
`download.update`'s eight fields, the VERSION bump and migration note) is **not** done in
|
||||
this change — recorded here so DAEMON isn't blocked on relitigating the design, tracked as
|
||||
its own PROTO PR per the normal contracts process (schema + regenerated code + fixtures +
|
||||
VERSION bump together, CLAUDE.md §2).
|
||||
|
||||
## Alternatives rejected
|
||||
|
||||
**An explicit clear list** (e.g. `patch.clearFields: ["categoryId", …]`, plain non-nullable
|
||||
`optional<T>` fields otherwise). Rejected: the wire contract "an explicit null clears a
|
||||
nullable field" is already written into `download.update`'s schema description and is what
|
||||
DAEMON built against — this would be a real, disruptive wire redesign to route around a
|
||||
generator gap, not a fix for it. It also doesn't compose: every patch-shaped object gains a
|
||||
second array to keep in sync with the first, by hand, forever.
|
||||
|
||||
**A parallel "which fields were present" bitset** (struct of `optional<T>` fields plus a
|
||||
sibling presence-flags struct or bitset). Rejected: two things to check per field instead
|
||||
of one, and nothing stops a caller from reading the optional and forgetting the presence
|
||||
bit — exactly the class of bug this ADR exists to close.
|
||||
|
||||
**Apply the tri-state to every `nullable && !required` field automatically**, using the IR
|
||||
flags already present, no annotation needed. Rejected: `emit_field_parse` only backs
|
||||
`parse<T>()`, used for *params* types the daemon receives — but the conformance C++ runner
|
||||
also instantiates `parse<T>()` for **result** types (round-tripping golden fixtures), and
|
||||
plenty of those are nullable-optional with no patch semantics at all (`TaskSummary.effectiveUrl`,
|
||||
"null until the first probe succeeds" — a plain nullable value, not a leave-or-clear
|
||||
choice). Blanket application would retype those too, forcing every read site across the
|
||||
daemon that already does `if (summary.effectiveUrl)` into an unwanted double-unwrap for a
|
||||
distinction that field doesn't have. Opt-in keeps the blast radius at exactly the fields
|
||||
that need it.
|
||||
@@ -0,0 +1,45 @@
|
||||
# ADR 0019 — The live-`veloxd` conformance runner is a required check, as-is
|
||||
|
||||
**Status:** accepted · **Date:** 2026-09-12 · **Lane:** PKG/QA
|
||||
|
||||
## Context
|
||||
|
||||
`tests/conformance/run.sh` step 3b (ADR 0014's `conformance` ctest, already required
|
||||
per `.github/BRANCH_PROTECTION.md`) replays every fixture against a real, isolated
|
||||
`veloxd` it builds and starts — not just mockd, which only proves the TS client and the
|
||||
fixtures agree with each other. This is the runner that can catch `veloxd` disagreeing
|
||||
with its own contract, and it is unconditional in `run.sh` (`set -euo pipefail`, no
|
||||
skip flag): it already runs, and already blocks, inside the `conformance` job.
|
||||
|
||||
What was open was whether to treat that as a settled, defended gate or as something
|
||||
still provisional while `daemon/docs/deferrals.md`'s D1-D4b stub handlers were excused
|
||||
via `veloxd-xfail.json`. PROTO's update: 57/57 fixtures pass on `main`, and the xfail
|
||||
allowlist is down to 18 entries from 34 as DAEMON lands the deferred handlers behind
|
||||
them.
|
||||
|
||||
## Decision
|
||||
|
||||
The live-`veloxd` conformance runner stays required — no change to CI is needed, since
|
||||
it already runs inside the already-required `conformance` job (ADR 0014). What this ADR
|
||||
records is the standing: PKG/QA is not carving out an exception, a `continue-on-error`,
|
||||
or a separate advisory job for it while the xfail list shrinks. A regression here fails
|
||||
the same required check a schema mismatch would.
|
||||
|
||||
Verified, not assumed: `tests/conformance/veloxd-xfail.json` has 18 entries as of this
|
||||
ADR (`python3 -c "import json; print(len(json.load(open('tests/conformance/veloxd-xfail.json'))))"`).
|
||||
Each remaining entry excuses one still-stubbed handler on `deferrals.md`'s D-list, not a
|
||||
real disagreement between `veloxd` and its contract — `tests/conformance/README.md`
|
||||
already draws that line (an entry for anything else is a `run.sh`-detected "xfail entry
|
||||
unexpectedly passed" or a straight failure, not a quiet pass).
|
||||
|
||||
## Consequences
|
||||
|
||||
- No `ci.yml` or `BRANCH_PROTECTION.md` change: `conformance` was already listed
|
||||
required, and this runner was already inside it.
|
||||
- The xfail list is a visible, shrinking number, not a static allowance — as DAEMON
|
||||
clears more of `deferrals.md`'s D-list, entries come out of
|
||||
`tests/conformance/veloxd-xfail.json`, and `replay.ts` fails loudly (per
|
||||
`applyXfail`'s "unexpectedly passed" check) if one is left in after its handler ships.
|
||||
- Nothing here changes who owns what: `veloxd-xfail.json` and `run.sh` stay PROTO's;
|
||||
PKG/QA's role is the branch-protection policy this ADR confirms, not the runner
|
||||
itself.
|
||||
@@ -0,0 +1,69 @@
|
||||
# ADR 0020 — Porting to macOS (and later Windows) without forking the tree
|
||||
|
||||
Status: **accepted** · Supersedes nothing · Applies to every lane
|
||||
|
||||
## Context
|
||||
|
||||
The project was built Ubuntu-first and says so in its own project description. A survey of
|
||||
`main` (2026-09-15, 57/57 green) found the Linux-specific surface is far smaller than the
|
||||
Ubuntu-first framing suggests — roughly ten files, all of them already isolated at the
|
||||
bottom of the stack:
|
||||
|
||||
| Area | Linux-only thing | Files |
|
||||
|---|---|---|
|
||||
| `core/io` | `posix_fallocate`, `posix_fadvise`, `fdatasync` | `sparse_file.cpp` |
|
||||
| `core/meta` | `fdatasync` | `veloxpart.cpp` |
|
||||
| `daemon/rpc` | `eventfd` | `event_loop.{hpp,cpp}`, `main.cpp` |
|
||||
| `daemon/rpc` | `timerfd` | `main.cpp`, `sched/scheduler.hpp` |
|
||||
| `daemon/rpc` | `SO_PEERCRED` / `struct ucred` | `uds_server.{hpp,cpp}` |
|
||||
| `daemon/rpc` | abstract-namespace socket lock | `single_instance.{hpp,cpp}` |
|
||||
| `daemon/rpc` | `$XDG_RUNTIME_DIR` layout | `runtime_dir.{hpp,cpp}` |
|
||||
| build | `libsecret-1` is `REQUIRED` at the root | `CMakeLists.txt` |
|
||||
|
||||
Everything else is already portable: the event loop is `poll(2)` (POSIX, not `epoll`), the
|
||||
transfer engine is libcurl, the GUI is Qt 6, `tools/mockd` and `tests/conformance` are
|
||||
Node, `tools/testserver` is stdlib Python, and the extension is a WebExtension.
|
||||
|
||||
## Decision
|
||||
|
||||
**One tree, one logic, per-OS backends behind seams.**
|
||||
|
||||
1. **No `#ifdef` in logic.** Platform differences live behind a narrow interface, with one
|
||||
implementation file per OS. A reader of `sparse_file.cpp` must not need to know which OS
|
||||
they are on. `#ifdef` is allowed only inside a `platform/<os>/` file.
|
||||
2. **Linux is the reference implementation.** A port may never change Linux behaviour. The
|
||||
gate is mechanical: the full suite stays green on Ubuntu, and the Linux backend keeps
|
||||
the same syscalls it uses today. If a port needs a semantic change, that is an ADR of
|
||||
its own, not a port commit.
|
||||
3. **Seams are introduced by the owning lane, on Linux, before any port work.** CORE
|
||||
introduces `core`'s seams; DAEMON introduces `daemon`'s. This is a pure refactor: move
|
||||
the existing Linux code behind the interface unchanged, prove the suite is still green.
|
||||
4. **A new lane, PORT, owns only the per-OS backends** — `**/platform/<os>/**` and
|
||||
`cmake/platform*.cmake`. It never writes logic. That keeps CLAUDE.md §1 intact: CORE
|
||||
still owns `core/`'s logic, DAEMON still owns `daemon/`'s, and macOS work can proceed in
|
||||
parallel without cross-lane writes.
|
||||
5. **`VELOX_OS_*` is set once, centrally**, in `cmake/platform.cmake`, and platform sources
|
||||
are selected there — not by globbing, and not per-target ad hoc.
|
||||
6. **Optional dependencies are gated, never removed.** `libsecret` stays `REQUIRED` on
|
||||
Linux and is replaced by Keychain on macOS behind the same credential-store seam.
|
||||
|
||||
## Phases
|
||||
|
||||
- **Phase 0 — on Ubuntu, by CORE + DAEMON.** Introduce the seams and move today's Linux
|
||||
code behind them. Zero behaviour change; 57/57 stays green. No macOS code exists yet.
|
||||
- **Phase 1 — on macOS, by PORT.** Implement the macOS backends until `veloxd`, `velox`
|
||||
and `libveloxcore` build and their unit tests pass.
|
||||
- **Phase 2 — parity.** `tools/testserver`, conformance and the engine's hostile-mode
|
||||
matrix pass on macOS. The GUI builds against Qt 6 for macOS.
|
||||
- **Phase 3 — packaging.** `.dmg`/Homebrew, `launchd` instead of `systemd`, and the macOS
|
||||
native-messaging manifest locations. Deliberately last.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Windows later is the same shape: a third backend directory, no new strategy. The seams
|
||||
chosen here are POSIX-flavoured but interface-level, so a Win32 backend is additive.
|
||||
- CI must build both, or macOS rots silently. Until a macOS runner exists, Phase 1 is
|
||||
verified by hand on the porting machine and the Linux gate stays authoritative.
|
||||
- The cost is one indirection at the bottom of the I/O and RPC stacks. Measured against
|
||||
the alternative — `#ifdef` drift, or a forked repo that diverges in a month — it is
|
||||
cheap.
|
||||
@@ -43,7 +43,11 @@ Read `contracts/`, `core/include/`, `docs/`. Never write in `core/`, `gui/`, or
|
||||
output. Build this early: it is how you test the daemon before the GUI exists.
|
||||
9. **`velox-nmhost`** — 4-byte-length-prefixed stdio ⇄ Unix socket pump. **Under 300 lines,
|
||||
zero business logic**, and it must exit cleanly when Firefox closes the pipe. Install
|
||||
manifests to all four locations listed in `docs/05` §4.
|
||||
manifests to the three real locations in `docs/05` §4 / `docs/adr/0003` — not four:
|
||||
spike S1 found `~/snap/firefox/common/.mozilla/native-messaging-hosts/` (the intuitive
|
||||
"inside the snap" path) is not actually read by snap Firefox, and corrected `docs/05` §4
|
||||
down from its original four-location list. `packaging/nativehost/README.md` has the
|
||||
current table.
|
||||
|
||||
## Definition of done (M1)
|
||||
- Passes the full conformance suite as a server, over **both** transports.
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
# Agent brief — PORT (per-OS backends)
|
||||
|
||||
**Starts after Phase 0: CORE and DAEMON have landed the seams on Ubuntu.**
|
||||
|
||||
## You own
|
||||
```
|
||||
core/src/**/platform/<os>/** daemon/src/**/platform/<os>/**
|
||||
cmake/platform*.cmake packaging/macos/** packaging/windows/**
|
||||
```
|
||||
You may read everything. **You never write logic** — not in `core/src/io/*.cpp`, not in
|
||||
`daemon/src/rpc/*.cpp`, not in `gui/`, never in `contracts/`.
|
||||
|
||||
## Read first
|
||||
`docs/adr/0020-cross-platform-strategy.md`, then `docs/08-porting.md` — it has the seam
|
||||
list, the API mapping and the dependency table. Then `CLAUDE.md`.
|
||||
|
||||
## The one rule that matters
|
||||
**Linux behaviour never changes.** If your port needs a seam that doesn't exist, or a seam
|
||||
whose shape is wrong, you do not widen it yourself — you file a request with the owning
|
||||
lane (CORE for `core/`, DAEMON for `daemon/`) exactly as every other lane does. A port that
|
||||
quietly edits shared logic is how one tree becomes two.
|
||||
|
||||
## Build order (macOS)
|
||||
1. **Configure at all.** The root `CMakeLists.txt` requires `libsecret-1` unconditionally;
|
||||
that must be Linux-gated before anything else compiles. File it with PKG/QA — root build
|
||||
files are theirs.
|
||||
2. `core/` — `preallocate`, `advise_dontneed`, `flush_durable`. Use `F_FULLFSYNC`, not
|
||||
`fsync`: `.veloxpart` resume correctness depends on a real flush.
|
||||
3. `daemon/` — `Wakeup` (self-pipe), `peer_of` (`getpeereid`), `instance_lock` (socket file
|
||||
+ `flock`, with stale-socket cleanup), `runtime_dir` (`$TMPDIR`, `~/Library/Application
|
||||
Support/Velox`).
|
||||
4. Unit tests green for both lanes.
|
||||
5. `tools/testserver` (stdlib Python) and conformance (Node) — expected to run unchanged.
|
||||
If they don't, that's a bug worth reporting, not patching around.
|
||||
6. GUI against Qt 6 for macOS.
|
||||
7. Packaging — `.dmg`/Homebrew, `launchd`, native-messaging manifest locations. Last.
|
||||
|
||||
## Definition of done (Phase 1–2)
|
||||
- macOS builds `libveloxcore`, `veloxd` and `velox` with `-Wall -Wextra -Werror`.
|
||||
- Core and daemon unit tests pass on macOS.
|
||||
- One real download completes on macOS against `tools/testserver` with a SHA-256 that
|
||||
matches the server's reference — the same gate the Linux vertical slice passed.
|
||||
- `kill -9` mid-download, restart, resume completes and the checksum still matches.
|
||||
- The Ubuntu suite is **still** 57/57 from the same commit.
|
||||
|
||||
## Do not
|
||||
- Do not add `#ifdef` outside a `platform/<os>/` file.
|
||||
- Do not change a fixture, a schema or a hostile-mode expectation to make macOS pass.
|
||||
- Do not substitute `F_NOCACHE` for `posix_fadvise(DONTNEED)` — see `docs/08` for why.
|
||||
- Do not start with packaging.
|
||||
@@ -19,7 +19,12 @@ import {
|
||||
} from './context-menus.js';
|
||||
import { MediaBridge, notifyTab } from './media-bridge.js';
|
||||
import { createTransport, transportStorage, type TransportStatus, type VeloxTransport } from './transport/index.js';
|
||||
import type { CaptureOfferParams, CaptureRules, DownloadSpec } from '../shared/protocol/index.js';
|
||||
import {
|
||||
SESSION_SUBSCRIBE_PARAMS_EVENTS_ITEM_VALUES,
|
||||
type CaptureOfferParams,
|
||||
type CaptureRules,
|
||||
type DownloadSpec,
|
||||
} from '../shared/protocol/index.js';
|
||||
|
||||
let transport: VeloxTransport | undefined;
|
||||
let rules: CaptureRules = DEFAULT_CAPTURE_RULES;
|
||||
@@ -75,10 +80,31 @@ async function refreshRules(): Promise<void> {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* "Nothing is delivered until this is called" (session.subscribe's own description) —
|
||||
* without it, event.task.progress et al. never reach this connection at all, no matter
|
||||
* how many listeners bridge.ts registers locally. Requests the whole set every time
|
||||
* because any popup/options document could open at any moment and none of them narrow
|
||||
* per-tab; a fresh connection (first connect, or after a drop) starts with nothing
|
||||
* subscribed until this runs again.
|
||||
*/
|
||||
async function subscribeToEvents(): Promise<void> {
|
||||
try {
|
||||
await mustTransport().call('session.subscribe', {
|
||||
events: [...SESSION_SUBSCRIBE_PARAMS_EVENTS_ITEM_VALUES],
|
||||
});
|
||||
} catch {
|
||||
// Best-effort; a reconnect (or the next event.settings.changed-driven refresh) retries.
|
||||
}
|
||||
}
|
||||
|
||||
function onTransportState(status: TransportStatus): void {
|
||||
const detail = status.fatal ?? (status.needsPairing ? 'needs pairing' : '');
|
||||
console.debug(`[velox] transport ${status.state}${detail ? ` — ${detail}` : ''}`);
|
||||
if (status.state === 'connected') void refreshRules();
|
||||
if (status.state === 'connected') {
|
||||
void refreshRules();
|
||||
void subscribeToEvents();
|
||||
}
|
||||
}
|
||||
|
||||
async function setOverride(override: 'auto' | 'ws' | 'uds'): Promise<void> {
|
||||
|
||||
@@ -0,0 +1,389 @@
|
||||
// Runs the transport, the capture hook, and the popup event path against a REAL veloxd
|
||||
// — not FakeDaemon. Everything else in this suite is faithful to the documented wire
|
||||
// protocol, but "faithful" isn't "real"; this is what actually proves it.
|
||||
//
|
||||
// Requires VELOXD_BIN (path to a built veloxd) in the environment. Skips itself with a
|
||||
// clear message otherwise, so `npm test` and CI (no daemon binary lying around) are
|
||||
// unaffected. Run it like:
|
||||
//
|
||||
// VELOXD_BIN=/path/to/build/dev/bin/veloxd npx vitest run tests/live
|
||||
//
|
||||
// Each veloxd instance gets its own scratch XDG_RUNTIME_DIR/XDG_DATA_HOME/
|
||||
// XDG_CONFIG_HOME/HOME (main.cpp's single-instance lock is keyed to the runtime dir, so
|
||||
// this can run alongside another developer's or CI's own veloxd on the same machine).
|
||||
// VELOX_PAIR_AUTO=1 stands in for the GUI's Allow-prompt approver during dev/test
|
||||
// (rpc/pairing.hpp's EnvAutoApprover) — pairing itself is exercised for real, only the
|
||||
// human click is stubbed.
|
||||
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process';
|
||||
import { mkdtempSync, mkdirSync, rmSync } from 'node:fs';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join, resolve } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { WebSocket as WsClient } from 'ws';
|
||||
|
||||
import { RpcError, TransportClosedError } from '../../src/background/transport/types.js';
|
||||
import { WebSocketTransport, type WebSocketCtor, type WebSocketTransportDeps } from '../../src/background/transport/websocket.js';
|
||||
import { CaptureHook } from '../../src/background/capture/index.js';
|
||||
import type { OnHeadersReceivedDetails } from '../../src/background/capture/index.js';
|
||||
import type { CaptureRules, DownloadSpec, TaskProgressEvent, TaskStateEvent } from '../../src/shared/protocol/index.js';
|
||||
|
||||
const VELOXD_BIN = process.env.VELOXD_BIN;
|
||||
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '../../..');
|
||||
const TESTSERVER_PY = join(REPO_ROOT, 'tools/testserver/testserver.py');
|
||||
|
||||
// A fixed moz-extension origin, used both as session.pair's extensionId and as the WS
|
||||
// upgrade's Origin header — real Firefox sets the latter itself; ws's client needs it
|
||||
// spelled out (docs/05 §4: the daemon refuses the upgrade without a moz-extension:// Origin).
|
||||
const EXTENSION_ID = '11111111-2222-3333-4444-555555555555';
|
||||
const ORIGIN = `moz-extension://${EXTENSION_ID}`;
|
||||
|
||||
class OriginWebSocket extends WsClient {
|
||||
constructor(url: string) {
|
||||
super(url, { origin: ORIGIN });
|
||||
}
|
||||
}
|
||||
const CTOR = OriginWebSocket as unknown as WebSocketCtor;
|
||||
|
||||
function sleep(ms: number): Promise<void> {
|
||||
return new Promise((r) => setTimeout(r, ms));
|
||||
}
|
||||
|
||||
async function waitFor(cond: () => Promise<boolean> | boolean, timeoutMs: number, what: string): Promise<void> {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
for (;;) {
|
||||
if (await cond()) return;
|
||||
if (Date.now() > deadline) throw new Error(`timed out waiting for ${what}`);
|
||||
await sleep(50);
|
||||
}
|
||||
}
|
||||
|
||||
interface VeloxdInstance {
|
||||
proc: ChildProcessWithoutNullStreams;
|
||||
scratch: string;
|
||||
wsPort: number;
|
||||
/** True once the process has actually exited, by signal or otherwise. Node only sets
|
||||
* `proc.exitCode` for a normal exit — a signal-killed process reports its death via
|
||||
* `signalCode` and an `exit` event instead, never a non-null `exitCode`. */
|
||||
hasExited(): boolean;
|
||||
kill(signal?: NodeJS.Signals): void;
|
||||
}
|
||||
|
||||
async function startVeloxd(bin: string): Promise<VeloxdInstance> {
|
||||
const scratch = mkdtempSync(join(tmpdir(), 'velox-live-'));
|
||||
const runtime = join(scratch, 'rt');
|
||||
const data = join(scratch, 'data');
|
||||
const config = join(scratch, 'cfg');
|
||||
const home = join(scratch, 'home');
|
||||
mkdirSync(runtime, { mode: 0o700 });
|
||||
mkdirSync(data, { recursive: true });
|
||||
mkdirSync(config, { recursive: true });
|
||||
mkdirSync(join(home, 'Downloads'), { recursive: true });
|
||||
|
||||
const proc = spawn(bin, [], {
|
||||
env: {
|
||||
...process.env,
|
||||
VELOX_PAIR_AUTO: '1',
|
||||
XDG_RUNTIME_DIR: runtime,
|
||||
XDG_DATA_HOME: data,
|
||||
XDG_CONFIG_HOME: config,
|
||||
HOME: home,
|
||||
},
|
||||
});
|
||||
let exited = false;
|
||||
proc.on('exit', () => {
|
||||
exited = true;
|
||||
});
|
||||
let stderr = '';
|
||||
proc.stderr.on('data', (d) => {
|
||||
stderr += String(d);
|
||||
});
|
||||
|
||||
const portFile = join(runtime, 'velox', 'ws.port');
|
||||
try {
|
||||
await waitFor(async () => {
|
||||
if (exited) throw new Error(`veloxd exited early (code ${proc.exitCode}, signal ${proc.signalCode}): ${stderr}`);
|
||||
try {
|
||||
await readFile(portFile);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}, 10_000, 'veloxd to write ws.port');
|
||||
} catch (e) {
|
||||
proc.kill('SIGKILL');
|
||||
rmSync(scratch, { recursive: true, force: true });
|
||||
throw e;
|
||||
}
|
||||
const wsPort = Number((await readFile(portFile, 'utf8')).trim());
|
||||
|
||||
return {
|
||||
proc,
|
||||
scratch,
|
||||
wsPort,
|
||||
hasExited: () => exited,
|
||||
kill(signal: NodeJS.Signals = 'SIGTERM') {
|
||||
proc.kill(signal);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
interface TestServerInstance {
|
||||
proc: ChildProcessWithoutNullStreams;
|
||||
baseUrl: string;
|
||||
}
|
||||
|
||||
async function startTestServer(): Promise<TestServerInstance> {
|
||||
const proc = spawn('python3', [TESTSERVER_PY, '--port', '0'], {});
|
||||
let stdout = '';
|
||||
let port: number | null = null;
|
||||
proc.stdout.on('data', (d) => {
|
||||
stdout += String(d);
|
||||
const m = /^(\d+)\s*$/m.exec(stdout);
|
||||
if (m) port = Number(m[1]);
|
||||
});
|
||||
await waitFor(() => port !== null, 5_000, 'testserver to print its port');
|
||||
const baseUrl = `http://127.0.0.1:${port}`;
|
||||
await waitFor(async () => {
|
||||
try {
|
||||
const res = await fetch(`${baseUrl}/__health`);
|
||||
return res.ok;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}, 5_000, 'testserver /__health');
|
||||
return { proc, baseUrl };
|
||||
}
|
||||
|
||||
function memDeps(init: { token?: string | null } = {}): { deps: WebSocketTransportDeps; store: { token: string | null } } {
|
||||
const store = { token: init.token ?? null };
|
||||
return {
|
||||
store,
|
||||
deps: {
|
||||
getToken: async () => store.token,
|
||||
setToken: async (t) => {
|
||||
store.token = t;
|
||||
},
|
||||
getCachedPort: async () => null,
|
||||
setCachedPort: async () => undefined,
|
||||
extensionId: EXTENSION_ID,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function makeTransport(port: number, deps: WebSocketTransportDeps, extra: Partial<WebSocketTransportDeps> = {}): WebSocketTransport {
|
||||
return new WebSocketTransport({
|
||||
...deps,
|
||||
...extra,
|
||||
webSocketCtor: CTOR,
|
||||
portRange: { start: port, end: port },
|
||||
openTimeoutMs: 2000,
|
||||
});
|
||||
}
|
||||
|
||||
const maybeDescribe = VELOXD_BIN ? describe : describe.skip;
|
||||
|
||||
if (!VELOXD_BIN) {
|
||||
console.warn('tests/live/real-veloxd.test.ts: VELOXD_BIN not set — skipping (see file header).');
|
||||
}
|
||||
|
||||
maybeDescribe('WebSocketTransport against a real veloxd', () => {
|
||||
let daemon: VeloxdInstance;
|
||||
let testserver: TestServerInstance;
|
||||
// The mid-test fail-open case kills `daemon` and a later test starts a replacement —
|
||||
// every scratch dir that ever existed gets cleaned up here, not just the last one.
|
||||
const allScratchDirs: string[] = [];
|
||||
|
||||
async function freshVeloxd(): Promise<VeloxdInstance> {
|
||||
const d = await startVeloxd(VELOXD_BIN!);
|
||||
allScratchDirs.push(d.scratch);
|
||||
return d;
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
daemon = await freshVeloxd();
|
||||
testserver = await startTestServer();
|
||||
}, 20_000);
|
||||
|
||||
afterAll(() => {
|
||||
daemon?.kill('SIGKILL');
|
||||
testserver?.proc.kill('SIGKILL');
|
||||
for (const dir of allScratchDirs) rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('session.hello without a token surfaces NotPaired / needsPairing', async () => {
|
||||
const { deps } = memDeps();
|
||||
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
|
||||
await expect(t.connect()).rejects.toBeInstanceOf(RpcError);
|
||||
expect(t.status.needsPairing).toBe(true);
|
||||
t.disconnect();
|
||||
});
|
||||
|
||||
let pairedToken: string;
|
||||
|
||||
it('pairs (VELOX_PAIR_AUTO=1 stands in for the human Allow click) and hellos with the issued token', async () => {
|
||||
const { deps, store } = memDeps();
|
||||
const t = makeTransport(daemon.wsPort, deps); // autoPair: true (default)
|
||||
await t.connect();
|
||||
expect(t.state).toBe('connected');
|
||||
expect(t.status.daemonVersion).toBeTruthy();
|
||||
expect(store.token).toBeTruthy();
|
||||
pairedToken = store.token!;
|
||||
t.disconnect();
|
||||
});
|
||||
|
||||
it('the pairing token survives a reconnect: a fresh transport reuses it with no fresh pairing', async () => {
|
||||
const { deps } = memDeps({ token: pairedToken });
|
||||
// autoPair: false — if this succeeds at all, it can only be because the stored
|
||||
// token from the previous test was accepted outright, not because this transport
|
||||
// silently re-paired.
|
||||
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
|
||||
await t.connect();
|
||||
expect(t.state).toBe('connected');
|
||||
t.disconnect();
|
||||
});
|
||||
|
||||
it('a wrong token is rejected, and repeating it rate-limits the next pairing attempt', async () => {
|
||||
// Five failed session.hello attempts from this origin (ws_server.cpp records a
|
||||
// rate-limiter failure on every not-paired hello, not only on a failed session.pair)
|
||||
// exhausts the window; the sixth thing this origin tries — a pairing attempt — gets
|
||||
// RateLimited rather than a fresh token.
|
||||
for (let i = 0; i < 5; i += 1) {
|
||||
const { deps } = memDeps({ token: 'not-the-real-token' });
|
||||
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
|
||||
await expect(t.connect()).rejects.toBeInstanceOf(RpcError);
|
||||
t.disconnect();
|
||||
}
|
||||
|
||||
const { deps } = memDeps(); // no token -> autoPair kicks in -> session.pair
|
||||
const t = makeTransport(daemon.wsPort, deps);
|
||||
await expect(t.connect()).rejects.toBeInstanceOf(RpcError);
|
||||
expect(t.status.needsPairing).toBe(true);
|
||||
expect(t.status.retryAfterSec).toBeGreaterThan(0);
|
||||
t.disconnect();
|
||||
});
|
||||
|
||||
it('download.add creates a real task the engine picks up', async () => {
|
||||
const { deps } = memDeps({ token: pairedToken });
|
||||
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
|
||||
await t.connect();
|
||||
try {
|
||||
const spec: DownloadSpec = { url: `${testserver.baseUrl}/plain/file/64K`, filename: 'plain-download.bin' };
|
||||
const added = await t.call('download.add', spec);
|
||||
expect(added.taskId).toBeTruthy();
|
||||
|
||||
await waitFor(async () => {
|
||||
const detail = await t.call('download.get', { taskId: added.taskId });
|
||||
const state = detail.summary.state;
|
||||
return state === 'complete' || state === 'downloading' || state === 'verifying';
|
||||
}, 10_000, 'the task to leave the queued state');
|
||||
} finally {
|
||||
t.disconnect();
|
||||
}
|
||||
}, 15_000);
|
||||
|
||||
it('capture.offer end to end: the real capture path takes a monitored download, ignores its own duplicate, and fails open when the daemon dies mid-offer', async () => {
|
||||
const { deps } = memDeps({ token: pairedToken });
|
||||
const t = makeTransport(daemon.wsPort, deps, { autoPair: false });
|
||||
await t.connect();
|
||||
|
||||
const rules: CaptureRules = await t.call('capture.getRules', {});
|
||||
expect(rules.monitoredExtensions).toContain('zip'); // seeded default (0001_initial.sql)
|
||||
|
||||
const hook = new CaptureHook({
|
||||
offer: (params, opts) => t.call('capture.offer', params, opts),
|
||||
stash: { take: () => undefined, peek: () => undefined },
|
||||
getCookies: async () => [],
|
||||
getRules: () => rules,
|
||||
origin: ORIGIN,
|
||||
});
|
||||
|
||||
// A throttled URL so the task the first offer creates is still active (not yet
|
||||
// complete) when the dedupe offer for the same URL follows immediately after.
|
||||
const url = `${testserver.baseUrl}/throttled/file/512K`;
|
||||
const details: OnHeadersReceivedDetails = {
|
||||
requestId: 'live-1',
|
||||
url,
|
||||
method: 'GET',
|
||||
type: 'other',
|
||||
statusCode: 200,
|
||||
tabId: 1,
|
||||
responseHeaders: [{ name: 'content-disposition', value: 'attachment; filename="live-capture.zip"' }],
|
||||
};
|
||||
|
||||
const first = await hook.handle(details);
|
||||
expect(first).toEqual({ cancel: true }); // the daemon took it — Firefox never starts its own download
|
||||
|
||||
const list = await t.call('download.list', { filter: { query: 'live-capture' } });
|
||||
expect(list.items.length).toBeGreaterThan(0);
|
||||
const task = list.items[0]!;
|
||||
expect(task.categoryId).toBe('programs'); // "zip" routes to the built-in Programs category
|
||||
expect(task.saveDir).toContain('Downloads/Programs');
|
||||
|
||||
// Same URL again, task still active: the daemon's own dedupe (has_active_duplicate)
|
||||
// says Ignore, so the hook proceeds instead of cancelling a second time.
|
||||
const dup = await hook.handle({ ...details, requestId: 'live-2' });
|
||||
expect(dup).toEqual({});
|
||||
|
||||
// Now kill the daemon mid-offer and prove fail-open holds against the REAL binary,
|
||||
// not just FakeDaemon: the hook must still resolve to {} (Firefox downloads
|
||||
// normally) well inside its own 750 ms budget.
|
||||
daemon.kill('SIGKILL');
|
||||
await waitFor(() => daemon.hasExited(), 5_000, 'veloxd to actually die');
|
||||
|
||||
const started = Date.now();
|
||||
const afterDeath = await hook.handle({ ...details, requestId: 'live-3', url: `${url}?after-death=1` });
|
||||
const elapsedMs = Date.now() - started;
|
||||
expect(afterDeath).toEqual({}); // fail open — never {cancel: true} with a dead daemon
|
||||
expect(elapsedMs).toBeLessThan(900); // budget is 750ms; the hook's own timer bounds this
|
||||
|
||||
t.disconnect();
|
||||
}, 20_000);
|
||||
|
||||
it('event.task.progress reaches a subscribed client (the popup\'s own path)', async () => {
|
||||
if (daemon.hasExited()) {
|
||||
// The previous test kills the daemon on purpose to prove fail-open; start a fresh
|
||||
// one so this test still exercises the real event path end to end.
|
||||
daemon = await freshVeloxd();
|
||||
}
|
||||
const { deps } = memDeps();
|
||||
const t = makeTransport(daemon.wsPort, deps); // fresh daemon instance -> fresh pairing
|
||||
await t.connect();
|
||||
try {
|
||||
await t.call('session.subscribe', {
|
||||
events: ['event.task.added', 'event.task.state', 'event.task.progress'],
|
||||
});
|
||||
|
||||
const progressEvents: TaskProgressEvent[] = [];
|
||||
const stateEvents: TaskStateEvent[] = [];
|
||||
t.on('event.task.progress', (p) => progressEvents.push(p as TaskProgressEvent));
|
||||
t.on('event.task.state', (p) => stateEvents.push(p as TaskStateEvent));
|
||||
|
||||
const spec: DownloadSpec = { url: `${testserver.baseUrl}/throttled/file/1M`, filename: 'progress-check.bin' };
|
||||
const added = await t.call('download.add', spec);
|
||||
|
||||
// /throttled defaults to 1 MiB/s, so a 1 MiB file takes ~1s — long enough that at
|
||||
// least one 4 Hz progress tick (event.task.progress's documented cap) lands before
|
||||
// it completes, exactly the path popup/store.ts consumes in the real extension.
|
||||
await waitFor(
|
||||
() => progressEvents.some((e) => e.tasks.some((row) => row.taskId === added.taskId)),
|
||||
8_000,
|
||||
'a live event.task.progress tick for our task',
|
||||
);
|
||||
expect(stateEvents.some((e) => e.taskId === added.taskId)).toBe(true);
|
||||
} finally {
|
||||
t.disconnect();
|
||||
}
|
||||
}, 15_000);
|
||||
|
||||
it('fail-open also holds through the transport itself: a call against a dead socket rejects, never hangs past its deadline', async () => {
|
||||
const { deps } = memDeps();
|
||||
const t = makeTransport(daemon.wsPort, deps);
|
||||
await t.connect();
|
||||
t.disconnect(); // closes the socket without telling the daemon anything is wrong
|
||||
await expect(t.call('capture.offer', { url: 'https://example.com/x.zip', method: 'GET', tabUrl: '' }, { timeoutMs: 200 })).rejects.toBeInstanceOf(
|
||||
TransportClosedError,
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,14 @@
|
||||
# nmhost/ — velox-nmhost, the Firefox native-messaging host. Owned by lane DAEMON.
|
||||
#
|
||||
# Deliberately dependency-free: no veloxd_* library, no nlohmann_json, no SQLite. It is a
|
||||
# byte-level pump between two framings (see src/main.cpp's own header comment) and runs
|
||||
# unconfined outside Firefox's sandbox (ADR 0003) — the less it links, the less there is to
|
||||
# go wrong running from wherever a snap/deb/flatpak install puts it.
|
||||
|
||||
add_executable(velox-nmhost src/main.cpp)
|
||||
target_compile_features(velox-nmhost PRIVATE cxx_std_23)
|
||||
target_compile_options(velox-nmhost PRIVATE -Wall -Wextra -Wpedantic -Werror)
|
||||
|
||||
if(VELOX_BUILD_TESTS AND EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/tests/CMakeLists.txt)
|
||||
add_subdirectory(tests)
|
||||
endif()
|
||||
@@ -0,0 +1,203 @@
|
||||
// velox-nmhost — Firefox native-messaging host. A dumb pump between two framings, nothing
|
||||
// else: stdin/stdout speak Firefox's own protocol (a 4-byte native-byte-order length
|
||||
// prefix, then that many bytes of UTF-8 JSON); $XDG_RUNTIME_DIR/velox/velox.sock speaks
|
||||
// veloxd's own NDJSON (one '\n'-terminated JSON value per line, daemon/src/rpc/ndjson.hpp).
|
||||
// Reframing between the two is the entire job.
|
||||
//
|
||||
// Runs unconfined outside Firefox's snap sandbox with the real $HOME and
|
||||
// $XDG_RUNTIME_DIR (ADR 0003 §Q2) — see packaging/nativehost/ for the manifest this is
|
||||
// installed as, and which native-messaging-hosts directory actually gets read by which
|
||||
// Firefox flavour.
|
||||
//
|
||||
// No business logic: no JSON parsing (frames are pure byte spans; only the length prefix
|
||||
// and the line boundary matter here), no retry/backoff (the extension re-launches a fresh
|
||||
// host on its own reconnect), no protocol version check (veloxd and the extension settle
|
||||
// that between themselves once the pump hands their bytes through). Exits the moment
|
||||
// either side closes.
|
||||
|
||||
#include <fcntl.h>
|
||||
#include <sys/socket.h>
|
||||
#include <sys/un.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <cerrno>
|
||||
#include <cstdint>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#include <poll.h>
|
||||
#include <string>
|
||||
|
||||
namespace {
|
||||
|
||||
// Firefox's own cap (host -> browser) is 1 MiB; this is just a sanity backstop against a
|
||||
// runaway peer so a malformed stream can't grow a buffer without bound.
|
||||
constexpr std::size_t kMaxFrameBytes = 8 * 1024 * 1024;
|
||||
|
||||
std::string socket_path() {
|
||||
const char* xdg = std::getenv("XDG_RUNTIME_DIR");
|
||||
std::string base = (xdg != nullptr && xdg[0] != '\0') ? xdg
|
||||
: ("/run/user/" + std::to_string(::getuid()));
|
||||
if (!base.empty() && base.back() == '/') base.pop_back();
|
||||
return base + "/velox/velox.sock";
|
||||
}
|
||||
|
||||
int connect_socket(const std::string& path) {
|
||||
const int fd = ::socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
|
||||
if (fd < 0) return -1;
|
||||
sockaddr_un addr{};
|
||||
addr.sun_family = AF_UNIX;
|
||||
if (path.size() + 1 > sizeof(addr.sun_path)) {
|
||||
::close(fd);
|
||||
return -1;
|
||||
}
|
||||
std::memcpy(addr.sun_path, path.c_str(), path.size());
|
||||
if (::connect(fd, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) != 0) {
|
||||
::close(fd);
|
||||
return -1;
|
||||
}
|
||||
return fd;
|
||||
}
|
||||
|
||||
// Reads everything currently available into `buf`. true on EOF, false otherwise (short
|
||||
// reads / EAGAIN just return with whatever was appended).
|
||||
bool read_available(int fd, std::string& buf) {
|
||||
char chunk[65536];
|
||||
for (;;) {
|
||||
const ssize_t n = ::read(fd, chunk, sizeof(chunk));
|
||||
if (n > 0) {
|
||||
buf.append(chunk, static_cast<std::size_t>(n));
|
||||
continue;
|
||||
}
|
||||
if (n == 0) return true; // EOF
|
||||
if (errno == EAGAIN || errno == EWOULDBLOCK) return false;
|
||||
if (errno == EINTR) continue;
|
||||
return true; // treat any other error as if the peer hung up
|
||||
}
|
||||
}
|
||||
|
||||
// Writes as much of `buf` as the fd accepts right now, trimming what was sent. Returns
|
||||
// false on a hard error (peer gone); EAGAIN is not an error, just "try again later".
|
||||
bool flush_some(int fd, std::string& buf) {
|
||||
while (!buf.empty()) {
|
||||
const ssize_t n = ::write(fd, buf.data(), buf.size());
|
||||
if (n > 0) {
|
||||
buf.erase(0, static_cast<std::size_t>(n));
|
||||
continue;
|
||||
}
|
||||
if (n < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) return true;
|
||||
if (n < 0 && errno == EINTR) continue;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// stdin frames (4-byte length + payload) -> NDJSON lines appended to `to_socket`.
|
||||
// Malformed (oversized) length is a hard stop.
|
||||
bool drain_stdin_frames(std::string& in, std::string& to_socket) {
|
||||
for (;;) {
|
||||
if (in.size() < 4) return true;
|
||||
std::uint32_t len;
|
||||
std::memcpy(&len, in.data(), 4);
|
||||
if (len > kMaxFrameBytes) return false;
|
||||
if (in.size() < 4 + len) return true;
|
||||
to_socket.append(in, 4, len);
|
||||
to_socket.push_back('\n');
|
||||
in.erase(0, 4 + len);
|
||||
}
|
||||
}
|
||||
|
||||
// NDJSON lines from the socket -> stdout frames (4-byte length + payload) appended to
|
||||
// `to_stdout`.
|
||||
bool drain_socket_lines(std::string& in, std::string& to_stdout) {
|
||||
for (;;) {
|
||||
const auto nl = in.find('\n');
|
||||
if (nl == std::string::npos) {
|
||||
if (in.size() > kMaxFrameBytes) return false;
|
||||
return true;
|
||||
}
|
||||
std::string_view line(in.data(), nl);
|
||||
if (!line.empty() && line.back() == '\r') line.remove_suffix(1);
|
||||
if (line.size() > kMaxFrameBytes) return false;
|
||||
const auto len = static_cast<std::uint32_t>(line.size());
|
||||
to_stdout.append(reinterpret_cast<const char*>(&len), 4);
|
||||
to_stdout.append(line);
|
||||
in.erase(0, nl + 1);
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
int main() {
|
||||
const int sock = connect_socket(socket_path());
|
||||
if (sock < 0) return 1; // daemon not running / socket missing: nothing to pump
|
||||
|
||||
// Every fd this pumps must be non-blocking: read_available()'s own drain loop keeps
|
||||
// calling read() until it actually sees EAGAIN, and a blocking fd never returns that —
|
||||
// it just blocks inside the "drain what's available" loop instead of going back to
|
||||
// poll(), which stalls the whole pump the moment one side has more to send than fits
|
||||
// in a single read().
|
||||
::fcntl(0, F_SETFL, ::fcntl(0, F_GETFL) | O_NONBLOCK);
|
||||
::fcntl(1, F_SETFL, ::fcntl(1, F_GETFL) | O_NONBLOCK);
|
||||
::fcntl(sock, F_SETFL, ::fcntl(sock, F_GETFL) | O_NONBLOCK);
|
||||
|
||||
std::string stdin_buf, stdout_buf, socket_in_buf, socket_out_buf;
|
||||
bool stdin_eof = false;
|
||||
|
||||
// Three distinct descriptors, not two: stdin (0) and stdout (1) are separate pipes
|
||||
// (read-only and write-only respectively — never the same fd, even though they sit
|
||||
// next to each other in a shell's mental model of "the process's stdio"), plus the
|
||||
// bidirectional socket.
|
||||
enum { kStdin, kStdout, kSock };
|
||||
for (;;) {
|
||||
pollfd fds[3] = {
|
||||
{0, 0, 0},
|
||||
{1, 0, 0},
|
||||
{sock, 0, 0},
|
||||
};
|
||||
if (!stdin_eof) fds[kStdin].events |= POLLIN;
|
||||
if (!stdout_buf.empty()) fds[kStdout].events |= POLLOUT;
|
||||
fds[kSock].events |= POLLIN;
|
||||
if (!socket_out_buf.empty()) fds[kSock].events |= POLLOUT;
|
||||
|
||||
// Nothing left to wait for: both directions exhausted.
|
||||
if (fds[kStdin].events == 0 && fds[kStdout].events == 0 && fds[kSock].events == 0) break;
|
||||
|
||||
const int n = ::poll(fds, 3, -1);
|
||||
if (n < 0) {
|
||||
if (errno == EINTR) continue;
|
||||
break;
|
||||
}
|
||||
|
||||
if (fds[kStdout].revents & POLLOUT) {
|
||||
if (!flush_some(1, stdout_buf)) break; // Firefox closed our stdout
|
||||
}
|
||||
if (fds[kSock].revents & POLLOUT) {
|
||||
if (!flush_some(sock, socket_out_buf)) break;
|
||||
}
|
||||
if (fds[kStdin].revents & (POLLIN | POLLHUP)) {
|
||||
if (read_available(0, stdin_buf)) stdin_eof = true;
|
||||
if (!drain_stdin_frames(stdin_buf, socket_out_buf)) break;
|
||||
}
|
||||
if (fds[kSock].revents & (POLLIN | POLLHUP)) {
|
||||
const bool socket_eof = read_available(sock, socket_in_buf);
|
||||
if (!drain_socket_lines(socket_in_buf, stdout_buf)) break;
|
||||
if (socket_eof) {
|
||||
// The daemon is gone. Flush whatever we already turned into stdout
|
||||
// frames, then stop — there is nothing left to relay either direction.
|
||||
(void)flush_some(1, stdout_buf);
|
||||
break;
|
||||
}
|
||||
}
|
||||
if ((fds[kStdin].revents | fds[kStdout].revents | fds[kSock].revents) &
|
||||
(POLLERR | POLLNVAL))
|
||||
break;
|
||||
|
||||
// Firefox closed the pipe: nothing more will ever arrive on stdin, and once our
|
||||
// own outbound backlog drains there is nothing left to send it either. Stop
|
||||
// rather than idle forever relaying replies nobody reads.
|
||||
if (stdin_eof && socket_out_buf.empty()) break;
|
||||
}
|
||||
|
||||
::close(sock);
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
# Integration test only — velox-nmhost has no internal functions worth unit-testing in
|
||||
# isolation (it's ~15 lines of byte-shuffling helpers around one poll() loop); what matters
|
||||
# is the real binary's observable behaviour over real pipes and a real socket.
|
||||
|
||||
add_executable(velox_nmhost_pump_test pump_test.cpp)
|
||||
target_compile_features(velox_nmhost_pump_test PRIVATE cxx_std_23)
|
||||
target_compile_options(velox_nmhost_pump_test PRIVATE -Wall -Wextra -Wpedantic -Werror)
|
||||
target_compile_definitions(velox_nmhost_pump_test PRIVATE
|
||||
VELOX_NMHOST_BIN="$<TARGET_FILE:velox-nmhost>")
|
||||
add_dependencies(velox_nmhost_pump_test velox-nmhost)
|
||||
|
||||
add_test(NAME nmhost.pump COMMAND velox_nmhost_pump_test)
|
||||
set_tests_properties(nmhost.pump PROPERTIES TIMEOUT 30)
|
||||
@@ -0,0 +1,54 @@
|
||||
#pragma once
|
||||
|
||||
// Minimal test harness: CHECK accumulates failures, TEST_MAIN reports and sets the exit
|
||||
// code. Copied from daemon/tests/check.hpp rather than shared across a build-dependency —
|
||||
// nmhost is deliberately dependency-free, tests included.
|
||||
|
||||
#include <cstdio>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace veloxd_test {
|
||||
|
||||
inline std::vector<std::string>& failures() {
|
||||
static std::vector<std::string> f;
|
||||
return f;
|
||||
}
|
||||
inline int& checks() {
|
||||
static int n = 0;
|
||||
return n;
|
||||
}
|
||||
|
||||
} // namespace veloxd_test
|
||||
|
||||
#define CHECK(cond) \
|
||||
do { \
|
||||
++::veloxd_test::checks(); \
|
||||
if (!(cond)) { \
|
||||
::veloxd_test::failures().push_back(std::string(__FILE__) + ":" + \
|
||||
std::to_string(__LINE__) + ": " + #cond); \
|
||||
} \
|
||||
} while (0)
|
||||
|
||||
#define CHECK_EQ(a, b) \
|
||||
do { \
|
||||
++::veloxd_test::checks(); \
|
||||
auto _va = (a); \
|
||||
auto _vb = (b); \
|
||||
if (!(_va == _vb)) { \
|
||||
::veloxd_test::failures().push_back(std::string(__FILE__) + ":" + \
|
||||
std::to_string(__LINE__) + ": " + #a + \
|
||||
" == " + #b); \
|
||||
} \
|
||||
} while (0)
|
||||
|
||||
#define TEST_MAIN() \
|
||||
int main() { \
|
||||
run(); \
|
||||
for (const auto& f : ::veloxd_test::failures()) std::printf("FAIL %s\n", f.c_str()); \
|
||||
std::printf("%d/%d checks passed\n", \
|
||||
::veloxd_test::checks() - \
|
||||
static_cast<int>(::veloxd_test::failures().size()), \
|
||||
::veloxd_test::checks()); \
|
||||
return ::veloxd_test::failures().empty() ? 0 : 1; \
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
// Integration test for velox-nmhost: spawns the real binary, feeds it a framed stdin
|
||||
// message, answers over a fake Unix socket standing in for veloxd, and checks what comes
|
||||
// back out on stdout — plus that it exits promptly once stdin closes.
|
||||
|
||||
#include <sys/socket.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/un.h>
|
||||
#include <sys/wait.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <cstdint>
|
||||
#include <cstdlib>
|
||||
#include <cstring>
|
||||
#include <ctime>
|
||||
#include <string>
|
||||
|
||||
#include "check.hpp"
|
||||
|
||||
namespace {
|
||||
|
||||
std::string make_temp_dir() {
|
||||
char tmpl[] = "/tmp/velox-nmhost-test-XXXXXX";
|
||||
const char* dir = ::mkdtemp(tmpl);
|
||||
return dir ? dir : "/tmp";
|
||||
}
|
||||
|
||||
// Firefox's own framing: 4-byte native-byte-order length, then that many bytes.
|
||||
std::string frame(const std::string& payload) {
|
||||
std::uint32_t len = static_cast<std::uint32_t>(payload.size());
|
||||
std::string out(reinterpret_cast<char*>(&len), 4);
|
||||
out += payload;
|
||||
return out;
|
||||
}
|
||||
|
||||
// Reads exactly one framed message off `fd`, blocking. Empty on EOF/short read.
|
||||
std::string read_frame(int fd) {
|
||||
std::uint32_t len = 0;
|
||||
std::size_t got = 0;
|
||||
while (got < 4) {
|
||||
const ssize_t n = ::read(fd, reinterpret_cast<char*>(&len) + got, 4 - got);
|
||||
if (n <= 0) return {};
|
||||
got += static_cast<std::size_t>(n);
|
||||
}
|
||||
std::string payload(len, '\0');
|
||||
got = 0;
|
||||
while (got < len) {
|
||||
const ssize_t n = ::read(fd, payload.data() + got, len - got);
|
||||
if (n <= 0) return {};
|
||||
got += static_cast<std::size_t>(n);
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
bool write_all(int fd, const std::string& s) {
|
||||
std::size_t off = 0;
|
||||
while (off < s.size()) {
|
||||
const ssize_t n = ::write(fd, s.data() + off, s.size() - off);
|
||||
if (n <= 0) return false;
|
||||
off += static_cast<std::size_t>(n);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
void run() {
|
||||
const std::string dir = make_temp_dir();
|
||||
const std::string velox_dir = dir + "/velox";
|
||||
CHECK(::mkdir(velox_dir.c_str(), 0700) == 0);
|
||||
const std::string sock_path = velox_dir + "/velox.sock";
|
||||
|
||||
// A bare listening socket standing in for veloxd.
|
||||
const int listen_fd = ::socket(AF_UNIX, SOCK_STREAM, 0);
|
||||
CHECK(listen_fd >= 0);
|
||||
sockaddr_un addr{};
|
||||
addr.sun_family = AF_UNIX;
|
||||
std::memcpy(addr.sun_path, sock_path.c_str(), sock_path.size());
|
||||
CHECK(::bind(listen_fd, reinterpret_cast<sockaddr*>(&addr), sizeof(addr)) == 0);
|
||||
CHECK(::listen(listen_fd, 1) == 0);
|
||||
|
||||
int child_stdin[2]; // [0] read (child), [1] write (parent)
|
||||
int child_stdout[2]; // [0] read (parent), [1] write (child)
|
||||
CHECK(::pipe(child_stdin) == 0);
|
||||
CHECK(::pipe(child_stdout) == 0);
|
||||
|
||||
::setenv("XDG_RUNTIME_DIR", dir.c_str(), 1);
|
||||
|
||||
const pid_t pid = ::fork();
|
||||
CHECK(pid >= 0);
|
||||
if (pid == 0) {
|
||||
::dup2(child_stdin[0], 0);
|
||||
::dup2(child_stdout[1], 1);
|
||||
::close(child_stdin[0]);
|
||||
::close(child_stdin[1]);
|
||||
::close(child_stdout[0]);
|
||||
::close(child_stdout[1]);
|
||||
::close(listen_fd);
|
||||
::execl(VELOX_NMHOST_BIN, "velox-nmhost", nullptr);
|
||||
::_exit(127);
|
||||
}
|
||||
::close(child_stdin[0]);
|
||||
::close(child_stdout[1]);
|
||||
|
||||
// Accept nmhost's connection.
|
||||
const int conn = ::accept(listen_fd, nullptr, nullptr);
|
||||
CHECK(conn >= 0);
|
||||
|
||||
// stdin (framed) -> socket (NDJSON line).
|
||||
CHECK(write_all(child_stdin[1], frame(R"({"hello":1})")));
|
||||
char line[256] = {};
|
||||
ssize_t n = ::read(conn, line, sizeof(line) - 1);
|
||||
CHECK(n > 0);
|
||||
CHECK_EQ(std::string(line, static_cast<std::size_t>(n)), std::string("{\"hello\":1}\n"));
|
||||
|
||||
// socket (NDJSON line) -> stdout (framed).
|
||||
CHECK(write_all(conn, "{\"world\":2}\n"));
|
||||
const std::string got = read_frame(child_stdout[0]);
|
||||
CHECK_EQ(got, std::string(R"({"world":2})"));
|
||||
|
||||
// A second round trip on the same connection, to prove buffering across calls works
|
||||
// (not just "the first message happens to line up with one read()").
|
||||
CHECK(write_all(child_stdin[1], frame(R"({"again":3})")));
|
||||
n = ::read(conn, line, sizeof(line) - 1);
|
||||
CHECK(n > 0);
|
||||
CHECK_EQ(std::string(line, static_cast<std::size_t>(n)), std::string("{\"again\":3}\n"));
|
||||
|
||||
// Firefox closes the pipe: nmhost must exit promptly rather than hang.
|
||||
::close(child_stdin[1]);
|
||||
int status = 0;
|
||||
pid_t waited = -1;
|
||||
for (int i = 0; i < 50 && waited != pid; ++i) {
|
||||
waited = ::waitpid(pid, &status, WNOHANG);
|
||||
if (waited == pid) break;
|
||||
struct timespec ts{0, 20'000'000}; // 20ms
|
||||
::nanosleep(&ts, nullptr);
|
||||
}
|
||||
CHECK_EQ(waited, pid);
|
||||
if (waited == pid) CHECK(WIFEXITED(status));
|
||||
|
||||
::close(conn);
|
||||
::close(child_stdout[0]);
|
||||
::close(listen_fd);
|
||||
::unlink(sock_path.c_str());
|
||||
::rmdir(velox_dir.c_str());
|
||||
::rmdir(dir.c_str());
|
||||
}
|
||||
|
||||
TEST_MAIN()
|
||||
@@ -0,0 +1,97 @@
|
||||
# packaging/nativehost — the Firefox native-messaging manifest
|
||||
|
||||
Owner: lane DAEMON (`packaging/nativehost/**` is explicitly DAEMON's per `CLAUDE.md`'s
|
||||
lane table, unlike the rest of `packaging/`, which is PKG/QA's). This directory holds the
|
||||
manifest `velox-nmhost` (built from `nmhost/`) is registered under, and this file is the
|
||||
one place that says exactly which on-disk locations need it and why — the
|
||||
"install manifests to all four locations" line in `docs/agents/AGENT-DAEMON.md`'s build
|
||||
order predates `docs/adr/0003-native-messaging-under-snap.md`, which cut that down to
|
||||
three *real* ones. Read the ADR before touching this list; it is the empirical spike, not
|
||||
a guess.
|
||||
|
||||
## The manifest
|
||||
|
||||
`com.velox.host.json` in this directory is the template:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "com.velox.host",
|
||||
"description": "Velox download manager — native messaging bridge to veloxd",
|
||||
"path": "/usr/libexec/velox/velox-nmhost",
|
||||
"type": "stdio",
|
||||
"allowed_extensions": ["velox@velox.download"]
|
||||
}
|
||||
```
|
||||
|
||||
- `name` is what the extension calls `browser.runtime.connectNative("com.velox.host")`
|
||||
with (`extension/src/background/transport/native.ts`'s `HOST_NAME`) — do not rename one
|
||||
without the other.
|
||||
- `allowed_extensions` must match `extension/manifest.json`'s
|
||||
`browser_specific_settings.gecko.id` exactly (`[email protected]`). A mismatch here is
|
||||
a silent failure: Firefox reports "no such native application" as if the manifest didn't
|
||||
exist at all, which reads exactly like a missing-file bug and is easy to chase in the
|
||||
wrong place.
|
||||
- `path` is `/usr/libexec/velox/velox-nmhost` — the `.deb`'s install layout
|
||||
(`docs/07-packaging.md`). **Every other packaging format needs a different `path`** —
|
||||
the binary doesn't live at that absolute path inside a Flatpak sandbox or an AppImage
|
||||
mount, so whatever installs the manifest for those formats must rewrite this field to
|
||||
wherever it actually put the binary, not ship this file verbatim. That substitution is
|
||||
the packaging step's job, not this template's.
|
||||
|
||||
## Where it has to be written, and why
|
||||
|
||||
Per ADR 0003 (spike S1, run against the real snap Firefox on the target machine — not a
|
||||
guess about how confinement "should" work):
|
||||
|
||||
| Location | Covers | Why |
|
||||
|---|---|---|
|
||||
| `~/.mozilla/native-messaging-hosts/com.velox.host.json` | **deb/tarball Firefox, and snap Firefox** | The one location snap Firefox 154 actually reads. Firefox's snap launches native-messaging hosts *outside* the sandbox with the user's real `$HOME` — so this is not "the deb location that happens to also work"; it is *the* snap location, full stop. `~/snap/firefox/common/.mozilla/native-messaging-hosts/` (the intuitive "inside the snap" path) is **not** read by Firefox 154 snap — confirmed empirically, not inferred. |
|
||||
| `/usr/lib/mozilla/native-messaging-hosts/com.velox.host.json` | **deb/tarball Firefox only** | System-wide, so it covers every user on the box for a real (non-snap) Firefox install — but confirmed **not** read by snap Firefox. Do not treat this path as "covers snap too"; that was the wrong assumption `docs/05` §4 corrected in the same commit as the ADR. |
|
||||
| `~/.var/app/org.mozilla.firefox/.mozilla/native-messaging-hosts/com.velox.host.json` | **Flatpak Firefox** | Flatpak's own sandboxed home. Untested on this machine (Firefox here is the snap, not flatpak) — carried over from `docs/05` §4's original four-location list, which this table otherwise supersedes. Verify before relying on it in a release checklist. |
|
||||
|
||||
That is three real locations, not four — the fourth
|
||||
(`~/snap/firefox/common/.mozilla/native-messaging-hosts/`) was the pre-ADR guess the spike
|
||||
disproved. If `docs/agents/AGENT-DAEMON.md`'s build order still says "four locations" when
|
||||
you read this, it is stale; this table is the current source of truth alongside the ADR
|
||||
itself.
|
||||
|
||||
## What this means for `postinst` (PKG/QA's file, not this one)
|
||||
|
||||
This directory ships the manifest template and documents the target paths; it does not
|
||||
install anything itself (`packaging/**` outside this one directory is PKG/QA's — see
|
||||
`CLAUDE.md`'s lane table — and `postinst` specifically is a `.deb`-packaging concern this
|
||||
lane doesn't own the file for). For whoever writes it:
|
||||
|
||||
- **The two `~/`-relative locations are per-user.** `postinst` runs as root, once, at
|
||||
install time — it does not run once per user session. It needs either a real-user
|
||||
enumeration at install time (every UID with a home directory and no login shell of
|
||||
`/usr/sbin/nologin`-style exclusions, roughly what `deluser --remove-home` scripts already
|
||||
have to reason about) or a first-run hook that runs as the logged-in user (a systemd user
|
||||
unit's `ExecStartPre`, or the GUI's own first-run wizard) and writes its own manifest the
|
||||
first time it starts. `docs/adr/0003`'s own follow-up section flagged this as open; it
|
||||
still is.
|
||||
- **`/usr/lib/mozilla/native-messaging-hosts/` is the only one of the three that's a plain
|
||||
root-owned, install-time write** — no per-user enumeration needed for that one.
|
||||
- **Detect snap Firefox and say so.** `docs/07-packaging.md` already commits to this
|
||||
("detects whether Firefox is a snap... prints... a one-line note that the extension will
|
||||
pair over loopback") — the detection matters here specifically because if Firefox turns
|
||||
out to be neither deb/tarball nor a snap this spike covers (a genuinely unknown or future
|
||||
packaging of Firefox), silently trusting `NativeTransport` to work is exactly the failure
|
||||
mode ADR 0003 exists to prevent. `WebSocketTransport` is the guaranteed fallback either
|
||||
way (ADR 0003's own decision) — nothing breaks if the manifest doesn't land correctly, it
|
||||
just means the extension pairs over loopback instead of the (opportunistic, not
|
||||
required) native path.
|
||||
- **`docs/07-packaging.md`'s own install layout line currently lists only the
|
||||
system-wide `/usr/lib/mozilla/...` path.** That line is correct as far as it goes (it *is*
|
||||
one of the three locations, and the only pure root-owned one) but reads as if it were the
|
||||
whole story for native messaging; it predates this file and the ADR. Worth a line
|
||||
pointing here so the two documents don't quietly disagree — PKG/QA's call, not edited
|
||||
here since `docs/07` is PKG/QA's own file.
|
||||
|
||||
## The binary
|
||||
|
||||
`nmhost/` builds `velox-nmhost` — see that directory's own `src/main.cpp` for what it does
|
||||
(a byte-level pump, no protocol logic) and `docs/05-extension-spec.md` §4 for the two
|
||||
transports it sits behind. It is deliberately dependency-free (not linked against any
|
||||
`veloxd_*` library) so wherever a packaging format's sandbox puts it, it has nothing else
|
||||
to go looking for at runtime.
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"name": "com.velox.host",
|
||||
"description": "Velox download manager — native messaging bridge to veloxd",
|
||||
"path": "/usr/libexec/velox/velox-nmhost",
|
||||
"type": "stdio",
|
||||
"allowed_extensions": ["[email protected]"]
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
# packaging/systemd — the user unit pair for veloxd
|
||||
|
||||
Provided by lane DAEMON (`daemon/docs/AGENT-DAEMON.md` build step 7: "systemd user units
|
||||
— `velox.service` + `velox.socket` for socket activation"), for PKG/QA to install per
|
||||
`docs/07-packaging.md`'s layout:
|
||||
|
||||
```
|
||||
/usr/lib/systemd/user/velox.service
|
||||
/usr/lib/systemd/user/velox.socket
|
||||
```
|
||||
|
||||
`packaging/` outside `nativehost/` is PKG/QA's per `CLAUDE.md`'s lane table; these two
|
||||
files are here because they are inputs to that packaging step, not a claim on the rest of
|
||||
the directory — same relationship `packaging/nativehost/` already has.
|
||||
|
||||
## Why both files, and what `RuntimeDirectory=` is doing
|
||||
|
||||
`velox.socket` binds `$XDG_RUNTIME_DIR/velox/velox.sock` **before `veloxd` ever runs** and
|
||||
hands the daemon the already-listening fd at startup (`daemon/src/rpc/systemd_activation.cpp`
|
||||
implements the receiving half — `LISTEN_PID`/`LISTEN_FDS`, fd 3 — without a `libsystemd`
|
||||
link). Two things this buys over the daemon binding its own socket on every start:
|
||||
|
||||
- **No window where a client gets `ECONNREFUSED`.** The socket exists and queues
|
||||
connections from the moment `velox.socket` is active, not from whenever `veloxd`
|
||||
finishes starting up — this is the actual point of socket activation, not just "start
|
||||
on demand."
|
||||
- **Cold-boot ordering is free.** Nothing has to wait for `veloxd` to be ready before the
|
||||
GUI, the CLI, or a native-messaging host can attempt a connection; the kernel queues it.
|
||||
|
||||
`RuntimeDirectory=velox` on the socket unit creates `%t/velox` (mode 0700) before the
|
||||
`ListenStream=` bind — without it, binding fails outright the first time (nothing has
|
||||
created the parent directory yet). `veloxd` itself creates that same directory
|
||||
(`ensure_private_dir` in `runtime_dir.cpp`) for the case where it's started directly,
|
||||
outside systemd (`./veloxd` in a terminal, still supported and how most of this daemon's
|
||||
own testing runs) — the two paths converge on the same directory with the same mode
|
||||
either way.
|
||||
|
||||
`UdsServer::start()` (`daemon/src/rpc/uds_server.cpp`) checks for the activated fd first
|
||||
and, if present, skips create/bind/chmod/listen entirely — the socket file's lifecycle then
|
||||
belongs to the unit (including `RemoveOnStop=yes` on stop), not to the daemon. Falls back
|
||||
to binding its own socket exactly as before when not socket-activated (a manual run, or a
|
||||
distro that ships the daemon without the unit files).
|
||||
|
||||
## What was deliberately left out
|
||||
|
||||
`velox.service` does **not** set `ProtectSystem=`, `ProtectHome=`, or `ReadWritePaths=`.
|
||||
`saveTo.allowedRoots` is user-configurable to anywhere on the filesystem — an external
|
||||
drive, a second mount, anywhere `fs/safepath.hpp`'s own canonicalize-and-check accepts —
|
||||
not a fixed set of directories a unit file could enumerate ahead of time. A filesystem-level
|
||||
sandbox here would turn a legitimately-configured save location into an opaque
|
||||
`EROFS`/`EACCES` the daemon can't explain, in place of its own clear `-32011` — worse than
|
||||
no sandbox, specifically for a download manager. `NoNewPrivileges=yes` is kept: it has no
|
||||
such trade-off.
|
||||
|
||||
## Verifying socket activation without a real install
|
||||
|
||||
`systemd-analyze verify --user velox.service velox.socket` checks unit-file syntax (it
|
||||
will complain that `/usr/bin/veloxd` and the `velox(1)` man page don't exist on a dev
|
||||
box that hasn't installed the package — expected, not a unit bug). To exercise the actual
|
||||
activation handshake without installing anything: bind a Unix socket, `dup2` it onto fd 3,
|
||||
fork, set `LISTEN_PID=<child pid>` and `LISTEN_FDS=1` in the child's environment, clear
|
||||
`FD_CLOEXEC` on fd 3, and `execve` `veloxd` — a real `session.hello` round-trips over that
|
||||
fd with no `bind()`/`listen()` call ever happening inside the daemon for that run. This is
|
||||
exactly what `velox.socket`'s `Requires=`/`ExecStart` sequence does in production; systemd
|
||||
supplies the fd, `veloxd` doesn't know the difference.
|
||||
@@ -0,0 +1,38 @@
|
||||
[Unit]
|
||||
Description=Velox download manager daemon
|
||||
Documentation=man:velox(1)
|
||||
# Socket activation (velox.socket) means this unit does not need to be enabled or started
|
||||
# directly for the RPC transport to come up on demand — the first connection attempt after
|
||||
# boot starts veloxd with the listening socket already bound (see velox.socket's own
|
||||
# comment). Requires=/After= still matter for a manual `systemctl --user start velox`.
|
||||
Requires=velox.socket
|
||||
After=velox.socket
|
||||
|
||||
# Never more than one real instance for this user regardless of how it was started — the
|
||||
# abstract-socket single-instance lock (main.cpp, keyed off the resolved runtime dir) is
|
||||
# the actual enforcement; this just keeps systemd itself from racing two starts.
|
||||
StartLimitIntervalSec=60
|
||||
StartLimitBurst=5
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/bin/veloxd
|
||||
# main.cpp's SIGTERM handler stops the event loop and falls through to a clean shutdown
|
||||
# (flushes buffers, closes the store, releases the single-instance lock) — the default
|
||||
# KillSignal=SIGTERM and TimeoutStopSec are already the right shape for that; no
|
||||
# ExecStop/KillMode override needed.
|
||||
Restart=on-failure
|
||||
RestartSec=2
|
||||
|
||||
# Hardening deliberately stops here, not at ProtectSystem=/ProtectHome=/ReadWritePaths=:
|
||||
# saveTo.allowedRoots is user-configurable to anywhere (an external drive, a second
|
||||
# mount — fs/safepath.hpp is the daemon's own validation boundary, not a fixed set of
|
||||
# directories a unit file could enumerate up front). A filesystem-level sandbox here would
|
||||
# silently turn a legitimately-configured save location into an opaque EROFS/EACCES the
|
||||
# daemon can't explain, instead of its own clear -32011 — worse than no sandbox, for a
|
||||
# download manager specifically. NoNewPrivileges is free of that trade-off.
|
||||
NoNewPrivileges=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
Also=velox.socket
|
||||
@@ -0,0 +1,28 @@
|
||||
[Unit]
|
||||
Description=Velox download manager — RPC socket
|
||||
|
||||
[Socket]
|
||||
# %t is $XDG_RUNTIME_DIR for a user unit — the exact path veloxd itself resolves
|
||||
# (daemon/src/rpc/runtime_dir.cpp's resolve_runtime_dir), and the exact path velox(1)
|
||||
# resolves too (default_socket_path() in cli/src/client.cpp). All three must agree; this
|
||||
# is the one line that has to.
|
||||
ListenStream=%t/velox/velox.sock
|
||||
|
||||
# RuntimeDirectory creates %t/velox (mode 0700, this user's own) before binding, so the
|
||||
# ListenStream= path above always has somewhere to land — veloxd itself does the same
|
||||
# thing (ensure_private_dir) when it creates the directory unassisted (the non-activated
|
||||
# path, e.g. a manual `veloxd` run outside systemd).
|
||||
RuntimeDirectory=velox
|
||||
RuntimeDirectoryMode=0700
|
||||
|
||||
# Same-UID-only, matching the socket's own authorization once a connection is accepted
|
||||
# (SO_PEERCRED, checked again in uds_server.cpp regardless of this mode bit — belt and
|
||||
# braces, not a substitute for it).
|
||||
SocketMode=0600
|
||||
|
||||
# The socket file belongs to systemd's socket-activation state, not to whatever's on disk
|
||||
# from a previous boot; remove it on stop so a stale entry never shadows the next start.
|
||||
RemoveOnStop=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=sockets.target
|
||||
@@ -160,11 +160,18 @@ if [ -z "$EXTERNAL_UDS" ] && [ -z "$EXTERNAL_WS" ]; then
|
||||
for _ in $(seq 1 50); do [ -S "$VUDS" ] && break; sleep 0.2; done
|
||||
[ -S "$VUDS" ] || { echo "veloxd did not start:"; cat "$WORK/veloxd.log"; exit 1; }
|
||||
|
||||
# saveTo.allowedRoots defaults to ["~/Downloads"]; download.add.json (fixture) asks
|
||||
# for a saveDir under $HOME/Downloads, so both that and download.add's own isolated
|
||||
# downloads dir need to be allowed roots, or every download.add fixture fails -32011
|
||||
# before the point of this runner is even reached. settings.set is itself a D3 stub,
|
||||
# so this is written straight into the isolated velox.db rather than over the wire.
|
||||
# saveTo.allowedRoots defaults to ["~/Downloads"]; download.add's own isolated
|
||||
# downloads dir needs to be an allowed root too, or every download.add fixture fails
|
||||
# -32011 before the point of this runner is even reached. $HOME/Downloads stays in
|
||||
# the list alongside it: a few fixtures still set an explicit saveDir there
|
||||
# (category.upsert.json, download.update.json) rather than take the default.
|
||||
# capture.minSizeBytes defaults to 0 (nothing is ever "too small"), which makes
|
||||
# errors/capture.offer.ignore.json's below-minimum-size case impossible to reach
|
||||
# against a fresh daemon; raised here so that fixture's scenario is actually
|
||||
# reachable. Written straight into the isolated velox.db, before veloxd has any RPC
|
||||
# session to write it through: settings.set is real now (D9), but this has to be in
|
||||
# place before the very first fixture runs, and setup happens before any connection
|
||||
# exists.
|
||||
python3 - "$VXDG/data/velox/velox.db" "$VXDG/downloads" "$HOME/Downloads" <<'PY'
|
||||
import json, sqlite3, sys
|
||||
db_path, isolated_downloads, home_downloads = sys.argv[1:4]
|
||||
@@ -174,6 +181,11 @@ db.execute(
|
||||
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
|
||||
("saveTo.allowedRoots", json.dumps([isolated_downloads, home_downloads])),
|
||||
)
|
||||
db.execute(
|
||||
"INSERT INTO settings(key, value) VALUES(?, ?) "
|
||||
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
|
||||
("capture.minSizeBytes", json.dumps(1000000)),
|
||||
)
|
||||
db.execute(
|
||||
"INSERT INTO settings(key, value) VALUES(?, ?) "
|
||||
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
|
||||
|
||||
@@ -1,40 +1,26 @@
|
||||
[
|
||||
{ "fixture": "contracts/fixtures/download.refreshUrl.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/download.update.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/grabber.harvest.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
|
||||
{ "fixture": "contracts/fixtures/grabber.start.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
|
||||
{ "fixture": "contracts/fixtures/grabber.status.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/rules.list.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/rules.upsert.json", "reason": "D3: stub handler, -32603" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/settings.get.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/settings.set.json", "reason": "D3: stub handler, -32603" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/limiter.get.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/limiter.set.json", "reason": "D3: stub handler, -32603" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/schedule.get.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/schedule.set.json", "reason": "D3: stub handler, -32603" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/queue.reorder.json", "reason": "D3: stub handler, -32603" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/grabber.harvest.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/grabber.start.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/grabber.status.json", "reason": "D3: stub handler, -32603" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/media.addVariant.json", "reason": "D3: stub handler, -32603" },
|
||||
{ "fixture": "contracts/fixtures/media.listVariants.json", "reason": "D3: stub handler, -32603" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/capture.getRules.json", "reason": "D3: stub handler, -32603 -- DAEMON is filing capture.offer next" },
|
||||
{ "fixture": "contracts/fixtures/capture.offer.take.json", "reason": "D3: stub handler, -32603 -- DAEMON is filing capture.offer next" },
|
||||
{ "fixture": "contracts/fixtures/errors/capture.offer.ignore.json", "reason": "D3: stub handler, -32603 -- DAEMON is filing capture.offer next" },
|
||||
{ "fixture": "contracts/fixtures/media.addVariant.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
|
||||
{ "fixture": "contracts/fixtures/media.listVariants.json", "reason": "D3: stub handler, -32603 (M4 territory per deferrals.md)" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/errors/download.provideAuth.not-found.json", "reason": "real bug: on_download_provideAuth (dispatcher.cpp) never checks the task exists -- TaskActionPort::provide_auth returns false for an unknown id, which the handler folds into a normal {ok:false} result instead of -32010" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/category.list.json", "reason": "documented gap (deferrals.md D3a note): categories table has no mimeTypes/sortOrder columns, so category.upsert accepts them but category.list never echoes mimeTypes back" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/capture.getRules.json", "reason": "not a bug: capture.monitoredMimeTypes defaults to [] (store/settings.cpp's kDefaults) on a fresh daemon; the golden's non-empty example illustrates a configured one" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/download.probe.json", "reason": "not a bug: requiresAuth is optional-and-omitted-when-false (schema doesn't require it); the golden shows it because that fixture's probe hit a 401, this run's doesn't" },
|
||||
{ "fixture": "contracts/fixtures/download.get.json", "reason": "not a bug: effectiveUrl is 'null until the first probe succeeds' (schema) and omitted rather than sent as null; our bound $taskId is a fresh, never-started task, so it's never been probed -- the golden depicts an in-progress download instead" },
|
||||
{ "fixture": "contracts/fixtures/download.list.json", "reason": "same as download.get.json: effectiveUrl omitted for our never-started bound tasks, golden depicts an in-progress download" },
|
||||
{ "fixture": "contracts/fixtures/session.hello.json", "reason": "not a bug: capabilities is genuinely empty because media/grabber/Secret Service aren't implemented yet; the golden's ['media','grabber','secretservice'] illustrates a future daemon, not this one" },
|
||||
{ "fixture": "contracts/fixtures/download.update.json", "reason": "not a bug: etaSeconds is only known for a task the engine has probed/is running; our bound $taskId is a fresh, never-started task, so it's absent -- same class as download.get.json's effectiveUrl" },
|
||||
{ "fixture": "contracts/fixtures/session.hello.json", "reason": "not a bug: capabilities is genuinely empty because media/grabber aren't implemented yet (capture/Secret Service's parts of it now are); the golden's example list illustrates a future daemon, not this one" },
|
||||
|
||||
{ "fixture": "contracts/fixtures/queue.start.json", "reason": "not a bug: startedTaskIds is empty because nothing is a member of queue 'main' in this isolated run; the golden depicts a queue with real membership" },
|
||||
{ "fixture": "contracts/fixtures/queue.reorder.json", "reason": "not a bug: the fixture's taskIds are two literal ids that only ever existed in a seeded mock; queue 'main' has no members at all in this isolated run, so any non-empty list is correctly rejected as not a permutation of (empty) membership" },
|
||||
{ "fixture": "contracts/fixtures/rules.list.json", "reason": "not a bug: no rule is ever seeded in a fresh daemon; the golden depicts a configured rule set" },
|
||||
{ "fixture": "contracts/fixtures/schedule.set.json", "reason": "documented gap (deferrals.md D3f note): nextRunAt is deliberately left unset -- computing it needs DST-aware next-transition logic sched/schedule_window.hpp doesn't have yet" },
|
||||
{ "fixture": "contracts/fixtures/category.remove.json", "reason": "not a bug: reassignedTaskIds is empty because nothing was ever filed under the 'firmware' category this run creates; the golden depicts a category with real membership" }
|
||||
]
|
||||
|
||||
+46
-72
@@ -88,88 +88,62 @@ manual testing.
|
||||
nowhere to run. GUI owns the harness; PKG/QA owns the CI job. This is the wiring contract
|
||||
so the two halves meet without another round trip.
|
||||
|
||||
### What PKG/QA needs from GUI
|
||||
### What GUI built, and the invocation contract
|
||||
|
||||
A driver invoked as `gui/tests/dod/run.sh <gate> [--json <path>]` (exact path TBD by GUI),
|
||||
headless-capable (Xvfb or offscreen `QT_QPA_PLATFORM`), with:
|
||||
`gui/tests/dod/run.sh <gate> [--json <path>]` (`gate` one of `scroll-60fps` / `rss-flat`
|
||||
/ `unhappy-path`), backed by `gui/tests/dod/dod_harness.cpp` (target `gui-dod-harness`).
|
||||
Headless by default — `run.sh` sets `QT_QPA_PLATFORM=offscreen` itself, so the CI jobs
|
||||
below need no Xvfb.
|
||||
|
||||
| `<gate>` | Pass / fail condition | Budget |
|
||||
|---|---|---|
|
||||
| `scroll-60fps` | `mockd --tasks 10000`, scripted fling scroll; **fail** if p99 frame > 16.6 ms | per-PR |
|
||||
| `rss-flat` | `mockd --tasks 10000` + progress events, 10 min; **fail** if RSS growth > a fixed slack (GUI picks the number, states it) | nightly |
|
||||
| `unhappy-path` | `mockd --slow` / `--flaky <f>` / `--drop-connection <s>`; **fail** on crash, on watchdog-detected hang, or if connection state never returns to `Connected` | per-PR |
|
||||
| `scroll-60fps` | `mockd --tasks 10000`, scripted fling scroll; **fail** if p99 frame > 16.6 ms (×4 under an ASan/UBSan build — `VELOX_DOD_FRAME_BUDGET_MS` overrides outright) | per-PR |
|
||||
| `rss-flat` | `mockd --tasks 10000` + progress events, 10 min; **fail** if RSS growth past warm-up exceeds a 20 MiB slack (`VELOX_DOD_RSS_SLACK_KIB` overrides) | nightly |
|
||||
| `unhappy-path` | Three phases (`--slow` / `--flaky <f>` / `--drop-connection <s>`), one `mockd` restart each; **fail** on crash, on the harness's own watchdog firing (75 s), or if connection state never (re)reaches `Connected` | per-PR |
|
||||
|
||||
Contract:
|
||||
Contract, as built: exit `0` pass, non-zero fail; a hang is the harness's own watchdog
|
||||
converting itself into a non-zero exit (`3`), never something CI needs `timeout(1)`
|
||||
around; `--json` writes one result object per gate (`unhappy-path` merges its three
|
||||
phases into one file); no network, no writes outside a tempdir, no leaked child process
|
||||
on any exit path (`run.sh`'s `trap cleanup EXIT INT TERM`).
|
||||
|
||||
* exit `0` pass, non-zero fail; a hang is the harness's own watchdog to catch and turn
|
||||
into a non-zero exit, not something CI should have to `timeout(1)` around.
|
||||
* `--json` writes one machine-readable result file (measured p99, RSS series, recovery
|
||||
time) so the job can upload it as an artifact and a regression is a diff, not a re-run.
|
||||
* no network, no writes outside a tempdir, no leaked child processes on failure.
|
||||
### CI jobs — wired
|
||||
|
||||
### CI job — pre-drafted, add once the harness path is fixed
|
||||
`gui-dod` (per-PR: `scroll-60fps` + `unhappy-path`) and `gui-dod-nightly` (`rss-flat`,
|
||||
`schedule`/`workflow_dispatch` only) are live in `ci.yml`. Both bootstrap, build
|
||||
`gui-dod-harness`, `npm ci` in `tools/mockd`, run `gui/tests/dod/run.sh`, and upload the
|
||||
`--json` output as an artifact — no Xvfb step, since `run.sh` already runs offscreen.
|
||||
|
||||
```yaml
|
||||
gui-dod:
|
||||
# Per-PR GUI gates. The 10-minute rss-flat gate is in gui-dod-nightly, not here.
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Bootstrap toolchain
|
||||
run: sudo ./tools/bootstrap.sh
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22' # tools/mockd
|
||||
- name: Configure + build
|
||||
run: |
|
||||
cmake --preset dev
|
||||
cmake --build --preset dev --target velox-gui
|
||||
- name: Install mockd
|
||||
run: cd tools/mockd && npm ci
|
||||
- name: Xvfb + gates
|
||||
run: |
|
||||
sudo apt-get install -y --no-install-recommends xvfb
|
||||
xvfb-run -a gui/tests/dod/run.sh scroll-60fps --json scroll.json # TODO(GUI): path
|
||||
xvfb-run -a gui/tests/dod/run.sh unhappy-path --json unhappy.json
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: gui-dod-${{ github.run_id }}
|
||||
path: "*.json"
|
||||
### Forced red, once per gate, before wiring it required
|
||||
|
||||
gui-dod-nightly:
|
||||
if: github.event_name == 'schedule'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Bootstrap toolchain
|
||||
run: sudo ./tools/bootstrap.sh
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
- name: Configure + build
|
||||
run: |
|
||||
cmake --preset dev
|
||||
cmake --build --preset dev --target velox-gui
|
||||
- run: cd tools/mockd && npm ci
|
||||
- name: RSS soak (10 min)
|
||||
run: |
|
||||
sudo apt-get install -y --no-install-recommends xvfb
|
||||
xvfb-run -a gui/tests/dod/run.sh rss-flat --json rss.json
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: gui-dod-rss-${{ github.run_id }}
|
||||
path: rss.json
|
||||
```
|
||||
| Gate | Forced via | Observed |
|
||||
|---|---|---|
|
||||
| `scroll-60fps` | `VELOX_DOD_FRAME_BUDGET_MS=0.01` | `FAIL: p99=50.73 ms mean=31.34 ms max=52.88 ms budget=0.01 ms over 240 steps, 10000 rows`, exit 1 |
|
||||
| `rss-flat` | `VELOX_DOD_RSS_SLACK_KIB=-999999999` (guarantees `growthKiB > slack` regardless of actual RSS behavior that run) | `FAIL: growth=13644 KiB slack=-999999999 KiB over 15s (warmup 1s), 10000 rows`, exit 1 |
|
||||
| `unhappy-path` | ran `gui-dod-harness` directly (bypassing `run.sh`, which always starts a working `mockd`) against a `--sock` path with nothing listening | 75 s of `Connecting`/`Reconnecting`, then `FAIL: dod_harness watchdog fired — hung past 75s`, exit 3 |
|
||||
|
||||
`gui-dod-nightly` needs `if: github.event_name == 'schedule'` (the `nightly-integration`
|
||||
job below already added that trigger to `ci.yml` — `cron: '17 3 * * *'` — so this no
|
||||
longer needs its own).
|
||||
The real (non-forced) runs all pass live: `scroll-60fps` p99 51.28 ms against a 66.4 ms
|
||||
budget (ASan/UBSan build), `rss-flat` growth 13504 KiB against the 20480 KiB slack over a
|
||||
15 s smoke duration, `unhappy-path` all three phases `PASS` with `finalConnected=true`.
|
||||
|
||||
### Known gap: `unhappy-path`'s drop-connection phase doesn't drop anything
|
||||
|
||||
Filed by GUI in `gui/docs/proto-requests-m1.md`: `mockd --drop-connection` only works
|
||||
over WebSocket — `startUds()` never wires the periodic-drop timer `startWs()` has, so the
|
||||
UDS transport (GUI/CLI/nmhost's only one) never sees a connection actually die. GUI's own
|
||||
live verification: 45 s observing a `--drop-connection 5` mockd over UDS, `stateChanged`
|
||||
never fires. The phase still runs and its JSON honestly records `sawDisruption: false`
|
||||
rather than silently passing as if it proved something — that's what the real run above
|
||||
shows, `pass: true` alongside `sawDisruption: false`, so a reviewer reading the JSON sees
|
||||
exactly how much this phase currently covers.
|
||||
|
||||
This is PROTO's fix, not PKG/QA's or GUI's to route around locally — coordinating on it
|
||||
rather than patching mockd from this lane. Once PROTO lands `dropEverySec` on `startUds`,
|
||||
`unhappy-path`'s drop-connection phase starts exercising a real drop and this note comes
|
||||
out; until then `gui-dod` stays required as specified (crash/hang/never-reconnects still
|
||||
catch real regressions), just not yet catching a swallowed real disconnect.
|
||||
|
||||
### Status
|
||||
|
||||
Blocked on GUI's harness. Not urgent (GUI M1 DoD, not M0). When GUI files the follow-up
|
||||
with the real `run.sh` path and the `rss-flat` slack number, PKG/QA drops the `TODO(GUI)`
|
||||
markers and marks `gui-dod` required. The `schedule:` trigger `gui-dod-nightly` needs is
|
||||
already in `ci.yml`.
|
||||
Wired and required: `gui-dod` runs per-PR, `gui-dod-nightly` on schedule. Revisit once
|
||||
PROTO's UDS `--drop-connection` fix lands (see above).
|
||||
|
||||
@@ -288,7 +288,10 @@ export class Dispatcher {
|
||||
}
|
||||
|
||||
case 'limiter.get':
|
||||
return state.limiter;
|
||||
// applyToRunning is a write-only instruction on limiter.set ("retune already-
|
||||
// running transfers now"), not a persisted setting, so get never echoes it back
|
||||
// (contracts/fixtures/limiter.get.json).
|
||||
return { enabled: state.limiter.enabled, globalBps: state.limiter.globalBps };
|
||||
|
||||
case 'limiter.set': {
|
||||
state.limiter = {
|
||||
|
||||
Reference in New Issue
Block a user