scaffold: project structure, wire contract, roadmap and agent briefs

Lays out Velox Download Manager (IDM-class download manager for Ubuntu
26.04) as a monorepo ready for parallel lane development. No implementation
code by design.

- docs/: architecture, roadmap M0-M7, IDM-parity GUI spec, engine design,
  Firefox extension spec, risks/spikes, packaging
- contracts/: wire-contract skeleton (JSON Schema + fixture templates) —
  the single synchronization point between lanes
- docs/agents/: one brief per lane (PROTO, CORE, DAEMON, GUI, EXT, PKG/QA)
  with owned directories, build order and definition of done
- CLAUDE.md: rules of engagement — lane ownership, layering, non-negotiables
- CMake scaffolding with dev/tsan/release/ci presets

Two environment findings shape the design: Firefox here is the Mozilla snap
(native-messaging risk, so the extension carries a loopback-WebSocket
fallback), and Wayland forbids passive clipboard monitoring (so clipboard
capture is explicit-action-first).

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
2026-09-09 18:21:11 +04:00
co-authored by Claude Opus 5
commit 8bb683b09d
90 changed files with 1814 additions and 0 deletions
+60
View File
@@ -0,0 +1,60 @@
# Agent brief — CORE (`libveloxcore`)
**Starts when PROTO freezes `VERSION`. The critical path runs through you.**
## You own
```
core/** tools/bench/** tools/fuzz/**
```
You may read `contracts/` and `docs/`. You write nowhere else — in particular **you never
touch `daemon/`**: if the daemon needs something, expose it as a core API and tell DAEMON.
## Read first
`docs/04-engine-design.md` end to end. It is your specification, not background reading.
## Hard architectural constraints
- **No JSON, no SQL, no Qt, no RPC in `core/`.** Your public API takes a `DownloadSpec`
and emits typed callbacks. If you find yourself including a protocol header, stop.
- C++23, `-Wall -Wextra -Werror`, no raw `new`/`delete`, no naked `pthread`.
- Every public header under `core/include/vdm/` compiles standalone.
- Errors are returned (`std::expected`-style `Result<T>`), not thrown, on the transfer path.
- **No allocation in the curl write callback.** The ring buffer is preallocated at task
start. This is checked in review and by a bench assertion.
## Build order
1. `util/``Result<T>`, event bus, thread pool, logging, byte-span helpers.
2. `net/http_client` — libcurl multi wrapper, one `curl_multi` per worker thread, poll
loop, proxy/auth/redirect/cookie plumbing.
3. `net/probe` — HEAD then ranged-GET fallback, header parsing, **`Content-Disposition`
RFC 5987/6266 including the legacy forms** (this is a classic source of mojibake — give
it its own test table and a fuzz target).
4. `io/sparse_file` + `io/write_buffer``posix_fallocate`, per-segment ring buffer sized
by `buffer_bytes`, `pwrite` at absolute offsets, `posix_fadvise(DONTNEED)`, timed
`fdatasync`.
5. `meta/veloxpart` — the resume sidecar in `docs/04` §5, CRC-verified, `fdatasync`'d at
segment boundaries. **Write the reader first and fuzz it** — this file is attacker-
adjacent (it lives in a world-writable-ish download dir).
6. `segment/segmenter` + `segment/stealer` — dynamic segment stealing, `min_segment_bytes`
floor, per-host connection caps.
7. `rate/token_bucket` — hierarchical global → queue → task.
8. `task/download_task` — the state machine in `docs/04` §1, retry/backoff policy, mirrors.
9. `rules/` — filename sanitization + collision policy + category matching (pure functions;
DAEMON supplies the rule table).
10. `media/`**M4, not now.** Leave the directory empty.
## Definition of done (M1)
- 5 GB download saturates a 1 Gbit link at ≤ 8 % of one core (recorded in `tools/bench/`).
- `kill -9` at ~60 % → resume completes → SHA-256 matches the reference byte for byte.
- Every hostile mode in `tools/testserver` handled: no-Range, lying `Accept-Ranges`,
ETag change mid-download, 401, 416, redirect chains, slow-loris, connection reset,
expiring signed URL, `Content-Length` mismatch.
- ASan + UBSan + TSan clean under a 20-task load test.
- Fuzz targets for `Content-Disposition`, the `.veloxpart.meta` reader, and URL parsing run
1 M+ execs with no crash.
- Public API documented in `core/include/vdm/README.md` and reviewed by DAEMON before M2.
## Do not
- Do not start with `io_uring`. It's a post-1.0 experiment gated on a ≥10 % bench win.
- Do not implement HLS/DASH in M1.
- Do not add a "just for testing" JSON dependency.
+59
View File
@@ -0,0 +1,59 @@
# Agent brief — DAEMON (`veloxd`)
**Starts with CORE. You are the only process that owns state.**
## You own
```
daemon/** cli/** nmhost/** packaging/nativehost/**
```
Read `contracts/`, `core/include/`, `docs/`. Never write in `core/`, `gui/`, or `extension/`.
## Read first
`docs/01-architecture.md` §2–§5, `contracts/README.md`, `docs/05-extension-spec.md` §4
(you implement the daemon half of pairing).
## Build order
1. **RPC server**`rpc/uds_server` (NDJSON over `$XDG_RUNTIME_DIR/velox/velox.sock`,
0600, `SO_PEERCRED` same-UID check) and `rpc/ws_server` (bind `127.0.0.1` only, first
free port in 5200052016, write the chosen port to `ws.port`). One dispatcher, generated
from `contracts/`. Never block the RPC loop — disk and network work goes to CORE's pools.
2. **Auth & pairing**`session.pair` triggers a user prompt (GUI dialog if connected,
else a desktop notification with actions). Tokens: 256-bit, stored **hashed**,
per-install, revocable. Failed-auth rate limit 5/min then 60 s lockout. Enforce
`x-transports` and `x-privileged` from the schema: privileged methods are refused over
WS with `-32003`.
3. **Store** — SQLite WAL. Tables: `tasks`, `segments`, `categories`, `queues`, `rules`,
`settings`, `history`, `pairings`. Numbered migrations in `store/migrations/`, applied
at startup, with a forward-only test from every released schema version.
**Credentials never go in SQLite** — Secret Service via libsecret.
4. **Scheduler & queues** — concurrency governor (global max active, per-queue max, per-host
caps), time windows, days-of-week, one-shot vs periodic, "when queue completes" actions.
5. **Event fan-out** — per-subscription filtering, and **`event.task.progress` batched at
≤ 4 Hz into a single array message**. Do not emit one message per task per tick; that is
how you turn 20 downloads into a GUI that burns a core.
6. **Capture endpoint**`capture.offer` must answer within **750 ms**, always. Apply the
rules table, resolve the category folder, dedupe against active tasks, return
`take`/`ignore`. If anything internally is slow, answer `ignore` and let Firefox have
it. Never make the browser wait.
7. **Integration** — systemd user units (`velox.service` + `velox.socket` for socket
activation), single-instance lock, XDG autostart, `org.freedesktop.Notifications`,
graceful shutdown that flushes buffers and meta files.
8. **`velox` CLI** — `add`, `ls`, `pause`, `resume`, `rm`, `queue`, `settings`, `--json`
output. Build this early: it is how you test the daemon before the GUI exists.
9. **`velox-nmhost`** — 4-byte-length-prefixed stdio ⇄ Unix socket pump. **Under 300 lines,
zero business logic**, and it must exit cleanly when Firefox closes the pipe. Install
manifests to all four locations listed in `docs/05` §4.
## Definition of done (M1)
- Passes the full conformance suite as a server, over **both** transports.
- Kill and restart the daemon mid-download: all tasks reload with correct state and resume.
- 1 000 tasks in the DB: `download.list` with paging under 50 ms.
- Pairing flow works from a real Firefox extension; unpair revokes immediately.
- `systemctl --user status velox` clean; socket activation verified from cold.
- Security review passed: no bind beyond loopback, no path traversal in `saveDir`
(canonicalize and check against allowed roots → `-32011`), no plaintext secrets.
## Do not
- Do not put download logic here — that's CORE. You schedule and persist; CORE transfers.
- Do not invent protocol fields. File a request with PROTO.
+67
View File
@@ -0,0 +1,67 @@
# Agent brief — EXT (Firefox extension)
**Starts the day PROTO freezes the contract. Develops against `tools/mockd` over the
WebSocket transport — you never wait for `veloxd`.**
## You own
```
extension/**
```
Read `contracts/`, `docs/`. Never write in `core/`, `daemon/`, `gui/`, or `nmhost/`
(the native host belongs to DAEMON; if you need a change there, file it).
## Read first
`docs/05-extension-spec.md` in full, then `docs/06` R1 (snap Firefox) and R3 (AMO).
## Your first task is a spike, not code
**Spike S1 (2 days, blocks nothing else):** on a clean Ubuntu 26.04 VM with snap Firefox,
determine empirically which of the four native-messaging manifest locations actually work,
and whether the launched host can reach `$XDG_RUNTIME_DIR`. Write
`docs/adr/0003-native-messaging-under-snap.md`. Then build the WebSocket transport first
regardless of the answer — it is the one that is known to work here
(`snap connections firefox` shows `network`/`network-bind` connected).
## Build order
1. **`transport/`** — the `Transport` interface, `WebSocketTransport` (port discovery over
5200052016 + `session.hello` verification), pairing flow with token in
`browser.storage.local`, reconnect with backoff, then `NativeTransport`, then
`transport/index.ts` picking at runtime with a manual override in Options.
2. **`capture/headers.ts`** — `onBeforeSendHeaders` stash keyed by `requestId`, ring buffer,
5-minute TTL, bounded size (a leak here eats the browser's memory).
3. **`capture/rules.ts`** — `shouldCapture()` exactly as specified in `docs/05` §2.
**Table-driven tests before implementation.** This function is where the bugs will be:
too eager and you hijack page navigations; too shy and you're not a download manager.
4. **`capture/index.ts`** — the `onHeadersReceived` blocking hook: gather cookies, call
`capture.offer` with a **750 ms budget**, `{cancel: true}` only on `take`.
**Fail-open is a hard requirement** — daemon down, slow, or erroring means Firefox
downloads normally. Write that test before the feature.
5. **`capture/downloads-api.ts`** — the `downloads.onCreated` safety net for what slips past.
6. **`context-menus.ts`**, **`popup/`** (live progress from relayed events, pause/resume,
status dot), **`options/`** (transport, pairing/unpair, monitored types synced via
`capture.getRules`, min size, exclusions, default category, bypass modifier).
7. **`content/` media detection** — `.m3u8`/`.mpd`/MIME sniffing plus a `MediaSource`
observer; in-page "Download this video ▾" panel listing variants from
`media.listVariants`. **The extension never parses manifests** — the daemon does.
Detect DRM/EME and grey the button out with "Protected content".
## Definition of done (M1)
- Intercepts a real download in real Firefox and hands it to `mockd`.
- **Fail-open proven by an automated test:** kill the mock mid-flow, the file still
downloads through Firefox, no error dialog, no lost download.
- Pairing works, unpair revokes, token survives a browser restart, and a wrong token is
rejected and rate-limited.
- `web-ext lint` clean; no remote code, no `eval`, no CDN script — AMO rejects those.
- `shouldCapture` test table covers: attachment, monitored extension, monitored MIME,
size threshold, excluded host, HTML navigation, blob URL, bypass modifier held,
streaming media, and a range request the page itself issued.
- Popup shows live progress at ≤ 4 Hz without pinning a core.
- Permission justification written and committed for AMO submission.
## Do not
- Do not implement any download logic in the extension. You collect URL + headers + cookies
and hand them over. That's the whole job.
- Do not add a framework (React/Vue) for a popup and an options page; plain TS keeps the
AMO review and the bundle small.
- Do not invent protocol fields — file a request with PROTO.
+56
View File
@@ -0,0 +1,56 @@
# Agent brief — GUI (`velox-gui`)
**Starts the day PROTO freezes the contract. You never wait for the daemon — you develop
entirely against `tools/mockd`.**
## You own
```
gui/**
```
Read `contracts/`, `docs/`. Never write in `core/`, `daemon/`, or `extension/`.
## Read first
`docs/03-gui-spec.md` — it is the spec, screen by screen. Build in the order below.
## Build order
1. **`rpc/` client** — wrap the generated C++ client: async calls on a worker thread,
Qt signals on the main thread, auto-reconnect with exponential backoff, an offline
banner instead of a modal error, and a "Connected/Reconnecting" dot in the status bar.
Get this right first; everything else depends on it.
2. **`models/DownloadTableModel`** — `QAbstractItemModel` over `TaskSummary`. Apply
`event.task.progress` batches as **row patches with narrow `dataChanged` ranges**.
Never `beginResetModel()` on a progress tick. Sorting/filtering via `QSortFilterProxyModel`
per category tree selection.
3. **Main window** — menus, toolbar, splitter, category tree, table, status bar, column
persistence in `QSettings`.
4. **`widgets/ProgressDelegate`** — the in-cell progress bar; also
`SegmentBarsWidget` and `SpeedGraphWidget` (60 s rolling, 1 Hz, painted with
`QPainterPath`, no per-frame allocation).
5. **Dialogs**, in this order: Add URL → Download File Info (async probe, spinner while it
resolves) → Download Progress → Options (all tabs, every control bound to a
`settings.*` key that exists in the schema) → Scheduler → Speed Limiter → Batch →
Grabber wizard.
6. **Tray + floating drop target** — frameless always-on-top drop widget accepting dropped
URLs and text; position persisted.
7. **Clipboard**`clipboard/monitor.*`, but read `docs/06` R2 first: this is
best-effort under Wayland. Implement the **explicit** paths (Add-URL prefill, portal
global shortcut) as the primary UX and treat passive monitoring as a bonus that spike S2
may or may not unlock. **Don't advertise it in the UI until S2 answers.**
8. **Theming**`resources/qss/idm-like.qss` + `dark.qss`, colours in one variables block,
follow `QStyleHints::colorScheme()`.
## Definition of done (M1)
- Every screen in `docs/03-gui-spec.md` exists and is driven **only** by `mockd`.
- 10 000 synthetic rows: scrolling holds 60 fps, memory flat over 10 minutes of progress
events (`mockd --tasks 10000`).
- Unhappy paths handled: `mockd --slow`, `--flaky`, `--drop-connection` produce a banner
and a clean recovery, never a freeze or a crash.
- Zero download logic in `gui/` — grep for `curl`, `pwrite`, `sqlite` must return nothing.
- All strings wrapped in `tr()`; a stub Arabic `.ts` proves the RTL layout survives.
- No blocking call on the UI thread: verified with a 200 ms watchdog in debug builds.
## Icons
Ship an **original or compatibly-licensed** icon set (Papirus/Breeze-derived is fine) laid
out in IDM's positions. Do not copy IDM's artwork. Record the icon licence in
`gui/resources/icons/LICENSE`.
+62
View File
@@ -0,0 +1,62 @@
# Agent brief — PKG/QA (build, packaging, test infrastructure)
**Starts in M0 alongside PROTO. You unblock everyone else and you own the release.**
## You own
```
packaging/** .github/workflows/** tools/testserver/** tests/integration/** tests/e2e/**
CMakeLists.txt CMakePresets.json .clang-format .clang-tidy .editorconfig .gitignore
```
Do not write feature code in any lane's directory.
## M0 — unblock the lanes (do these first, in this order)
1. **Toolchain script**`tools/bootstrap.sh` installing the `apt` list in the README,
verified on a clean 26.04 VM. The dev machine already has git 2.53, CMake 4.2.3,
g++ 15.2, ninja, Qt 6 dev, libcurl, SQLite, nlohmann-json and ffmpeg; only
`libqt6svg6-dev`, `libsecret-1-dev`, `nodejs`/`npm` and (optionally) `clang` are
missing. The script must still install the full list for clean machines.
⚠ CMake 4.x rejects `cmake_minimum_required` below 3.5 — check every dependency.
2. **CMake** — top-level `CMakeLists.txt` + `CMakePresets.json` with presets:
`dev` (Debug + ASan/UBSan), `tsan`, `release` (RelWithDebInfo + LTO), `ci`.
Ninja, C++23, `-Wall -Wextra -Werror`.
3. **`tools/testserver`** — the hostile HTTP server every lane tests against. Modes, each
toggleable by URL path or flag:
`no-range` · `lies-about-accept-ranges` · `etag-changes` · `flaky-reset` ·
`slow-loris` · `redirect-chain` · `401-basic` · `401-digest` · `403-without-referer` ·
`416-always` · `content-length-mismatch` · `expiring-signed-url` · `throttled` ·
`chunked-no-length` · `utf8-content-disposition` · `legacy-content-disposition`.
**CORE's definition of done is written in terms of this server, so it must exist first.**
4. **CI** — GitHub Actions: build matrix (gcc + clang), unit tests, ASan/UBSan/TSan jobs,
`clang-format --dry-run -Werror`, `clang-tidy`, `web-ext lint`, and **conformance as a
required check on every PR**.
## M1M5 — keep it honest
- Nightly integration run: real `veloxd` + `testserver`, 50 concurrent downloads, assert
checksums and zero leaked FDs.
- `tests/e2e/` with Playwright: real Firefox + real daemon + a real file on disk.
- 72-hour soak job (M7 gate): 500 queued tasks, memory and FD graphs must be flat.
- `tools/bench` results published per commit so a performance regression is visible the day
it lands, not in M7.
## M6 — packaging
- **`.deb`** (primary): `veloxd`, `velox-gui`, `velox`, `velox-nmhost`, desktop entry,
systemd **user** units, icons, man pages, and native-messaging manifests installed to
`/usr/lib/mozilla/native-messaging-hosts/` plus a postinst that also drops the per-user
snap and flatpak copies where applicable. `lintian` clean. Publish via PPA.
- **Flatpak** (secondary): note that the sandbox changes the native-messaging story again —
test it, don't assume it.
- **AppImage** (optional): if you bundle Qt, bundle it as shared objects and honour LGPLv3
relink requirements (`docs/06` R5).
- **AMO**: signed XPI, reproducible build script, permission justification from EXT.
- **Uninstall test**: removes manifests, units, and sockets; leaves user data alone unless
purged.
- **Upgrade test**: install N-1, create tasks, upgrade, confirm the DB migrates and
in-flight `.veloxpart` files still resume.
## Definition of done
- One command builds everything from a clean checkout on a clean 26.04 VM.
- CI red on: format, tidy, sanitizer failure, conformance failure, `web-ext lint` failure.
- A fresh VM can install the `.deb`, install the extension, and complete the M2 vertical
slice with **snap Firefox** — no manual steps beyond clicking "Allow" once at pairing.
+53
View File
@@ -0,0 +1,53 @@
# Agent brief — PROTO (contract owner)
**Runs first, alone, in M0. Then stays on call as gatekeeper for the whole project.**
## You own
```
contracts/** tools/mockd/** tests/conformance/**
```
## You may read everything. You may write nowhere else.
## Mission
Make it impossible for the CORE, DAEMON, GUI and EXT lanes to become incompatible without
CI noticing the same day.
## M0 deliverables (in order)
1. **`contracts/schema/`** — JSON Schema (draft 2020-12) for every type, method and event
in `contracts/README.md`. Two templates already exist
(`types/TaskSummary.schema.json`, `methods/capture.offer.schema.json`) — follow their
shape, including the `x-privileged` / `x-transports` / `x-deadlineMs` annotations.
2. **`contracts/openrpc.json`** — generated from the schemas; it is the doc humans read.
3. **`contracts/codegen/gen_cpp.py`** → emits `core/generated/velox_proto.{hpp,cpp}`:
plain structs, `to_json`/`from_json` (nlohmann), a `Method` enum, and a
`dispatch(method, json) -> json` skeleton. No exceptions on the hot path; parse errors
return a `Result`.
4. **`contracts/codegen/gen_ts.py`** → emits `extension/src/shared/protocol/`: discriminated
union types, a typed `call<M>()` signature, event payload types, and runtime validators
for anything crossing the WS boundary (the daemon is not allowed to trust the wire, and
neither is the extension).
5. **`contracts/fixtures/`** — every method gets a success fixture; auth, timeout, and
not-found cases get error fixtures. Use `$uuid` / `$isoDate` placeholders for values
that can't be fixed.
6. **`tools/mockd`** — Node/TS. Serves the fixtures over **both** transports (Unix socket
NDJSON and loopback WS), fakes plausible progress events at 4 Hz, and has flags for
`--slow`, `--flaky`, `--drop-connection`, `--refuse-pairing` so GUI and EXT can test
their unhappy paths before `veloxd` exists.
7. **`tests/conformance/`** — one suite, two runners: replays each fixture against a live
`veloxd` (C++ side) and through the generated TS client. Wired into CI as a **required
check on every lane's PR**.
## Definition of done
- `mockd` answers all fixtures over both transports.
- Both generated clients round-trip every fixture with no hand-written types anywhere.
- Conformance is a required CI check.
- `VERSION` frozen at `1.0.0` and the freeze announced to all lanes.
## Standing duties after M0
- You are the **only** committer to `contracts/`. Other lanes file requests; you implement,
bump `VERSION`, regenerate, update fixtures, and notify the lanes in one PR.
- Reject "just add a field locally" every single time. That request is the M2 integration
disaster arriving early enough to stop.
- Optional field or new method → minor. Rename/remove/retype → major + an ADR.