1. download.add.json had startMode "now" against a real, large (~6 GB)
Ubuntu ISO with saveDir hardcoded to /home/sami/Downloads/Programs.
Against a real veloxd (tests/conformance/run.sh) that's a real
download into the real user's real home, every single run — it had
already happened twice. startMode -> "later" (exercises the add path,
hands nothing to the engine) and saveDir is dropped entirely (resolves
to saveTo.defaultDir instead, checked against allowedRoots the same
way). Documented the rule this fixture was breaking in
contracts/fixtures/README.md so it doesn't happen a third time.
Auditing the rest for the same shape (now real: capture.offer, D7)
found a second, subtler instance: capture.offer.take.json's "take"
admits a real, immediately-started task the same way download.add
does, and the "Programs" category's saveDir is a migration-seeded
builtin (~/Downloads/Programs) that no isolated test setup can
redirect -- so even after pointing the URL at example.org (RFC 2606),
a real ~6 GB sparse .veloxpart still landed in the real home on the
declared Content-Length alone. Shrunk to a plausible-but-small 5 MiB.
Also scoped to "transport": "uds" -- a real "take" persists an active
task, so replaying the same fixture again on the second live transport
against the same shared daemon was hitting capture.offer's own
dedupe-by-URL and failing on a missing taskId, not a bug.
download.add's other real-URL siblings (errors/*.invalid-path,
*.invalid-params, *.disk-full) all fail before admission or are
requires-gated; left alone.
2. ADR 0018: DAEMON can set a nullable field through download.update /
settings.set but never clear it back to null, because the generated
C++ parser collapses "absent" and "explicit null" to the same
std::nullopt for every optional field (contracts/codegen/gen_cpp.py,
on purpose, and correct for create-style params -- just wrong for
patch-style ones, which is the only place the schema documents
"explicit null clears"). Decision: an opt-in x-clearable schema
annotation makes just those fields std::optional<std::optional<T>> in
C++ (TS already round-trips this natively); not a blanket rule
(would retype response fields like TaskSummary.effectiveUrl that have
no clear-vs-absent distinction to make), not an explicit clear-list
field (would redesign a wire contract DAEMON already built against
just to route around a generator gap). Recorded, not implemented here
-- that's its own PROTO PR (schema annotations + gen_cpp.py + gen_ts.py
+ regeneration + a minor VERSION bump per ADR 0015), not bundled into
a fixture-safety pass. Left a pointer to the ADR at the generator
comment it concerns.
3. Re-verified every xfail entry against current deferrals.md rather
than trust the reasons already on file: D7/D8 (capture.offer/
getRules), D3d/e/f/g/h/i (rules, queue.reorder, schedule, limiter,
download.update/refreshUrl) and D9 (settings) have all closed since
the list was last pruned, so most of it was stale. Removed everything
that now cleanly passes; kept and re-reasoned everything that doesn't:
- errors/download.provideAuth.not-found.json stays, as asked: real
bug, on_download_provideAuth never checks the task exists.
- category.list.json (mimeTypes -- documented D3a gap), schedule.set.json
(nextRunAt -- documented D3f gap): unchanged in substance, reason
text was already accurate.
- download.probe/get/list/update.json, session.hello.json,
queue.start/reorder.json, category.remove.json: not bugs -- each
golden depicts a richer lifecycle/config state (a probed download,
real queue or category membership, media/grabber capabilities) than
this harness's fresh, never-started bound tasks and empty isolated
DB can produce.
- limiter.get.json: real fixture bug, not a daemon one -- applyToRunning
is a write-only instruction on limiter.set, on_limiter_get never
returns it; the golden shouldn't have had it either. Fixed the
fixture and tools/mockd's own limiter.get, which had the same field
hardcoded into its in-memory state independent of the fixture file.
- grabber.*/media.*: still genuinely stub (M4 territory).
Only remaining unexpected-pass surfaced while re-verifying
(errors/capture.offer.ignore.json, always "take" instead of "ignore")
traced to capture.minSizeBytes defaulting to 0 on a fresh daemon,
making its below-minimum-size scenario unreachable -- not a bug, so
raised the setting in run.sh's isolated seeding instead of xfailing it.
ctest -L conformance: green, 100% (2/2), ~87s.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01SFeUKLbdHizrJjLBeK7ffz
Per-task engine state, downloaded/effective_segments, every segment's own
state, and the engine's SegmentBudget snapshot -- printed once, when a task
times out, instead of needing to re-run the bench under a debugger or add
throwaway instrumentation to find out why. This is what actually diagnosed
the SegmentBudget over-admission bug fixed in the previous commit: the dump
showed budget.active pinned at max_active_segments while multiple tasks sat
starved, which is what pointed straight at confirm_slot()'s missing
engine-wide check rather than a per-task target bug.
Also updates the vdm_bench_load20 ctest registration's comment: the
straggler under --preset tsan that motivated running it at reduced
concurrency (docs/adr/0016's postscript) is now suspected to have been the
same SegmentBudget bug (docs/adr/0017), not the TSan-timing artifact first
guessed -- not reverified at the DoD's full shape under TSan in this change,
so the reduced-concurrency registration stays for now.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
For the M7 RSS gap (core/docs/m7-baseline.md: ~70 MB measured vs ADR 0012's
~45-50 MB estimate) -- "find where before tuning anything". No root here to
install massif or heaptrack, so this is a small in-process equivalent:
operator new/delete (already overridden for alloc-check's counter) now also
track, per call site (one return address via __builtin_return_address(0),
cheap enough to run for a whole scenario), live (not-yet-freed) bytes. Runs
the same 20-task concurrent scenario as `load`, waits for peak RSS to stop
growing, then prints the top sites by live bytes -- resolved via one batched
addr2line invocation against /proc/self/exe (dladdr alone only resolves
dynamic-symbol-table entries, which misses most of this codebase's
internal-linkage call sites), with dladdr as a per-site fallback.
Also adds the nothrow operator new/delete overloads alongside the existing
plain ones: without them, anything that allocates via the nothrow form (e.g.
std::stable_sort's std::get_temporary_buffer, hit once while investigating
the RSS gap) falls through to ASan's own default nothrow new while still
being freed through this file's plain delete override -- an
alloc-dealloc-mismatch ASan correctly flags as a real ABI-level bug in an
allocator override that claims to intercept "everything".
Using this tool, sum(effective_segments) across all 20 tasks vs.
Engine::segment_budget().budget() (now cross-checked and printed together)
showed the budget hands out well more than max_active_segments -- the actual
root cause, in core/src/segment/budget.cpp, not covered by this commit.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
Three subcommands in one binary, driving vdm::Engine directly (docs/04 §8):
- throughput: a single download against a fast local origin
(support/local_server.hpp, busybox httpd), reporting Mbps/CPU%/RSS. Gates
on --require-mbps/--max-cpu-pct only when passed, so the ctest smoke
registration stays a correctness check, not a hardware-dependent
perf gate -- the real 1-Gbit-link sign-off is a manual/CI job (see the
file's header comment).
- load: N concurrent tasks against tools/testserver's `throttled` mode
(support/testserver_client.hpp), reporting peak RSS via getrusage(). Paced
externally rather than through the engine's own rate::RateLimiter or
busybox: the limiter's pause/resume path allocates on every throttle event
(would contaminate alloc-check's measurement) and under heavy segment
contention was found to starve individual tasks indefinitely (see
docs/adr/0016, added here); busybox couldn't sustain the DoD's ~160
concurrent connections (20 tasks * default_segments=8) reliably. The
ctest registration runs at reduced concurrency under sanitizer presets --
see the CMakeLists.txt comment and the ADR's postscript.
- alloc-check: operator new/delete overridden process-wide, sampling the
allocation count across a steady mid-transfer window against a paced
tools/testserver origin. Caught a real bug in the same change (see the
http_client.cpp commit) and, by dropping its Engine mid-download to end
cleanly, also surfaced the quiesce() use-after-free (see that commit).
core/docs/m7-baseline.md records actual measured numbers against the M1/M7
DoD lines, including where they don't clear yet (RSS ~70 MB vs a 60 MB
target; throughput/CPU only measured on loopback, no 1 Gbit link available
here) rather than rounding them away.
docs/adr/0016 documents a rate::RateLimiter fairness gap found building the
load subcommand: a single shared TokenBucket under heavy segment contention
has no fairness ordering across its peek/commit race and can starve a
waiter well past what its configured rate implies. Filed as a follow-up
(it's a core/src/rate design question, not a tools/bench one) rather than
fixed here, along with a related TSan-only load-test straggler that could
not be root-caused in this environment.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
--check verified outcomes — binaries on PATH, pkg-config modules — on a box
that already installed everything. It never looked at the APT_* names, which
is the only part that breaks on a clean machine of the wrong release, as R1
just showed. Add a loop over the assembled PKGS array that fails on any name
with no installable candidate (apt-cache policy; no root, no network).
All-missing is treated as stale lists (warn), not 36 bad names.
The bootstrap-script job runs on a 24.04 runner, where the bad name still
resolves, so name validation there checks the wrong archive. Add
bootstrap-script-2604: a real --with-clang install in an ubuntu:26.04
container — the release the project ships on, and the first time the
fuzz-toolchain half of bootstrap is exercised anywhere (CORE had been running
clang++-21 directly). Also pass --with-clang/--packaging to the 24.04 --check
so the optional and M6 names can't rot unnoticed. BRANCH_PROTECTION.md gains
the 2604 row and the stale "when X merges" rows are corrected to "now".
gui/docs/pkg-qa-requests-m1.md R2 + the --with-clang note.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0143aKiohmDiyefJBwHDJJqw
`apt-cache policy libqt6svg6-dev` is "Candidate: (none)" on 26.04; the package
that carries the Svg headers and Qt6SvgConfig.cmake is qt6-svg-dev. Since
gui/CMakeLists.txt landed on main the root build's
find_package(Qt6 ... Svg REQUIRED) is live, so a clean 26.04 box could not
configure the project at all. Same name in README.md and AGENT-PKG-QA.md.
Reported in gui/docs/pkg-qa-requests-m1.md R1.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0143aKiohmDiyefJBwHDJJqw
util/crc32.hpp — header-only CRC-32 (zlib polynomial, reflected), used to
integrity-check the sidecar.
meta/veloxpart — the <name>.veloxpart.meta resume file (docs/04 §5).
Little-endian, versioned, CRC-32 over the whole record. Layout: magic,
version, flags, total_size, downloaded, url set (original/effective/
mirrors), etag/last-modified/content-type, segment records (start, end
INCLUSIVE, completed), optional sha256 streaming-hash blob.
parse_veloxpart() is the attacker-facing surface (the file sits in a
world-writable-ish download dir) and is total on any byte string: CRC
checked before any field is interpreted; magic, a version it understands,
every count and length bounded by a hard cap AND checked against the
remaining buffer; ByteReader latches on overrun; trailing bytes rejected.
Every malformation is meta_corrupt / meta_version_unsupported, never a
crash or an unbounded allocation. serialize_veloxpart() is deterministic
(unchanged sidecar isn't rewritten). File helpers write atomically
(temp + rename) and fdatasync the file and its directory.
Tests: crc32 known vector; full + minimal round-trips; deterministic
serialize; file round-trip; and a truncation/corruption table — bad
magic, CRC mismatch (payload and CRC-field flips), future version,
truncation at every stage, hostile url_count / segment_count / lp_string
length (the case the brief singles out), trailing bytes, impossible
segment.completed.
tools/fuzz/fuzz_veloxpart — feeds raw bytes and bytes-with-valid-CRC
(so the field parser and ByteReader bounds checks are actually reached),
and round-trip-stability-checks anything accepted. Ran 1.1M execs clean
under ASan+UBSan+libFuzzer (clang++-21); fuzz_content_disposition and
fuzz_url likewise re-run to 1.1M. tools/fuzz gains a -runs=0 seed-replay
CTest smoke per target (regression tripwire; the campaign stays manual).
Fuzz-found and fixed: parse_content_disposition could emit a filename
containing NUL / control bytes from a mangled filename* ext-value —
strip_path only removed path separators. Now sanitize_leaf() also drops
C0 controls and DEL (rules/ still owns the authoritative sanitize; `..`
and printable-unsafe content pass through as before).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
The last contract gap blocking an M1 definition-of-done item: CORE's "401
handled" has no return path without it, and B2a's sibling F2 was accepted in
proto-answers-m1.md but never actually landed.
download.provideAuth {taskId, username, password, save?} -> {ok}, exactly as
proposed there. Privileged and Unix-socket-only: a credential-bearing method
must never be reachable from the browser, which is the other half of the
promise event.auth.required's own description already makes ("never back
through this event, never into a log"). It answers the challenge; it does not
itself resume the task -- the daemon retries with the credential attached and
the ordinary event.task.state reports the task leaving retry_wait, the same
as any other state change.
save only tells the daemon whether to persist the credential in the Secret
Service for next time, or use it for this attempt alone -- it never touches
SQLite or a log either way, in keeping with CLAUDE.md's secrets rule.
Three fixtures: the success path, -32010 for a task that no longer exists
(credentials submitted for it are simply discarded), and -32003 confirming
the extension has no path to this method under any transport.
mockd gets a real handler rather than falling through to the generic fixture
responder: it validates the taskId exists (so the -32010 fixture is
replayable) and actually transitions the task out of retry_wait.
Minor bump, 1.1.0 -> 1.2.0: additive method, no existing type touched.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012fgjnqFCS5h5L7gZTZo3rV
GUI's M1 definition of done is "10 000 synthetic rows scroll at 60 fps with
flat memory over 10 minutes (mockd --tasks 10000)". This flag was missing from
the four unhappy-path flags that did land; the brief's own flag list omitted
it, which is corrected here too.
--tasks seeds a plausible population rather than N copies of one row: varied
state, size (log-uniform 50 KB - 20 GB), category, queue position and
description, drawn from the same category.list / queue.list fixtures the rest
of mockd already serves so a synthetic task can never name a category or
queue those methods don't also return. State distribution is roughly
55% complete / 8% failed / 4% cancelled / 6% paused / 2% retry_wait / 25%
queued, using the new TaskErrorCode taxonomy for failures.
"Progress advances across the whole set, not a handful of live rows" ruled
out the obvious cheap answer. A bounded, rotating pool of concurrently-active
downloads (--active-cap, default 24) is fed continuously from each queue's
FIFO — with the rest of that queue's queuePosition renumbered on every
promotion, as a real scheduler would — and a small fraction of active tasks
hit a transient failure and cycle through retry_wait before rejoining, so the
pool keeps rotating through new rows for the whole run instead of draining
once. Verified over a 10000-task, 60-second run: 61.5 MB RSS flat, and the
active pool's membership meaningfully different after 60s.
tick() only ever walks the active pool plus due retry-wait entries, never the
full task list, so its cost stays flat regardless of --tasks. A manual
download.add is still admitted immediately regardless of --active-cap — a
human driving the GUI by hand must never wait behind synthetic load.
Fixed a latent double-push while building this: any task 'connecting' at the
top of a tick was pushed to the progress batch once for the transition and
again at the loop's unconditional final push, inflating event.task.progress
payloads with a duplicate entry for that taskId. It predates this change (the
original tick() had the same shape) but only became visible once several
tasks are legitimately 'connecting' in the same tick, which --active-cap's
continuous promotion now does routinely.
--seed makes a run reproducible, which matters when a GUI bug only shows up
at a particular row.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012fgjnqFCS5h5L7gZTZo3rV
net/content_disposition — total parser for the mojibake-prone header:
RFC 6266 filename (quoted/token), RFC 5987 filename* ext-values
(charset'lang'pct-encoded, incl. RFC 2231 continuations), legacy RFC 2047
encoded-words (=?UTF-8?B?..?= / ?Q?), and raw Latin-1 bytes; prefers
filename* over filename; strips path components AFTER decoding (a base64
payload can hold '/'). 22-case test table.
net/text_codec (internal) — percent-decode, UTF-8 validation, Latin-1->
UTF-8, base64, RFC 2047 — shared by the CD parser and the URL splitter.
net/url — a small total URL splitter (scheme/userinfo/host/port/path/
query/fragment, http(s) validity) and url_filename() for the last path
segment; used for the filename fallback.
net/probe — HEAD then a ranged GET bytes=0-0 that PROVES resumability
(206 + matching Content-Range + a validator), rather than trusting
Accept-Ranges which servers lie about; the ranged GET is also the HEAD-
refused (403/405/501) fallback. 401/407 -> success result with
requires_auth, not an error. Runs on its own pool (max_concurrent,
default 4) outside the segment budget per ADR 0011 §5. suggest_filename()
does the resolution order (explicit -> disposition -> URL -> download.bin)
with a light strip; rules/ (stage 9) owns the authoritative sanitize.
tools/fuzz — libFuzzer targets for the CD parser and the URL splitter,
compiling the parser sources directly so they're fully instrumented;
self-guards on VELOX_BUILD_FUZZ + Clang (the top-level CMake adds every
tools/* unconditionally). Seed corpora included.
Fixed on the way: a p -> Transfer -> State -> cbs -> p reference cycle in
Prober that leaked every probe (drop the stored Transfer; the worker
keeps State alive). Tests green under ASan/UBSan and TSan.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
The conformance job was a required check that ran nothing. Two bugs:
1. Its guard tested for tests/conformance/CMakeLists.txt or package.json.
The suite ships as tests/conformance/run.sh; neither file exists, so the
guard was always false and the job took the "skipped" (success) branch.
2. Even forced true, it ran `ctest --preset dev -L conformance` — no test
carries that label, so ctest reported "Total Tests: 0" and exited 0.
Replace the job body with PROTO's intended wiring from
tests/conformance/README.md: bootstrap the toolchain, pin Node 22 (apt ships
< 20; the TS replay runner needs >= 20), and run ./tests/conformance/run.sh
directly. The suite starts its own mockd and builds its own C++ runner, so no
cmake configure is needed. Verified it goes red: an enum-invalid fixture makes
run.sh exit 1; reverting it returns to green.
check_contract.py imports jsonschema and referencing, which bootstrap.sh did
not install. Add python3-jsonschema / python3-referencing to the apt set and
to --check, so one command still provisions the whole suite.
Guards now fail loudly instead of passing quietly:
- conformance has no skip branch any more. run.sh has landed; the job runs
it unconditionally and errors if the entrypoint is missing.
- extension-lint keyed "has EXT landed?" to extension/package.json — the
same single-filename trap. Key it to a manifest instead, and once a
manifest exists, treat a missing package.json as a hard failure rather
than a green skip.
Mark conformance required now in BRANCH_PROTECTION.md — it is the M0 exit gate.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_0143aKiohmDiyefJBwHDJJqw
Schemas for the whole v1 surface: 38 methods, 9 events, 25 named types and the
JSON-RPC envelope, with x-privileged / x-transports / x-deadlineMs / x-errors
annotations that both generators emit as data rather than prose.
Four generators over one IR (contracts/codegen/schema_ir.py), so the C++ structs,
the TypeScript types and the OpenRPC document cannot disagree about what the
contract says:
gen_cpp.py -> core/generated/velox_proto.{hpp,cpp}
gen_ts.py -> extension/src/shared/protocol/
gen_openrpc.py -> contracts/openrpc.json
gen_cpp_conformance.py -> tests/conformance/cpp/fixture_dispatcher.hpp
Inbound parsing never throws: parse<T>() returns std::expected<T, ParseError> and
nlohmann's throwing ADL from_json is deliberately not emitted. Schema constraints
(minimum, maxLength, pattern, ...) become real runtime checks in both languages —
the daemon does not trust the extension and the extension does not trust the
daemon.
59 golden fixtures: a success case per method, 12 error cases, 9 events. Replayed
by tests/conformance/ against both the generated C++ and a live server over both
transports. tools/mockd serves the same fixtures with unhappy-path flags so the
GUI and EXT lanes never wait for veloxd.
run.sh also proves capture.offer fails open: with a daemon answering slower than
750 ms the client gives up and lets Firefox take the download.
core/generated/ is libveloxproto, a separate target from libveloxcore, which
still never sees JSON — see docs/adr/0009.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012fgjnqFCS5h5L7gZTZo3rV
Zero-dependency Python 3.11+ single file. 16 failure modes selectable by
URL path and combinable with '+': no-range, lies-about-accept-ranges,
etag-changes, flaky-reset (TCP RST mid-body, clean on the 3rd try),
slow-loris, redirect-chain, 401-basic, 401-digest, 403-without-referer,
416-always, content-length-mismatch, expiring-signed-url, throttled,
chunked-no-length, utf8/legacy content-disposition. Deterministic
synthetic bodies (byte i = f(seed, path, i)) with a /sha256/ reference
route so any range is independently verifiable. /__control {"reset":true}
clears flaky-mode counters between cases.
selftest.py exercises every mode (43 checks) and is registered as the
`testserver_selftest` CTest. README documents the full surface — CORE's
M1 DoD is written against it.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
Installs the full apt dependency set for a clean Ubuntu 26.04 machine
(build toolchain, Qt 6, libcurl/sqlite/openssl/secret, ffmpeg, node, lint
tools), then verifies versions: cmake >= 3.28, g++ with working
-std=c++23 <expected>, node >= 20, and every -dev package via pkg-config.
--with-clang adds LLVM for the M7 fuzz targets; --packaging adds .deb
tooling; --check verifies without installing. Flags the CMake 4.x
pre-3.5 cmake_minimum_required hazard from the brief.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
Lays out Velox Download Manager (IDM-class download manager for Ubuntu
26.04) as a monorepo ready for parallel lane development. No implementation
code by design.
- docs/: architecture, roadmap M0-M7, IDM-parity GUI spec, engine design,
Firefox extension spec, risks/spikes, packaging
- contracts/: wire-contract skeleton (JSON Schema + fixture templates) —
the single synchronization point between lanes
- docs/agents/: one brief per lane (PROTO, CORE, DAEMON, GUI, EXT, PKG/QA)
with owned directories, build order and definition of done
- CLAUDE.md: rules of engagement — lane ownership, layering, non-negotiables
- CMake scaffolding with dev/tsan/release/ci presets
Two environment findings shape the design: Firefox here is the Mozilla snap
(native-messaging risk, so the extension carries a loopback-WebSocket
fallback), and Wayland forbids passive clipboard monitoring (so clipboard
capture is explicit-action-first).
Co-Authored-By: Claude Opus 5 <[email protected]>
samiandClaude Sonnet 5