Per-task engine state, downloaded/effective_segments, every segment's own
state, and the engine's SegmentBudget snapshot -- printed once, when a task
times out, instead of needing to re-run the bench under a debugger or add
throwaway instrumentation to find out why. This is what actually diagnosed
the SegmentBudget over-admission bug fixed in the previous commit: the dump
showed budget.active pinned at max_active_segments while multiple tasks sat
starved, which is what pointed straight at confirm_slot()'s missing
engine-wide check rather than a per-task target bug.
Also updates the vdm_bench_load20 ctest registration's comment: the
straggler under --preset tsan that motivated running it at reduced
concurrency (docs/adr/0016's postscript) is now suspected to have been the
same SegmentBudget bug (docs/adr/0017), not the TSan-timing artifact first
guessed -- not reverified at the DoD's full shape under TSan in this change,
so the reduced-concurrency registration stays for now.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
Root cause of the M7 RSS gap (core/docs/m7-baseline.md: ~70 MB measured
against a 60 MB target): SegmentBudget::confirm_slot() only checked a task's
own held count against its own target -- never the engine-wide active_ sum.
reallocate_locked()'s two-pass fairness allocation does bound
sum(target) <= max_active_ at the moment it computes a plan, but that bound
says nothing about sum(held): a task can be legitimately holding more than
its own just-lowered target for a while (yield is deferred to a segment
boundary, never mid-segment -- ADR 0011 A1), and another task's target can
correctly rise to claim that capacity before the first task has physically
released it. Both confirm_slot() calls could then succeed against their own,
individually-correct targets while sum(held) exceeded max_active_ --
tools/bench heap-profile caught this directly: budget.active reading 56-86
against a total of 32.
confirm_slot() now also checks active_ < max_active_, unconditionally, as a
backstop that doesn't depend on any task's target bookkeeping being in sync
with what every other task holds. That creates a liveness question the
original design never answered: a task denied only by this new check has a
target that's already correct, so it never changes again and
reallocate_locked()'s plain "fire a callback when a task's target changes"
mechanism never revisits it. Task gained a waiting_for_slot flag, set on
exactly this denial; release_slot()/deregister_task() (the only two places
that free real capacity) now hand a freed slot directly to the
highest-priority waiting task via wake_one_waiter_locked(), if
reallocate_locked()'s own plan didn't already produce a callback for anyone.
download_task.cpp's fill_slots_locked() needed a matching fix: a woken
task's stalled segments (SegState::stalled -- backed off mid-retry, its own
release_slot() already called) have no live worker and never surface
through Segmenter::assign_slot(), which only hands out unassigned or fresh
ranges. fill_slots_locked() now restarts any stalled segment with no live
worker directly (bounded by slot_target, same as its assign_slot() loop)
before looking for new work; a segment it doesn't get to keeps its own
scheduled retry_worker() timer as a second chance.
Also fixes a real TSan-caught data race this work surfaced: SegWorker::
speed_bps was written only by its own segment's curl callback and, before
Progress.speed_bps's polled-path fix, only ever read from that same thread
-- safe without synchronization. snapshot_progress() reading it from
whatever thread calls DownloadHandle::progress() broke that invariant
(workers_mu's shared_lock protects the workers map's structure, not an
individual SegWorker's fields). Now std::atomic<double> with relaxed
ordering -- an informational EMA, nothing synchronizes real state on it --
rather than adding a lock to the write side.
core/tests/segment/budget_test.cpp adds two tests reproducing the actual
gap (budget_active_never_exceeds_max_active_segments_under_concurrent_load,
budget_wait_list_wakes_a_task_whose_target_never_changed) plus a sanity
baseline (budget_release_wakes_a_denied_waiter), and introduces AsyncFakeTask
+ TestTimer for the one existing test that drives the budget from multiple
concurrent threads -- mirroring production's real dispatch (register_task()'s
on_target lambda posts through host.schedule(), download_task.cpp, never a
synchronous call) rather than adding reentrancy-guarding machinery to
SegmentBudget itself to compensate for a synchronous test double being
unlike production. See docs/adr/0017 for the full writeup, including what an
earlier version of this fix got wrong chasing a same-thread reentrancy
hazard that doesn't actually exist in production.
core/docs/m7-baseline.md updated: the RSS number now clears the DoD line
(45.41 MiB via heap-profile), root-caused rather than just re-measured.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
For the M7 RSS gap (core/docs/m7-baseline.md: ~70 MB measured vs ADR 0012's
~45-50 MB estimate) -- "find where before tuning anything". No root here to
install massif or heaptrack, so this is a small in-process equivalent:
operator new/delete (already overridden for alloc-check's counter) now also
track, per call site (one return address via __builtin_return_address(0),
cheap enough to run for a whole scenario), live (not-yet-freed) bytes. Runs
the same 20-task concurrent scenario as `load`, waits for peak RSS to stop
growing, then prints the top sites by live bytes -- resolved via one batched
addr2line invocation against /proc/self/exe (dladdr alone only resolves
dynamic-symbol-table entries, which misses most of this codebase's
internal-linkage call sites), with dladdr as a per-site fallback.
Also adds the nothrow operator new/delete overloads alongside the existing
plain ones: without them, anything that allocates via the nothrow form (e.g.
std::stable_sort's std::get_temporary_buffer, hit once while investigating
the RSS gap) falls through to ASan's own default nothrow new while still
being freed through this file's plain delete override -- an
alloc-dealloc-mismatch ASan correctly flags as a real ABI-level bug in an
allocator override that claims to intercept "everything".
Using this tool, sum(effective_segments) across all 20 tasks vs.
Engine::segment_budget().budget() (now cross-checked and printed together)
showed the budget hands out well more than max_active_segments -- the actual
root cause, in core/src/segment/budget.cpp, not covered by this commit.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
DAEMON reported Progress.speed_bps reading 0 for the whole life of a live
throttled download while downloaded bytes visibly advanced. DAEMON reads
progress by polling DownloadHandle::progress() (engine_port_core.hpp), not
the on_progress push callback.
DownloadTaskState::snapshot_progress() -- the body behind progress() -- never
set speed_bps, per-segment speed_bps, or eta_seconds at all; only the
event-driven emit_progress_if_due() (which drives the on_progress callback)
computed them, from the same live SegWorker::speed_bps EMA seg_data()
maintains. snapshot_progress() now reads that same per-worker speed while
building its segment list, so a segment with no live worker (idle, paused,
complete, failed) correctly reports 0 and a segment with an active transfer
reports its real EMA, matching emit_progress_if_due()'s math including the
eta_seconds derivation.
engine_polled_progress_reports_nonzero_speed reproduces the bug (fails
without the fix, confirmed) by polling .progress() -- the same path DAEMON
uses -- during a throttled download and asserting speed_bps > 0 once real
progress has accumulated.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
Pure functions only, per AGENT-CORE.md's build order: no I/O, no JSON, no SQL,
no notion of the wire Rule type — DAEMON decodes its own stored/wire
representation into these plain structs and calls in.
- rules/filename.hpp: sanitize_filename() turns a raw candidate (from
net::parse_content_disposition or net::url_filename — neither is
filesystem-safe by design; both headers say so and point here) into one
safe to create on ext4/APFS/NTFS: strips separators and control bytes,
folds NTFS-illegal characters, neutralizes reserved Windows device names,
clamps length on a UTF-8 boundary. Total on hostile input; never empty.
Not the path-traversal security boundary — that's daemon/fs/safepath,
downstream of this and the one that actually matters adversarially.
- rules/collision.hpp: resolve_collision() finds the next free name
Explorer/Finder-style ("name (1).ext", ...) given an existence predicate,
or returns the desired name unchanged under an overwrite policy. Never
fabricates a guaranteed-unique name past its attempt bound — hands back
the last candidate tried rather than hiding a persistent collision.
- rules/match.hpp: match_rules() is the evaluation half of
contracts/schema/types/Rule.schema.json — priority order, first rule
whose present match clauses (extensions/mimeTypes/host & url glob/size
bounds) all hold, wins; a size clause never matches speculatively before
the probe fills in size_bytes. glob_match() is the iterative (not
recursive — bounded work on an all-'*' pattern) matcher both host_pattern
and url_pattern use.
Every header compiles standalone; tests (39 cases) pass under ASan+UBSan and
TSan. core/include/vdm/README.md documents the new public surface.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
Three subcommands in one binary, driving vdm::Engine directly (docs/04 §8):
- throughput: a single download against a fast local origin
(support/local_server.hpp, busybox httpd), reporting Mbps/CPU%/RSS. Gates
on --require-mbps/--max-cpu-pct only when passed, so the ctest smoke
registration stays a correctness check, not a hardware-dependent
perf gate -- the real 1-Gbit-link sign-off is a manual/CI job (see the
file's header comment).
- load: N concurrent tasks against tools/testserver's `throttled` mode
(support/testserver_client.hpp), reporting peak RSS via getrusage(). Paced
externally rather than through the engine's own rate::RateLimiter or
busybox: the limiter's pause/resume path allocates on every throttle event
(would contaminate alloc-check's measurement) and under heavy segment
contention was found to starve individual tasks indefinitely (see
docs/adr/0016, added here); busybox couldn't sustain the DoD's ~160
concurrent connections (20 tasks * default_segments=8) reliably. The
ctest registration runs at reduced concurrency under sanitizer presets --
see the CMakeLists.txt comment and the ADR's postscript.
- alloc-check: operator new/delete overridden process-wide, sampling the
allocation count across a steady mid-transfer window against a paced
tools/testserver origin. Caught a real bug in the same change (see the
http_client.cpp commit) and, by dropping its Engine mid-download to end
cleanly, also surfaced the quiesce() use-after-free (see that commit).
core/docs/m7-baseline.md records actual measured numbers against the M1/M7
DoD lines, including where they don't clear yet (RSS ~70 MB vs a 60 MB
target; throughput/CPU only measured on loopback, no 1 Gbit link available
here) rather than rounding them away.
docs/adr/0016 documents a rate::RateLimiter fairness gap found building the
load subcommand: a single shared TokenBucket under heavy segment contention
has no fairness ordering across its peek/commit race and can starve a
waiter well past what its configured rate implies. Filed as a follow-up
(it's a core/src/rate design question, not a tools/bench one) rather than
fixed here, along with a related TSan-only load-test straggler that could
not be root-caused in this environment.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
HttpClient::Impl::drain_commands() default-constructed a std::deque<Command>
every call, on every iteration of the worker thread's event loop (once per
curl_multi_poll wake -- i.e. once per socket-readiness event on the transfer
hot path), then swapped the (usually empty) command queue into it. In
libstdc++, an empty std::deque still allocates its map array on construction,
so this was a real allocation on the hot path regardless of whether any
command (add/pause/resume/cancel) was actually pending -- which is the
common case, since those are rare next to data arriving.
Found via tools/bench's alloc-check, which is built in this change and
exists specifically to catch this class of bug (AGENT-CORE.md: "no
allocation in the curl write callback... checked in review and by a bench
assertion"): before this fix it reported thousands of allocations/sec under
a sustained transfer; after, single digits.
Fixed by checking `w.queue.empty()` under the lock before touching `local`
at all, so the deque is only constructed when there's actually something to
swap into it.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
DownloadTaskState::quiesce() (Engine shutdown / ~Engine, via quiesce_task())
cancelled every live worker's transfer, then immediately cleared `workers` on
the calling thread. transfer.cancel() only *requests* the HttpClient worker
thread stop the transfer -- it does not wait for that to happen. If that
thread was mid write-callback (seg_data -> WriteBuffer::append ->
SparseFile::write_at), clearing the map destroyed the SegWorker (and its
ring buffer) it was still writing through: a heap-use-after-free, caught by
ASan via tools/bench alloc-check, which by design drops its Engine while a
download is still active mid-sample.
Every other exit path (verify/fail/auto_pause/demote, via begin_drain_locked)
already gets this right: cancel, then let each worker remove and flush
itself through seg_finished once HttpClient actually confirms the transfer
stopped, on the correct thread. quiesce() now does the same instead of
tearing the map down itself -- wait on a condition variable, notified from
seg_finished right after it erases, until `workers` is empty.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
Resumes work left mid-session on the M1 hostile-mode matrix. download_task.cpp:
- Replace the old cancel-then-clear teardown (cancel_all_transfers_locked /
start_assembly_locked) with a single begin_drain_locked()/PendingAction
mechanism: cancel every live worker, remember what to do (verify / fail /
auto_pause / demote), and let whichever worker's seg_finished finds the
worker map empty carry it out. Every sibling still flushes its buffer on
the way out, so no buffered-but-unflushed tail is lost when a download
finishes or fails while other segments are still mid-transfer.
- A 200 where 206 was expected (wrong_status) now checks the response's
ETag/Last-Modified against the probe's: a real mismatch asks the user
(server_file_changed, "ask, never silently corrupt" -- docs/04 §5); a match
means the server just stopped honouring Range for this connection, so
demote to one segment and keep going without a round trip (docs/04 §7).
- 416 mid-download (stale range metadata) now surfaces as a decision instead
of retrying the same now-invalid range to exhaustion.
- do_decide's abort path surfaces the actual reason a decision was asked
for (last_error) instead of hardcoding server_file_changed, which was
mislabeling a 416 abort.
engine_test.cpp adds the four hostile modes where a bug means silent
corruption rather than a visible failure: etag-changes, 416-always,
lies-about-accept-ranges, content-length-mismatch.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
Pointed the existing RPC client at a real veloxd instance (isolated
HOME/XDG_RUNTIME_DIR/XDG_DATA_HOME — never touched the real ~/Downloads),
added a real download.add against tools/testserver, and watched it render
live end to end with no GUI code changes:
- handshake, subscribe, category.list/queue.list all match what DAEMON
reported
- the full event.task.state sequence and batched event.task.progress
both applied correctly by DownloadTableModel
- the written file's SHA-256 matches the server's reference
Filed gui/docs/daemon-requests-m1.md: TaskSummary.sizeBytes is never
populated by this daemon build, even in download.get after the task
completes with the exact byte count already on disk. Not a GUI bug —
ProgressDelegate and the model already do the documented right thing
when size is unknown (fall back to plain text, no bar) — but it means
every task renders without a percentage against the real daemon today.
mockd always supplies sizeBytes so this doesn't block current GUI work;
flagging before the M1 GUI<->daemon integration pass.
mockd stays the primary harness for the unhappy paths
(--slow/--flaky/--drop-connection) that a real daemon won't misbehave on
command for.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016Ne28kx4VreeBWZv82Nksd
The two items aimed at pointing GUI at a real veloxd instead of mockd.
rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.
sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
the store's own prior row for previousState (authoritative
regardless of engine/scheduler timing), writes the error columns,
and — when a hub is supplied — publishes event.task.state with
{taskId, state, previousState, summary, error}. Wired into every
transition: scheduler-driven (admission -> probing, resume ->
connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
(EnginePort::progress(), a new interface method backed by
DownloadHandle::progress()), plus a store side-effect
(Tasks::update_progress) so download.list/get stay current between
state transitions. Returns rows; does NOT publish itself — batching
into one array message is the caller's job, per the schema's
x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
event.task.progress per tick, only when there's something to say.
dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").
store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).
Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.
Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.
deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
CORE stage 8 merged, so vdm::Engine is linkable. This closes D4a and
narrows D4b: `velox add <url>` now actually downloads.
- sched/engine_port_core.hpp — the real EnginePort: forwards to a live
vdm::Engine, keeps the DownloadHandle per task for pause/resume/
cancel/provide_auth/decide/refresh_url, drives set_task_order /
set_max_active_segments / set_host_segment_cap via
engine.segment_budget(). CORE confirmed the admission model: DAEMON
decides when to start(); the engine's own download_task calls
register_task/set_want internally — DAEMON never touches per-task
budget calls. EnginePort gains release(TaskId) so the port drops a
handle when the task goes terminal.
- rpc/event_loop — EventLoop::post(fn): thread-safe, runs fn on the
loop thread next iteration. The marshaller for engine-thread
callbacks.
- main.cpp — constructs vdm::Engine + EnginePortCore + Scheduler
(post_to_loop = loop.post). At startup: reconcile_after_restart()
(ADR 0013 §5), reload_config(), tick(). A 1 s timerfd on the loop
re-runs tick() (schedule windows, missed nudges); download.add nudges
via dispatcher.set_on_mutation.
End-to-end verified against tools/testserver: `velox add
http://127.0.0.1:.../file/512K` -> task queued -> scheduler admits ->
engine downloads 524288 bytes -> complete, file on disk. First
byte-path all the way through the project.
safepath-adversarial.md: re-verified per its own note — CORE landed
O_NOFOLLOW on the target open (core/src/io/sparse_file.cpp), so the
leaf-symlink TOCTOU is now closed; residual is down to one
intermediate-dir gap (documented post-M1 chase).
36 daemon/cli tests green; scheduler + uds_roundtrip TSan-clean.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
The Scheduler that D4 was waiting on. Built against CORE's engine
HEADERS (now in main); the real EnginePort and the veloxd wiring wait
for lane/core's stage-8 bodies to reach main (deferrals.md D4a/D4b) —
core/src/task/ is still .gitkeep there, so linking vdm::Engine now
would be an unresolved symbol.
- sched/engine_port — the abstract seam: start/pause/resume/cancel/
provide_auth/decide/refresh_url + the ADR 0011 admission config
(set_task_order / set_max_active_segments / set_host_segment_cap).
Keeps the Scheduler testable without a live engine and the daemon
unbound from the concrete vdm::Engine.
- sched/fake_engine_port — a recording impl for tests.
- sched/scheduler:
* owns the wire-UUID <-> vdm::TaskId map.
* tick(): snapshot queues (schedule window evaluated with an
injectable clock) + non-terminal tasks -> governor.evaluate ->
apply. to_start builds a vdm::task::DownloadSpec from the row and
calls EnginePort::start; to_resume -> resume(); to_pause ->
pause() + writes the pause_reason; priority_order -> set_task_order
over the mapped engine ids. `new` tasks are parked (startMode
manual) and skipped.
* on_engine_state(wire_id, state, err): projects an engine
transition onto the store row (state, pause_reason='auto' when an
error rides a paused transition per ADR 0013 §2, flattened error
columns) so the next tick sees ground truth. This is also the hook
event.task.state will fire from (D5).
* reconcile_after_restart(): CORE-owned states -> queued, paused
keeps its reason (ADR 0013 §5).
* reload_config(): reads connection.maxConcurrentDownloads /
maxActiveSegments + a daemon-local host-cap map, pushes caps to
the engine, updates the governor.
* Deps: injectable local-now clock and a post_to_loop marshaller
(engine callbacks arrive on engine threads; default runs inline
for tests).
Test veloxd.sched_scheduler (ASan+UBSan and TSan clean): admission +
ordering, a slot freeing on completion, queue-stop -> pause
(queue_stopped) then queue-restart -> resume (not a fresh start),
engine auto-pause -> pause_reason 'auto' + never auto-resumed,
reconcile_after_restart, reload_config caps push. 35 daemon/cli tests
green.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
Stage 8 of the CORE build order: the bodies behind the DownloadSpec /
callback API reviewed in core/docs/engine-api-m1.md. Wires probe -> segment
workers -> WriteBuffer -> SparseFile -> .veloxpart.meta -> retry/backoff ->
SegmentBudget -> RateLimiter -> callbacks into one event-driven machine.
- Engine (src/engine.cpp): owns HttpClient, Prober, SegmentBudget,
RateLimiter and one timer jthread (min-heap of scheduled fns). start()
builds a task and returns a DownloadHandle; ~Impl quiesces every task
before joining the timer so no callback fires during teardown.
- DownloadTaskState (src/task/download_task.cpp): one `mu` task lock; a
shared_mutex over the worker map for the curl write path; callbacks
collected under `mu` and fired after release via a separate deferred
queue; weak_from_this() in every async hop. State machine over the
CORE-owned EngineState subset, auto-pause on 401/407 and on a 200 where
206 was expected, validated resume via If-Range.
- digest (src/task/digest.cpp): OpenSSL EVP hash_file() for the optional
post-download checksum; links OpenSSL::Crypto PRIVATE.
- Segmenter::release_segment(): hand a paused segment back to the pool
unassigned so resume's assign_slot() picks it up instead of splitting a
still-"assigned" range and orphaning its front half.
- DownloadHandle now names the real control block (vdm::task::
DownloadTaskState, defined only in the engine TU) via a namespace-scope
fwd decl and a public-but-effectively-engine-only ctor, replacing the
nested State/friend pair. Every public signature is unchanged; DAEMON
(vdm-79) confirmed sched/ names only the public API.
Fixes found while building the end-to-end suite (tests/task/engine_test.cpp,
9 cases against tools/testserver, green under ASan/UBSan and TSan):
- a dropped connection lost its unflushed WriteBuffer tail while advance()
had already counted those bytes as done -> a retry resumed past an
unwritten hole. Flush on the failure path.
- when the byte counters hit total while other workers were still live,
teardown dropped their buffered tails. Now: cancel them and let each
worker's own seg_finished drain it (the `assembling` state), last one
starts verification -- no cross-thread buffer access.
- seg_head() let a 401 with credentials present abort before libcurl's
resend; now it proceeds once and acts on the final status.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
run_all() takes an optional substring list; vtest_main forwards argv and a
comma-separated $VT_ONLY. No filter => run everything, as before. Makes
iterating on one slow end-to-end case (the engine suite) practical
without a framework swap.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
libcurl with CURLAUTH_ANY answers a 401/407 by resending the request with
an Authorization header. Two spots in net/ cut that short:
- http_client's header callback delivered the response head exactly once
and latched `head_delivered`, so after an auth challenge the caller only
ever saw the 401 — never the 2xx of the authenticated resend. Reset the
latch when a fresh status line follows a delivered 401/407 (redirects
never reach that path — their head is suppressed).
- the prober's head callbacks return DataAction::abort to skip the body,
which also aborts the transfer mid-handshake. Return `proceed` for a
401/407 when credentials were supplied, so curl's resend can run; the
real status lands on the next header block.
Also give ProbeRequest an `auth` field (default scheme == none) and pass
it through base_request(), so a re-probe after a 401 can present the
credentials the user just entered. No behaviour change when no auth is
configured.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
VeloxDispatcher now takes a store::Db& and three handlers are real:
- download.list -> store::Tasks::list (filter / sort / paging in SQL) ->
to_summary per row. No more empty-table stub.
- download.add -> resolve saveDir (spec, else saveTo.defaultDir; ~ expanded)
and the leaf (spec.filename, else the URL's last segment percent-decoded,
else download.bin) -> fs::resolve_target against canonicalize_root'd
saveTo.allowedRoots. Any path-destination failure is -32011 with the
*original* saveDir in data.path. On success a TaskRow is inserted in
state `queued` (or `new` for startMode "manual") and {taskId, state}
returned. The scheduler that would then admit it is D4.
- download.get -> store::Tasks::get; a real -32010 + data.taskId for an
unknown id, else a TaskDetail (segmentDetail empty until the engine
segments the task, which the schema permits).
util/time.hpp: now_iso() factored out of ws_server.cpp.
main.cpp constructs the dispatcher with the opened db. The three
integration tests build an in-memory migrated db for it; velox.client
now drives the full slice through the CLI — add outside roots -> -32011
with data.path, add into an allowed root -> a task that download.list
shows and download.get details, unknown id -> -32010. Verified with the
real binaries: velox add persists, velox ls shows it, it survives a
daemon restart, /etc is refused.
ASan+UBSan and TSan clean; 34 daemon/cli tests green. deferrals.md:
D2 down to just download.probe; D3 down to categories/queues/rules/
settings/limiter/schedule.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
DAEMON's safepath-adversarial.md accepts a TOCTOU residual between its
canonicalise-and-check and the download starting, on the stated grounds
that CORE's O_NOFOLLOW open of the final file closes it. That flag was
never actually set: SparseFile::open used O_WRONLY|O_CREAT|O_CLOEXEC, so
a symlink swapped in as the final path component after DAEMON's check
would be followed and redirect our pwrites outside the allowed roots.
Add O_NOFOLLOW. A symlinked leaf now fails the open with ELOOP, which
errno_to_error already maps to Error::path_rejected. Regular files and
the O_CREAT of a fresh part file are unaffected; resume (existing regular
part file) is unaffected. Test that a symlinked destination is rejected
rather than silently followed, and that the link target is never touched.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
The residual section claimed the leaf/component TOCTOU is "closed in
practice by CORE's O_NOFOLLOW open of the final file". Verified: it is
not — core/src/io/sparse_file.cpp:77 opens O_WRONLY|O_CREAT|O_CLOEXEC,
no O_NOFOLLOW, no O_EXCL. Requested the flags from CORE via PKG/QA.
Doc now states the residual is currently OPEN, names the file:line and
flags checked and the date, says what actually limits exposure today
(0700 parent dirs), and flags this as the boundary where a reader
stops checking. Step 5 reworded the same way. Re-verify the flags when
the CORE change lands.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
The read/write surface the dispatcher handlers need, so download.add
persists and download.list / download.get project real rows when the
engine lands.
- store/tasks — TaskRow (1:1 with the schema), insert / get / remove /
set_state / count, and list(filter, sort, offset, limit) that does
all the WHERE / ORDER BY / LIMIT in SQL (M1 DoD: a 1000-row list
never materialised client-side). Filter covers states / category /
queue / case-insensitive filename+url substring / date range; sort
is a whitelisted column + direction with NULLs last, default
newest-first; the enum spellings in a state IN (...) come from
proto::to_string, never from user text. to_summary() projects a row
onto proto::TaskSummary including the flattened error block when the
task failed / retry_wait / auto-paused.
- store/settings — key -> JSON-text with a built-in default table
mirroring Settings.schema.json / ADR 0012; get_raw / set_raw /
overrides plus typed get_int / get_string / get_string_array for the
governor config and saveTo.allowedRoots. Full settings.get/set wire
projection lands with those handlers.
- veloxd_store now links velox::proto + nlohmann_json for the
projection.
Test veloxd.store_tasks: insert/get round trip, PK duplicate rejected,
the error-block projection, list total+paging+sort+every filter,
set_state pause_reason clear-on-unpause, remove, and settings default
vs override. ASan+UBSan and TSan clean; 34 daemon/cli tests green.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
veloxd is the one process that turns an untrusted string into a
filesystem destination, and via capture.offer that string can come
from a web page. CLAUDE.md §4 and the M1 DoD both name this.
daemon/docs/safepath-adversarial.md is the spec, written before the
code the way EXT did for shouldCapture: 21 rows — .. traversal
(A1/A2), absolute-outside-roots (A3), prefix-match confusion (A4),
symlink-out (A7), TOCTOU on a created tail (A8), NUL/control bytes in
the leaf that CORE's fuzzer hit through Content-Disposition (A9/A10),
degenerate and overlong leaves (A11/A13), overlong dir component
(A14), symlinked root (A16), destination-is-a-file (A17), and the
legitimate cases that must still pass — non-ASCII (A18), redundant "."
(A19), trailing space/dot trimming (A20).
fs/safepath.cpp:
- sanitize_leaf: strip <0x20 and 0x7F, trim ws, strip trailing dots,
reject ""/"."/".."/contains-'/', cap 255 UTF-8 bytes on a codepoint
boundary. Mirrors core/src/net/content_disposition.cpp.
- canonicalize_root: expand ~ and realpath each allowedRoots entry
once, so a symlinked root resolves to its target.
- resolve_target: reject relative saveDir and any ".." component
lexically; if the dir exists, realpath + component-wise containment
(a symlink that escapes is caught, one that stays inside passes); if
a tail is missing, realpath+check the deepest existing ancestor then
create the tail via an openat/mkdirat O_NOFOLLOW walk and re-derive
the final path from the fd. Every failure is -32011 with data.path =
the *original* saveDir (never the resolved path). Residual TOCTOU on
a pre-existing intermediate dir is documented and closed by CORE's
O_NOFOLLOW open of the file.
veloxd_fs static lib; veloxd_rpc links it for the download.add wiring
next. Test veloxd.safepath is the adversarial table, on a real temp
tree. ASan+UBSan and TSan clean; 33 daemon/cli tests green.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
The scheduling brain, built against CORE's headers (vdm/engine.hpp,
vdm/segment/budget.hpp) — signatures only; the Engine bodies land in
CORE stage 8 and the Scheduler that wires governor <-> store <-> engine
<-> timer comes after that (daemon/docs/deferrals.md D4).
- sched/governor — a pure decision function. In: a snapshot of every
task's coarse RunState and every queue's state (schedule windows
pre-resolved). Out: {to_start, to_resume, to_pause, pause_reasons,
priority_order}. Enforces, all in TASK units per ADR 0011 §1:
connection.maxConcurrentDownloads; the min(that, maxActiveSegments)
clamp (§2); Queue.maxConcurrent; the per-host task cap (§4); and a
stopped queue / closed window runs nothing. Never touches a task
paused for `user` or CORE's `auto` (ADR 0013 §3) — only Schedule /
QueueStopped / AdmissionReconcile are auto-resumable. Deterministic:
main-list before queued-in-queue, then queue order, then FIFO, then
task_id.
- sched/schedule_window — window_open(Schedule, local tm): disabled =>
always open; `once` => date + time match; `periodic` => weekday in
daysOfWeek (empty = every day) + time in [start, stop); null start =>
midnight, null stop => end of day, stop < start => overnight window.
Pure; re-evaluated every tick, no cached instants.
- veloxd_sched static lib; veloxd links it (nothing calls it yet).
Tests (ASan+UBSan and TSan clean): veloxd.sched_window (10 window
cases incl. overnight, once, null bounds), veloxd.sched_governor
(global/clamp/per-queue/per-host caps, stop vs window pause reasons,
resume-only-governor-reasons, auth-pause untouched, admission
reconcile, determinism under shuffled input). 32 daemon/cli tests
green; full tree green.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
CORE resolved all four (lane/core@3da4cd6): vdm::TaskId is hashable and
DAEMON never constructs one; DAEMON mkdir -p's save_path's parent
(engine -> Error::path_rejected if missing); sha512 added as the 4th
Checksum::Algo so no -32602 at the RPC edge; on_state(cancelled) then
on_finished(Err{Error::canceled}) -- note the one-L spelling in the
error taxonomy. Also notes rate/token_bucket + Engine::rate_limiter()
for the limiter.set wiring.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
rate/token_bucket.hpp — a lazily-refilled TokenBucket (starts full: burst
then throttle, IDM behaviour; rate 0 == unlimited; burst caps idle
accumulation) and RateLimiter, the global -> per-queue -> per-task
hierarchy (docs/04 §6). acquire(task, n) peeks every applicable level and
commits on all-or-none so a blocked attempt never leaks tokens at a level
that had them; held under one mutex so a concurrent detach can't dangle
the bucket it's using. vdm/ids.hpp gains QueueId.
Tests: burst/refill/cap/unlimited for the bucket; tightest-level-binds,
no-partial-consumption, detach-safety, and an 8-thread aggregate-rate
check for the hierarchy. Green under ASan/UBSan and TSan.
Engine-API review (DAEMON signed off, no sched/ or dispatch rewrite):
- Engine::rate_limiter() accessor added (limiter.set -> set_global_limit).
- Checksum::Algo gains sha512 to match the wire Checksum set.
- DownloadSpec: DAEMON creates save_path's parent dir before start();
missing dir -> Error::path_rejected (made explicit).
- cancel(): documented to fire on_state(_, cancelled, nullopt) then
on_finished(Err{canceled}), in that order; download.cancel ==
cancel(false), download.remove == cancel(true).
- engine-api-m1.md: the five open questions resolved with DAEMON's
answers (probe_hint optional, single cancel flag, {restart,
keep_partial, abort} is the whole set, per-task 4 Hz is fine,
refresh_url restarts all segments after a validating re-probe).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
Review of core/docs/engine-api-m1.md (lane/core@d6cf1fe) in
daemon/docs/engine-api-review.md. Sign off: nothing forces a sched/ or
RPC-dispatch rewrite; the split matches ADR 0011 and ADR 0013.
Answers: (1) keep probe_hint optional — DAEMON has a ProbeResult only
on the File Info path; (2) one cancel(discard_partial), download.remove
is cancel(true) + DAEMON-side row/file cleanup; (3) {restart,
keep_partial, abort} suffices if the engine owns the mechanical 416
re-probe/re-split; (4) per-task 4 Hz progress is fine — DAEMON
re-batches across tasks for event.task.progress anyway; (5) refresh_url
restarts all segments on the new URL (the signed-URL case), mirror
rotation is spec.mirrors not refresh_url.
Four things to confirm, none blocking: vdm::TaskId copy/hash semantics
and that DAEMON never constructs one; who mkdir -p's save_path's
parent; sha512 (in the wire Checksum, not the engine enum) rejected at
the RPC edge; on_finished(Err{cancelled}) code + ordering vs
on_state(_, cancelled, _).
Integration timing: wire after sched/ lands. sched/ builds against
these signatures in parallel with CORE stage 8.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
The extension's fallback transport (docs/05 §4). veloxd now also listens
on 127.0.0.1, first free port in 52000-52016, and writes it to
<runtime>/ws.port (0600).
- rpc/ws_frame — RFC 6455 frame codec. Incremental; reassembles
continuation frames; enforces "client frames MUST be masked" (§5.1);
caps a reassembled message at 8 MiB. This is the attacker-adjacent
parser, so it has its own test table.
- rpc/ws_handshake — HTTP upgrade parse, Sec-WebSocket-Accept
(SHA-1 + base64 via libcrypto), and the two non-negotiable checks:
an Origin header must be present and must be moz-extension:// (a page
cannot pair). Version must be 13.
- rpc/ws_server — per-connection Handshake -> Open state machine on the
shared EventLoop. Token gate: session.pair mints a token behind the
approver + rate limiter; session.hello must present a valid one;
every other method is -32002 until authed. Privileged methods are
refused -32003 by the generated dispatch(). Ping -> Pong; Close
echoed. session.hello major-version mismatch -> -32001.
- rpc/pairing — PairingApprover interface + EnvAutoApprover dev stub
(approves iff VELOX_PAIR_AUTO=1); PairingRateLimiter (5 failures / 60 s
per origin, then 60 s lockout -> -32014, survives reconnect); a
four-digit code generator.
- store/pairings — the pairings table: create() returns the plaintext
token once and stores only its SHA-256; find_active_by_token,
touch, revoke, list_active.
- util/crypto — sha1 / sha256_hex / base64 / random_token over libcrypto.
- store/sqlite — pin the DB file (and -wal/-shm) to 0600.
- runtime_dir — resolve_data_dir() for $XDG_DATA_HOME/velox (velox.db).
- main.cpp — opens + migrates velox.db, starts both transports; a WS
bind failure is logged, not fatal (capture must fail open, the Unix
socket still serves the GUI/CLI).
Real gap, flagged not hidden: the pairing prompt is EnvAutoApprover for
now — a GUI dialog / desktop notification is build step 7. Pairing
needs VELOX_PAIR_AUTO=1 until then.
Tests (ASan+UBSan and TSan clean): veloxd.ws_frame (codec + handshake
vectors incl. the RFC 6455 §1.3 accept sample), veloxd.pairings (token
create/find/revoke, hash-not-token, rate-limit window + lockout +
per-origin isolation + success reset), veloxd.ws_server (full flow: 101
handshake, -32002 gate, deny-then-approve pairing, hello-with-token,
-32003 privileged refusal, real download.list). 27 daemon/cli tests
green; full tree green.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
docs/05 §3: "Download with Velox" on a link (linkUrl) or a media element
(srcUrl) hands that one URL to download.add with the page as referrer; a
configurable command (Ctrl+Shift+U) does the same for the active tab. These
are explicit user requests, so they skip shouldCapture and the fail-open
path — a failure is surfaced to the user via notifications instead.
The page/selection "Download all links…" item waits on the content-script
link harvester (build-order step 7) and will be added there.
manifest: commands.velox-grab-current-tab. 9 tests. 110 total, lint clean.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Y9RU58hD1BuwP82DySUHk
- CategoryPanel: the left tree — All Downloads / Unfinished / Finished,
then Categories and Queues populated from category.list / queue.list,
with per-node task counts. Selecting a node emits a TaskSelection.
- DownloadFilterProxy: QSortFilterProxyModel keyed off that selection.
Client-side for M1 (the whole list fits); asTaskFilter() exposes the
equivalent TaskFilter for a server-side download.list once paging lands.
- MainWindow: menu bar (Tasks / Downloads / View / Help) sharing QAction
objects with the toolbar; QSplitter [panel | table]; Delete with a
confirm; Resume/Pause All; View menu toggles the panel. Actions
disabled while offline.
- Counts are computed off a throttled 400 ms timer, not the 4 Hz progress
path. Fixed a debounce-vs-throttle bug found in the first screenshot:
restarting the timer on every progress tick meant it never fired and
the status bar sat at "0 of 0 downloads".
- First-run column widths that fit the content.
- tst_downloadfilterproxy: nodes filter to their own rows, the
Finished/Unfinished split is correct, and the filter is dynamic (a row
that finishes leaves the Unfinished node with no re-list). Verified
against mockd --tasks 400 (screenshot: tree counts 75/84/89/83/69 sum
to 400, status bar "400 of 400, 21 active").
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016Ne28kx4VreeBWZv82Nksd
Downloads that never reach the header hook (form POST results, service-worker
responses, clicks Firefox routes straight to its downloader) surface here.
If one looks like the daemon's, offer it FIRST and only cancel + erase
Firefox's copy on {action:"take"} — a failed or slow offer can never leave
the user with nothing. blob:/data: downloads are left to Firefox (the daemon
can't fetch a blob URL).
- offered-urls.ts: short-lived, bounded TTL set of URLs the header hook has
already offered; the safety net checks it (via wasOffered) so nothing is
double-handled. Hook gains an onOffered hook to populate it.
- background/index.ts: both paths share one offer(), getCookies, rules
mirror, and OfferedUrls instance.
13 new tests incl. fail-open (offer rejects -> 'error', ignore ->
'offer_declined', cancel() throwing after take still returns 'taken').
101 tests green; web-ext lint clean.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Y9RU58hD1BuwP82DySUHk
The download entry point the AGENT-CORE brief asked for on day one and
that slipped. DAEMON has an RPC surface and a store and, until this is
agreed, nothing in velox::core to call.
vdm/task/download.hpp — DownloadSpec (the resolved subset DAEMON hands in:
absolute save_path, verbatim browser headers, requested segments/buffer,
optional probe_hint / checksum / auth, allow_resume), EngineState (the
CORE-owned subset of the wire TaskState), Progress / SegmentProgress,
DownloadCallbacks (on_progress <=4 Hz, on_state for every transition
incl. auto-pauses, on_auth_required, on_decision_needed, on_finished
last), DownloadHandle (pause/resume/cancel — idempotent per the ADR 0013
signature — plus provide_auth / decide / refresh_url, and synchronous
state()/progress() snapshots).
vdm/engine.hpp — Engine: start(spec, callbacks) -> handle, segment_budget()
(DAEMON's sched/ admission surface, ADR 0011), live connection.* setters,
a standalone probe() on the pool outside the segment budget.
core/docs/engine-api-m1.md — the review doc: field semantics, the state
machine, threading/lifetime rules (which thread callbacks arrive on, what
is legal from inside one, handle/engine lifetime), the shared-`paused`
idempotency contract as a signature, and five open questions for DAEMON.
Value types compile and are covered by api_compiles_test; Engine /
DownloadHandle bodies land in stage 8, built against whatever DAEMON
signs off here.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
For a response shouldCapture() likes: gather cookies, offer to the daemon
under a hard 750 ms budget, and return {cancel:true} ONLY on an explicit
{action:"take"}. Every other outcome — shouldCapture says no, daemon down /
slow / erroring, cookies fail, anything throws — resolves to {} and Firefox
downloads normally.
Fail-open tests written first (tests/capture/hook.test.ts): offer() rejects,
offer() never settles (resolves within budget), timeout-shaped rejection,
getRules() throws, malformed details. Plus the happy paths and a check that
the capture.offer payload is well-formed from details + stash + headers.
background/index.ts: wire the stash + hook onto browser.webRequest, mirror
capture rules via capture.getRules on connect and on a capture.* settings
change; DEFAULT_CAPTURE_RULES (enabled:false) until the first mirror lands.
84 tests green; web-ext lint clean.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Y9RU58hD1BuwP82DySUHk
Six lanes pick ADR numbers with no allocator. This lane dodged one collision
by taking 0012 while DAEMON drafted 0011, and just hit a real one — two 0014s
in the same integration round. Codify what already happened in practice: take
the next free number in main's docs/adr/, and on collision whoever merges
second renumbers and fixes cross-refs rather than round-tripping.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012fgjnqFCS5h5L7gZTZo3rV
PKG landed docs/adr/0014-conformance-runs-through-ctest.md (dad88fc) and this
lane's 0014-generated-binding-changes-and-versioning.md landed in the same
integration round, both as 0014. Renumbered this one to 0015 — second merger
renumbers. Updated the two cross-references (contracts/README.md rule 4,
proto-answers-daemon-m1.md P1) and the in-file header.
Also while in contracts/README.md: the file-tree comment and the "Method
surface" heading still said v1.2.0; both now v1.4.0 to match VERSION.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_012fgjnqFCS5h5L7gZTZo3rV
Rebased onto main at 1.4.0. The regenerated Dispatcher returns
HandlerResult<T> = expected<T, HandlerError{code, message, data}>
(ADR 0014); the covariant-return break on all 39 overrides is the swap
predicted in daemon/docs/proto-requests-m1.md P1.
- dispatcher.hpp/.cpp: Result<T> -> HandlerResult<T> on every override;
not_implemented() now returns HandlerError{InternalError, ...} rather
than a ParseError forwarded as -32603.
- download.get: returns -32010 TaskNotFound with data.taskId. Not a
placeholder — with no store, every id is genuinely not-found, which
is the real answer for contracts/ fixture download.get.not-found. It
becomes a store lookup when store/ is wired in.
- uds_roundtrip: the -32603-collapse guard is now a -32010 + data.taskId
assertion, the regression guard the P1 note promised.
download.add (-32011) and download.probe (-32013) stay InternalError
until they have real bodies (canonicalization / probe); they get their
fixture codes when that logic lands.
All 24 tests green; uds_roundtrip TSan-clean.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
The daemon's persistent state. SQLite in WAL mode, foreign keys on,
5 s busy timeout so a writer waits rather than SQLITE_BUSY under the
RPC loop.
- store/sqlite — RAII Db/Stmt over the C API; errors returned as
DbResult<T> (std::expected), never thrown — the RPC loop must not
unwind. transaction() helper: BEGIN / fn / COMMIT, ROLLBACK on error.
- store/migrations/0001_initial.sql — the eight tables from the brief:
settings, categories, queues, tasks, segments, rules, history,
pairings. Notable choices:
* tasks columns project onto proto TaskSummary with no computation;
requested vs effective segments/buffer split per ADR 0010/0012;
pause_reason column per ADR 0013.
* segments end_byte is NOT constrained >= 0 so a whole-file
zero-length download is one row with end_byte = -1 (ADR 0010 B3a).
* pairings stores only token_sha256 — the plaintext token is
returned once from session.pair and never persisted (CLAUDE.md §4).
* indices on tasks(state), (category_id), (queue_id, queue_position),
(created_at), (completed_at) for the "1000 tasks, download.list
under 50 ms" DoD.
* six built-in categories + a Main queue seeded.
- store/migrations — runs every embedded migration past PRAGMA
user_version, each in its own transaction, forward-only. SQL files
are embedded at build time by cmake/embed_migrations.cmake.
Test veloxd.store_migrations (ASan+UBSan and TSan clean): fresh DB ->
head, all tables present, seed rows, FK cascade (segment orphan
rejected, task delete cascades), the end_byte=-1 zero-length case,
idempotent re-run, and forward-only from every released user_version.
Also: daemon/docs/proto-requests-m1.md — P1 marked landed on lane/proto
as 1.4.0 (HandlerError/HandlerResult), to be adopted in rpc/ once that
merges to main; P2 resolved.
Not linked into the running daemon yet — the store is wired to the
dispatcher when download.add/list/get get real bodies, next.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
The scriptable client, built now rather than last: it is how the daemon
gets exercised before the GUI is pointed at it (AGENT-DAEMON.md).
- src/client — synchronous blocking RPC over the Unix socket: resolve
$XDG_RUNTIME_DIR/velox/velox.sock, connect, session.hello, one framed
request/reply per call. Distinguishes transport failure (exit 3) from
a daemon-returned error (exit 1).
- src/main — subcommands add/ls/pause/resume/rm; --json prints the raw
JSON-RPC result or error; --dir/--out/--segments on add;
--delete-file on rm. Usage errors exit 2.
ls works end to end against veloxd today (empty table). add and the
bulk verbs reach the daemon and surface its "not implemented" (-32603)
cleanly until the store lands — the plumbing is done, the commands
light up as handlers do.
Test velox.client: the real Client against an in-process UdsServer —
no-daemon path, session.hello, download.list, and a not-implemented
method surfacing as an RPC error rather than a transport error.
ASan+UBSan clean; full tree (21 tests, incl. conformance) green.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
First real code in daemon/. veloxd now listens on
$XDG_RUNTIME_DIR/velox/velox.sock (0600, SO_PEERCRED same-UID check),
frames NDJSON, and routes every method through the generated
velox::proto::dispatch(). The CLI and GUI have a server to talk to.
Modules:
- rpc/ndjson.hpp — newline-delimited framing, 8 MiB frame cap, CRLF-
tolerant, partial-tail buffering. Header-only, tested.
- rpc/event_loop — single-threaded poll(2) reactor; never blocks the
loop. stop()/wake() are async-signal-safe (eventfd).
- rpc/runtime_dir — $XDG_RUNTIME_DIR/velox resolution, 0700, owner-checked;
refuses an insecure fallback rather than using /tmp.
- rpc/uds_server — listener + non-blocking per-conn read/write with
backpressure; handles session.hello (protocol-major
check -> -32001, sessionId, transport=uds) and
session.subscribe in the server layer; routes the
rest through dispatch().
- rpc/dispatcher — VeloxDispatcher : proto::Dispatcher, all 39 methods.
download.list answers an empty table; the rest return
"not implemented" (-> -32603) until the store lands.
- main.cpp — abstract-namespace single-instance lock, signal ->
clean shutdown, socket unlinked on exit.
Tests (ASan+UBSan and TSan clean):
- veloxd.ndjson — framing edge cases
- veloxd.uds_roundtrip — real socket: hello ok / version mismatch / empty
list / -32601 / -32700 / pipelined requests, and a
guard on the -32603 collapse documented in P1.
Known gap, filed not worked around: daemon/docs/proto-requests-m1.md P1 —
the generated Dispatcher has no error channel below -32603, so handlers
cannot yet return -32010/-32011/-32013 with their data payloads. The
server layer handles -32001/-32002/-32003 around dispatch(); genuine
in-handler errors collapse to -32603 until PROTO gives handlers a real
error return. Three error fixtures are non-conformant until then.
Not in this drop: rpc/ws_server (next; needs the store for hashed pairing
tokens), store/, sched/, cli/. WS reuses this event loop.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
samiandClaude Sonnet 5