755d85964eb3dcc8ceffee624382d9db8735f07e
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  samiandClaude Sonnet 5
|
4f6fb1029d |
daemon: D2 — download.probe, genuinely async on both transports
download.probe was a stub (-32603). It needs an HTTP round trip on the engine's probe
pool (up to the schema's 30s x-deadlineMs), which cannot fit VeloxDispatcher's
synchronous on_download_probe -> HandlerResult<T> return without blocking the RPC
loop for the duration — a hard no per CLAUDE.md ("never block the RPC loop") and
AGENT-DAEMON.md build step 1.
uds_server.cpp and ws_server.cpp special-case "download.probe" before the generic
dispatch(), exactly the way they already special-case session.hello/session.subscribe:
parse the params, call the port, and queue the reply whenever the callback fires
(dropped silently if the connection is gone by then).
rpc::TaskActionPort gains probe_now(DownloadProbeParams, callback) — kept in proto/std
terms, no vdm::net::* in the signature, so veloxd_rpc never needs core/include's vdm
headers just to declare this. sched::Scheduler::probe_now is the implementation:
builds a vdm::net::ProbeRequest, runs it on the engine's probe pool, marshals the
engine-thread callback back onto the loop (deps_.post_to_loop, same as every other
engine callback here), maps a probe failure to -32013 ProbeFailed (data.httpStatus set
when there was an HTTP response), and fills suggestedCategoryId/suggestedSaveDir with a
plain extension match against the categories table — not the real rules engine, which
is still D3; noted in a comment.
Verified against real veloxd + tools/testserver, not just unit tests: a real probe
answers in ~5ms with size/resumable/etag/redirect chain; a bad host maps to -32013;
and — the actual point of the async design — a connection running a 10s slow-loris
probe does not block a second connection's download.list, which answers in ~1ms while
the probe is still outstanding.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
samiandClaude Sonnet 5
|
55f0c6099d |
daemon: D4b — download.pause/resume/start/cancel and queue.start/stop drive the scheduler
download.pause/resume/start/cancel and queue.start/stop were stubs; now they call into the scheduler and take effect immediately, not on the next 1s tick — pausing, resuming or cancelling a live transfer can't wait, and per ADR 0013 §3 the governor never touches a user-owned pause on its own. New rpc::TaskActionPort interface (owned by rpc/, implemented by sched::Scheduler) is what dispatcher.hpp depends on instead of sched/scheduler.hpp directly. Needed because veloxd_sched already links veloxd_rpc (for EventHub); dispatcher.hpp pulling in sched/scheduler.hpp directly would make it a real circular library dependency, breaking anything that links veloxd_rpc alone (cli's tests, as it turned out — hit and fixed during this change). Scheduler::user_pause/user_resume/user_start/user_cancel + pause_queue follow tick()'s existing to_pause pattern: call the engine (async, no synchronous effect) and transition the store eagerly so download.get/list are correct the instant the RPC call returns. Fixed a real bug surfaced while building this: transition() always overwrote pause_reason to NULL when the engine's own delayed pause-ack callback (on_state to paused, no error) arrived after whoever actually initiated the pause had already written the real reason — now it preserves the stored reason when the callback supplies none, instead of clobbering it. Covered by a regression check in sched_scheduler_test. store/queues gets get() and set_state() (was list()-only) for queue.start/stop. Verified against real veloxd + tools/testserver, not just unit tests: pausing a live single-segment throttled transfer freezes downloadedBytes, resume continues it from that point, cancel stops it; a bad taskId comes back in BulkTaskResult.failed with -32010, not a crash; queue.stop(pauseRunning:true) pauses the queue's running task immediately and queue.start resumes admission. Known gap: download.start's contract "a task in 'queued' jumps its queue" (priority bump) is not implemented — admission is still plain FIFO by created_at. Noted in deferrals.md. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP |
||
|
samiandClaude Sonnet 5
|
de748cc2fc |
daemon: fix startMode 'later' -32603 and isolate the single-instance lock by runtime dir
Two bugs blocking PROTO's live-veloxd conformance check.
1. tasks.start_mode's CHECK was ('auto','now','queue','manual') — not the
contract's StartMode enum (['now','later','queue']) at all. 'later', a real
documented value (the File Info dialog's Download Later button), hit the CHECK
on every insert and surfaced as an unhandled -32603; 'auto'/'manual' were
never contract values to begin with.
Migration 0003 rebuilds tasks (SQLite can't ALTER a CHECK) with the contract's
values, remapping existing rows by what they actually meant: 'auto' -> 'now'
(eligible for the scheduler immediately), 'manual' -> 'later' (parked, matching
StartMode's own "lands the task in paused" description). store_migrations_test
covers the remap and that 'later' inserts clean while the retired spellings
are rejected.
dispatcher.cpp's on_download_add matched: default (absent startMode) is now
'now' instead of the invented 'auto'; 'later' actually lands the task in
`paused` (pause_reason 'user') instead of a dead 'manual' -> `new` branch that
spec.startMode (typed as the 3-value enum) could never even reach.
TaskRow::start_mode's in-memory default followed suit ('now').
2. main.cpp's single-instance guard bound an abstract socket named
"velox-daemon-<euid>" — one name per user, system-wide. XDG_RUNTIME_DIR
isolation never reached it: a leaked test veloxd held the lock for 4h40m and
locked out every other isolated instance with the same euid (PROTO, EXT, the
orchestrator), real daemon included.
Extracted rpc/single_instance.{hpp,cpp} (was a static in main.cpp, untestable)
and derived the abstract-socket name from a hash of the resolved runtime dir
path instead of euid alone. The real per-user daemon is still unique (its
runtime dir is unique to it); isolated instances pointed at their own runtime
dirs now coexist. main() resolves the runtime dir before acquiring the lock
(was the other way around). New single_instance_test covers same-dir refusal,
different-dir coexistence, and release-on-close.
Verified against real veloxd binaries, not just unit tests: startMode: "later"
via a live download.add lands in `paused`; two veloxd with different runtime
dirs run concurrently, two with the same one and the second refuses with the
runtime dir named in the error. Full ctest: 39/39.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
samiandClaude Sonnet 5
|
824fa481bb |
daemon: event.* fan-out (D5) + category.list/queue.list (D3) — GUI-ready
The two items aimed at pointing GUI at a real veloxd instead of mockd.
rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.
sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
the store's own prior row for previousState (authoritative
regardless of engine/scheduler timing), writes the error columns,
and — when a hub is supplied — publishes event.task.state with
{taskId, state, previousState, summary, error}. Wired into every
transition: scheduler-driven (admission -> probing, resume ->
connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
(EnginePort::progress(), a new interface method backed by
DownloadHandle::progress()), plus a store side-effect
(Tasks::update_progress) so download.list/get stay current between
state transitions. Returns rows; does NOT publish itself — batching
into one array message is the caller's job, per the schema's
x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
event.task.progress per tick, only when there's something to say.
dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").
store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).
Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.
Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.
deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|
||
|
samiandClaude Sonnet 5
|
08d7ee9263 |
daemon: wire the engine into veloxd — the vertical slice runs end to end
CORE stage 8 merged, so vdm::Engine is linkable. This closes D4a and narrows D4b: `velox add <url>` now actually downloads. - sched/engine_port_core.hpp — the real EnginePort: forwards to a live vdm::Engine, keeps the DownloadHandle per task for pause/resume/ cancel/provide_auth/decide/refresh_url, drives set_task_order / set_max_active_segments / set_host_segment_cap via engine.segment_budget(). CORE confirmed the admission model: DAEMON decides when to start(); the engine's own download_task calls register_task/set_want internally — DAEMON never touches per-task budget calls. EnginePort gains release(TaskId) so the port drops a handle when the task goes terminal. - rpc/event_loop — EventLoop::post(fn): thread-safe, runs fn on the loop thread next iteration. The marshaller for engine-thread callbacks. - main.cpp — constructs vdm::Engine + EnginePortCore + Scheduler (post_to_loop = loop.post). At startup: reconcile_after_restart() (ADR 0013 §5), reload_config(), tick(). A 1 s timerfd on the loop re-runs tick() (schedule windows, missed nudges); download.add nudges via dispatcher.set_on_mutation. End-to-end verified against tools/testserver: `velox add http://127.0.0.1:.../file/512K` -> task queued -> scheduler admits -> engine downloads 524288 bytes -> complete, file on disk. First byte-path all the way through the project. safepath-adversarial.md: re-verified per its own note — CORE landed O_NOFOLLOW on the target open (core/src/io/sparse_file.cpp), so the leaf-symlink TOCTOU is now closed; residual is down to one intermediate-dir gap (documented post-M1 chase). 36 daemon/cli tests green; scheduler + uds_roundtrip TSan-clean. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig |
||
|
samiandClaude Sonnet 5
|
3db0d01f9d |
daemon: download.add / download.list / download.get behind the store
VeloxDispatcher now takes a store::Db& and three handlers are real:
- download.list -> store::Tasks::list (filter / sort / paging in SQL) ->
to_summary per row. No more empty-table stub.
- download.add -> resolve saveDir (spec, else saveTo.defaultDir; ~ expanded)
and the leaf (spec.filename, else the URL's last segment percent-decoded,
else download.bin) -> fs::resolve_target against canonicalize_root'd
saveTo.allowedRoots. Any path-destination failure is -32011 with the
*original* saveDir in data.path. On success a TaskRow is inserted in
state `queued` (or `new` for startMode "manual") and {taskId, state}
returned. The scheduler that would then admit it is D4.
- download.get -> store::Tasks::get; a real -32010 + data.taskId for an
unknown id, else a TaskDetail (segmentDetail empty until the engine
segments the task, which the schema permits).
util/time.hpp: now_iso() factored out of ws_server.cpp.
main.cpp constructs the dispatcher with the opened db. The three
integration tests build an in-memory migrated db for it; velox.client
now drives the full slice through the CLI — add outside roots -> -32011
with data.path, add into an allowed root -> a task that download.list
shows and download.get details, unknown id -> -32010. Verified with the
real binaries: velox add persists, velox ls shows it, it survives a
daemon restart, /etc is refused.
ASan+UBSan and TSan clean; 34 daemon/cli tests green. deferrals.md:
D2 down to just download.probe; D3 down to categories/queues/rules/
settings/limiter/schedule.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|
||
|
samiandClaude Sonnet 5
|
dab071c41a |
daemon: rpc/ws_server — loopback WebSocket transport + pairing (build step 1, second half)
The extension's fallback transport (docs/05 §4). veloxd now also listens on 127.0.0.1, first free port in 52000-52016, and writes it to <runtime>/ws.port (0600). - rpc/ws_frame — RFC 6455 frame codec. Incremental; reassembles continuation frames; enforces "client frames MUST be masked" (§5.1); caps a reassembled message at 8 MiB. This is the attacker-adjacent parser, so it has its own test table. - rpc/ws_handshake — HTTP upgrade parse, Sec-WebSocket-Accept (SHA-1 + base64 via libcrypto), and the two non-negotiable checks: an Origin header must be present and must be moz-extension:// (a page cannot pair). Version must be 13. - rpc/ws_server — per-connection Handshake -> Open state machine on the shared EventLoop. Token gate: session.pair mints a token behind the approver + rate limiter; session.hello must present a valid one; every other method is -32002 until authed. Privileged methods are refused -32003 by the generated dispatch(). Ping -> Pong; Close echoed. session.hello major-version mismatch -> -32001. - rpc/pairing — PairingApprover interface + EnvAutoApprover dev stub (approves iff VELOX_PAIR_AUTO=1); PairingRateLimiter (5 failures / 60 s per origin, then 60 s lockout -> -32014, survives reconnect); a four-digit code generator. - store/pairings — the pairings table: create() returns the plaintext token once and stores only its SHA-256; find_active_by_token, touch, revoke, list_active. - util/crypto — sha1 / sha256_hex / base64 / random_token over libcrypto. - store/sqlite — pin the DB file (and -wal/-shm) to 0600. - runtime_dir — resolve_data_dir() for $XDG_DATA_HOME/velox (velox.db). - main.cpp — opens + migrates velox.db, starts both transports; a WS bind failure is logged, not fatal (capture must fail open, the Unix socket still serves the GUI/CLI). Real gap, flagged not hidden: the pairing prompt is EnvAutoApprover for now — a GUI dialog / desktop notification is build step 7. Pairing needs VELOX_PAIR_AUTO=1 until then. Tests (ASan+UBSan and TSan clean): veloxd.ws_frame (codec + handshake vectors incl. the RFC 6455 §1.3 accept sample), veloxd.pairings (token create/find/revoke, hash-not-token, rate-limit window + lockout + per-origin isolation + success reset), veloxd.ws_server (full flow: 101 handshake, -32002 gate, deny-then-approve pairing, hello-with-token, -32003 privileged refusal, real download.list). 27 daemon/cli tests green; full tree green. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig |
||
|
samiandClaude Sonnet 5
|
e60d6669d8 |
daemon: rpc/ — Unix-socket transport + generated dispatch wiring (build step 1)
First real code in daemon/. veloxd now listens on
$XDG_RUNTIME_DIR/velox/velox.sock (0600, SO_PEERCRED same-UID check),
frames NDJSON, and routes every method through the generated
velox::proto::dispatch(). The CLI and GUI have a server to talk to.
Modules:
- rpc/ndjson.hpp — newline-delimited framing, 8 MiB frame cap, CRLF-
tolerant, partial-tail buffering. Header-only, tested.
- rpc/event_loop — single-threaded poll(2) reactor; never blocks the
loop. stop()/wake() are async-signal-safe (eventfd).
- rpc/runtime_dir — $XDG_RUNTIME_DIR/velox resolution, 0700, owner-checked;
refuses an insecure fallback rather than using /tmp.
- rpc/uds_server — listener + non-blocking per-conn read/write with
backpressure; handles session.hello (protocol-major
check -> -32001, sessionId, transport=uds) and
session.subscribe in the server layer; routes the
rest through dispatch().
- rpc/dispatcher — VeloxDispatcher : proto::Dispatcher, all 39 methods.
download.list answers an empty table; the rest return
"not implemented" (-> -32603) until the store lands.
- main.cpp — abstract-namespace single-instance lock, signal ->
clean shutdown, socket unlinked on exit.
Tests (ASan+UBSan and TSan clean):
- veloxd.ndjson — framing edge cases
- veloxd.uds_roundtrip — real socket: hello ok / version mismatch / empty
list / -32601 / -32700 / pipelined requests, and a
guard on the -32603 collapse documented in P1.
Known gap, filed not worked around: daemon/docs/proto-requests-m1.md P1 —
the generated Dispatcher has no error channel below -32603, so handlers
cannot yet return -32010/-32011/-32013 with their data payloads. The
server layer handles -32001/-32002/-32003 around dispatch(); genuine
in-handler errors collapse to -32603 until PROTO gives handlers a real
error return. Three error fixtures are non-conformant until then.
Not in this drop: rpc/ws_server (next; needs the store for hashed pairing
tokens), store/, sched/, cli/. WS reuses this event loop.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|