e0f22f808fe8b3f9159a8cc76c41915fef4dd524
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  samiandClaude Sonnet 5
|
6632b75099 |
daemon: capture.offer for real (D7/D8) — rules, category resolution, dedupe, deadline
capture.offer applies capture.enabled/excludedHosts/monitoredExtensions/
monitoredMimeTypes/minSizeBytes from settings, then the rules table (new
store::Rules + CORE's vdm::rules::match_rules/glob_match — DAEMON only converts its
own stored proto::Rule JSON into CORE's plain vocabulary, per that header's own
layering note), resolves the category folder (a rule's explicit categoryId/saveDir,
else store::Categories::guess_by_extension — the same extension guess
download.probe's suggestedCategoryId already used, now shared instead of
duplicated), dedupes against active (non-terminal) tasks by exact URL, and on `take`
calls add_one() — the same path download.add itself uses, so a captured download is
a real, admitted, persisted task, not a special case.
The 750ms deadline (CLAUDE.md §4 / AGENT-DAEMON.md build step 6) is checked
cooperatively between every step via a new rpc::CaptureDataSource seam: the real
implementation wraps store::Settings/Categories/Rules/Tasks; a test fake can jump its
own injected clock forward to simulate "the store was slow just now" with zero real
sleep. This catches the realistic failure mode (several slow steps adding up) though
it cannot preempt a single pathologically stuck call mid-flight — a true preemptive
guarantee would need the same async/background-thread treatment as download.probe,
which isn't safe to do against the same sqlite3 connection (opened SQLITE_OPEN_NOMUTEX,
explicitly not for concurrent use) without a second connection; left as a known,
documented limit of this pass rather than adding that plumbing speculatively.
capture.getRules returns the same settings-backed fields capture.offer itself reads,
so the two can never drift. rulesVersion is a placeholder constant (1) — no persisted
revision counter exists yet, and the extension already re-fetches on
event.settings.changed regardless.
New store/rules.{hpp,cpp}: rules table CRUD (list, and an atomic upsert+remove for
rules.upsert later). store::Categories::guess_by_extension replaces a duplicate copy
that used to live in sched/scheduler.cpp. store::Tasks::has_active_duplicate for the
dedupe check.
Verified against real veloxd + tools/testserver: a monitored-type offer answers in
~5ms and actually creates + downloads the task, correctly categorized; an
unmonitored type, an excluded host, a rule-vetoed host, and a second offer for a
still-active URL all answer ignore with the right reason; a bad category save dir
surfaces its real -32011. New capture_offer_test covers all of the above plus the
deadline itself (two cases, one per "slow" checkpoint), asserting real wall-clock
time barely moves even though the fake clock jumped 2 simulated seconds. Full ctest:
54/54 (excluding the pre-existing, unrelated conformance failure noted in the
previous commit).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
samiandClaude Sonnet 5
|
c89158ea09 |
daemon: D3 (partial) — category.upsert/remove, queue.upsert, download.remove/addBatch/provideAuth
Closes a bounded, high-value slice of the remaining D3 stubs. settings.*/rules.*/ limiter.*/schedule.*/grabber.*/media.*/capture.*/queue.reorder/download.refreshUrl/ download.update stay deferred — reasons noted individually in deferrals.md (settings.* specifically: a real, large field<->key<->JSON-type mapping table across ~43 fields / ~50 SettingKeys, not started rather than rushed). category.upsert/remove: store/categories.hpp gains get/upsert/remove. upsert generates an id when absent and always ignores the payload's `builtin` (preserved from the existing row on replace, false on create); saveDir goes through the same fs::resolve_target canonicalize-and-root-check as download.add. remove refuses a builtin at both layers (dispatcher's -32602 pre-check; the store's own "DELETE ... AND builtin = 0" as defense in depth) and reassigns member tasks to reassignTo (default "general") inside one transaction before deleting the row. queue.upsert: store/queues.hpp gains get/upsert (set_state already existed from D4b). Same create-generates-id pattern; a create always starts 'stopped', a replace keeps the queue's current run state (upsert edits config, not run state — that's queue.start/stop). Also fixed in passing: on_complete has been a real column since migration 0001 but Queues::list/get never projected it onto Queue.onComplete. download.remove: cancels with discard_partial=true (always drops the .veloxpart pair — unlike download.cancel, which keeps them, the row is gone either way), deletes the finished file only when deleteFile is true and the task was complete (best-effort), deletes the row (segments cascade via the FK), and publishes event.task.removed (closing the last open note under D5). download.addBatch: on_download_add's body is now a shared add_one(), called once per item after merging each item's unset fields against params.defaults. download.provideAuth: forwards to EnginePort::provide_auth via a new TaskActionPort::provide_auth. `remember`/persisting to the Secret Service is accepted but not acted on — nothing in this build talks to libsecret yet. Verified against real veloxd + tools/testserver: category create/replace/ remove-with-reassignment, queue create/replace-keeps-run-state, a batch add sharing defaults.saveDir, and download.remove with deleteFile actually deleting the file and the task then 404ing download.get with -32010. Full ctest: 39/39. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP |
||
|
samiandClaude Sonnet 5
|
55f0c6099d |
daemon: D4b — download.pause/resume/start/cancel and queue.start/stop drive the scheduler
download.pause/resume/start/cancel and queue.start/stop were stubs; now they call into the scheduler and take effect immediately, not on the next 1s tick — pausing, resuming or cancelling a live transfer can't wait, and per ADR 0013 §3 the governor never touches a user-owned pause on its own. New rpc::TaskActionPort interface (owned by rpc/, implemented by sched::Scheduler) is what dispatcher.hpp depends on instead of sched/scheduler.hpp directly. Needed because veloxd_sched already links veloxd_rpc (for EventHub); dispatcher.hpp pulling in sched/scheduler.hpp directly would make it a real circular library dependency, breaking anything that links veloxd_rpc alone (cli's tests, as it turned out — hit and fixed during this change). Scheduler::user_pause/user_resume/user_start/user_cancel + pause_queue follow tick()'s existing to_pause pattern: call the engine (async, no synchronous effect) and transition the store eagerly so download.get/list are correct the instant the RPC call returns. Fixed a real bug surfaced while building this: transition() always overwrote pause_reason to NULL when the engine's own delayed pause-ack callback (on_state to paused, no error) arrived after whoever actually initiated the pause had already written the real reason — now it preserves the stored reason when the callback supplies none, instead of clobbering it. Covered by a regression check in sched_scheduler_test. store/queues gets get() and set_state() (was list()-only) for queue.start/stop. Verified against real veloxd + tools/testserver, not just unit tests: pausing a live single-segment throttled transfer freezes downloadedBytes, resume continues it from that point, cancel stops it; a bad taskId comes back in BulkTaskResult.failed with -32010, not a crash; queue.stop(pauseRunning:true) pauses the queue's running task immediately and queue.start resumes admission. Known gap: download.start's contract "a task in 'queued' jumps its queue" (priority bump) is not implemented — admission is still plain FIFO by created_at. Noted in deferrals.md. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP |
||
|
samiandClaude Sonnet 5
|
824fa481bb |
daemon: event.* fan-out (D5) + category.list/queue.list (D3) — GUI-ready
The two items aimed at pointing GUI at a real veloxd instead of mockd.
rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.
sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
the store's own prior row for previousState (authoritative
regardless of engine/scheduler timing), writes the error columns,
and — when a hub is supplied — publishes event.task.state with
{taskId, state, previousState, summary, error}. Wired into every
transition: scheduler-driven (admission -> probing, resume ->
connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
(EnginePort::progress(), a new interface method backed by
DownloadHandle::progress()), plus a store side-effect
(Tasks::update_progress) so download.list/get stay current between
state transitions. Returns rows; does NOT publish itself — batching
into one array message is the caller's job, per the schema's
x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
event.task.progress per tick, only when there's something to say.
dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").
store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).
Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.
Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.
deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|
||
|
samiandClaude Sonnet 5
|
08d7ee9263 |
daemon: wire the engine into veloxd — the vertical slice runs end to end
CORE stage 8 merged, so vdm::Engine is linkable. This closes D4a and narrows D4b: `velox add <url>` now actually downloads. - sched/engine_port_core.hpp — the real EnginePort: forwards to a live vdm::Engine, keeps the DownloadHandle per task for pause/resume/ cancel/provide_auth/decide/refresh_url, drives set_task_order / set_max_active_segments / set_host_segment_cap via engine.segment_budget(). CORE confirmed the admission model: DAEMON decides when to start(); the engine's own download_task calls register_task/set_want internally — DAEMON never touches per-task budget calls. EnginePort gains release(TaskId) so the port drops a handle when the task goes terminal. - rpc/event_loop — EventLoop::post(fn): thread-safe, runs fn on the loop thread next iteration. The marshaller for engine-thread callbacks. - main.cpp — constructs vdm::Engine + EnginePortCore + Scheduler (post_to_loop = loop.post). At startup: reconcile_after_restart() (ADR 0013 §5), reload_config(), tick(). A 1 s timerfd on the loop re-runs tick() (schedule windows, missed nudges); download.add nudges via dispatcher.set_on_mutation. End-to-end verified against tools/testserver: `velox add http://127.0.0.1:.../file/512K` -> task queued -> scheduler admits -> engine downloads 524288 bytes -> complete, file on disk. First byte-path all the way through the project. safepath-adversarial.md: re-verified per its own note — CORE landed O_NOFOLLOW on the target open (core/src/io/sparse_file.cpp), so the leaf-symlink TOCTOU is now closed; residual is down to one intermediate-dir gap (documented post-M1 chase). 36 daemon/cli tests green; scheduler + uds_roundtrip TSan-clean. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig |
||
|
samiandClaude Sonnet 5
|
3db0d01f9d |
daemon: download.add / download.list / download.get behind the store
VeloxDispatcher now takes a store::Db& and three handlers are real:
- download.list -> store::Tasks::list (filter / sort / paging in SQL) ->
to_summary per row. No more empty-table stub.
- download.add -> resolve saveDir (spec, else saveTo.defaultDir; ~ expanded)
and the leaf (spec.filename, else the URL's last segment percent-decoded,
else download.bin) -> fs::resolve_target against canonicalize_root'd
saveTo.allowedRoots. Any path-destination failure is -32011 with the
*original* saveDir in data.path. On success a TaskRow is inserted in
state `queued` (or `new` for startMode "manual") and {taskId, state}
returned. The scheduler that would then admit it is D4.
- download.get -> store::Tasks::get; a real -32010 + data.taskId for an
unknown id, else a TaskDetail (segmentDetail empty until the engine
segments the task, which the schema permits).
util/time.hpp: now_iso() factored out of ws_server.cpp.
main.cpp constructs the dispatcher with the opened db. The three
integration tests build an in-memory migrated db for it; velox.client
now drives the full slice through the CLI — add outside roots -> -32011
with data.path, add into an allowed root -> a task that download.list
shows and download.get details, unknown id -> -32010. Verified with the
real binaries: velox add persists, velox ls shows it, it survives a
daemon restart, /etc is refused.
ASan+UBSan and TSan clean; 34 daemon/cli tests green. deferrals.md:
D2 down to just download.probe; D3 down to categories/queues/rules/
settings/limiter/schedule.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|
||
|
samiandClaude Sonnet 5
|
e585113daf |
daemon: adopt HandlerResult<T> — real error codes from handlers (contracts/ 1.4.0)
Rebased onto main at 1.4.0. The regenerated Dispatcher returns
HandlerResult<T> = expected<T, HandlerError{code, message, data}>
(ADR 0014); the covariant-return break on all 39 overrides is the swap
predicted in daemon/docs/proto-requests-m1.md P1.
- dispatcher.hpp/.cpp: Result<T> -> HandlerResult<T> on every override;
not_implemented() now returns HandlerError{InternalError, ...} rather
than a ParseError forwarded as -32603.
- download.get: returns -32010 TaskNotFound with data.taskId. Not a
placeholder — with no store, every id is genuinely not-found, which
is the real answer for contracts/ fixture download.get.not-found. It
becomes a store lookup when store/ is wired in.
- uds_roundtrip: the -32603-collapse guard is now a -32010 + data.taskId
assertion, the regression guard the P1 note promised.
download.add (-32011) and download.probe (-32013) stay InternalError
until they have real bodies (canonicalization / probe); they get their
fixture codes when that logic lands.
All 24 tests green; uds_roundtrip TSan-clean.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|
||
|
samiandClaude Sonnet 5
|
e60d6669d8 |
daemon: rpc/ — Unix-socket transport + generated dispatch wiring (build step 1)
First real code in daemon/. veloxd now listens on
$XDG_RUNTIME_DIR/velox/velox.sock (0600, SO_PEERCRED same-UID check),
frames NDJSON, and routes every method through the generated
velox::proto::dispatch(). The CLI and GUI have a server to talk to.
Modules:
- rpc/ndjson.hpp — newline-delimited framing, 8 MiB frame cap, CRLF-
tolerant, partial-tail buffering. Header-only, tested.
- rpc/event_loop — single-threaded poll(2) reactor; never blocks the
loop. stop()/wake() are async-signal-safe (eventfd).
- rpc/runtime_dir — $XDG_RUNTIME_DIR/velox resolution, 0700, owner-checked;
refuses an insecure fallback rather than using /tmp.
- rpc/uds_server — listener + non-blocking per-conn read/write with
backpressure; handles session.hello (protocol-major
check -> -32001, sessionId, transport=uds) and
session.subscribe in the server layer; routes the
rest through dispatch().
- rpc/dispatcher — VeloxDispatcher : proto::Dispatcher, all 39 methods.
download.list answers an empty table; the rest return
"not implemented" (-> -32603) until the store lands.
- main.cpp — abstract-namespace single-instance lock, signal ->
clean shutdown, socket unlinked on exit.
Tests (ASan+UBSan and TSan clean):
- veloxd.ndjson — framing edge cases
- veloxd.uds_roundtrip — real socket: hello ok / version mismatch / empty
list / -32601 / -32700 / pipelined requests, and a
guard on the -32603 collapse documented in P1.
Known gap, filed not worked around: daemon/docs/proto-requests-m1.md P1 —
the generated Dispatcher has no error channel below -32603, so handlers
cannot yet return -32010/-32011/-32013 with their data payloads. The
server layer handles -32001/-32002/-32003 around dispatch(); genuine
in-handler errors collapse to -32603 until PROTO gives handlers a real
error return. Three error fixtures are non-conformant until then.
Not in this drop: rpc/ws_server (next; needs the store for hashed pairing
tokens), store/, sched/, cli/. WS reuses this event loop.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|