6632b750992ad96d8bb88b039639fc64a2298aa1
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  samiandClaude Sonnet 5
|
6632b75099 |
daemon: capture.offer for real (D7/D8) — rules, category resolution, dedupe, deadline
capture.offer applies capture.enabled/excludedHosts/monitoredExtensions/
monitoredMimeTypes/minSizeBytes from settings, then the rules table (new
store::Rules + CORE's vdm::rules::match_rules/glob_match — DAEMON only converts its
own stored proto::Rule JSON into CORE's plain vocabulary, per that header's own
layering note), resolves the category folder (a rule's explicit categoryId/saveDir,
else store::Categories::guess_by_extension — the same extension guess
download.probe's suggestedCategoryId already used, now shared instead of
duplicated), dedupes against active (non-terminal) tasks by exact URL, and on `take`
calls add_one() — the same path download.add itself uses, so a captured download is
a real, admitted, persisted task, not a special case.
The 750ms deadline (CLAUDE.md §4 / AGENT-DAEMON.md build step 6) is checked
cooperatively between every step via a new rpc::CaptureDataSource seam: the real
implementation wraps store::Settings/Categories/Rules/Tasks; a test fake can jump its
own injected clock forward to simulate "the store was slow just now" with zero real
sleep. This catches the realistic failure mode (several slow steps adding up) though
it cannot preempt a single pathologically stuck call mid-flight — a true preemptive
guarantee would need the same async/background-thread treatment as download.probe,
which isn't safe to do against the same sqlite3 connection (opened SQLITE_OPEN_NOMUTEX,
explicitly not for concurrent use) without a second connection; left as a known,
documented limit of this pass rather than adding that plumbing speculatively.
capture.getRules returns the same settings-backed fields capture.offer itself reads,
so the two can never drift. rulesVersion is a placeholder constant (1) — no persisted
revision counter exists yet, and the extension already re-fetches on
event.settings.changed regardless.
New store/rules.{hpp,cpp}: rules table CRUD (list, and an atomic upsert+remove for
rules.upsert later). store::Categories::guess_by_extension replaces a duplicate copy
that used to live in sched/scheduler.cpp. store::Tasks::has_active_duplicate for the
dedupe check.
Verified against real veloxd + tools/testserver: a monitored-type offer answers in
~5ms and actually creates + downloads the task, correctly categorized; an
unmonitored type, an excluded host, a rule-vetoed host, and a second offer for a
still-active URL all answer ignore with the right reason; a bad category save dir
surfaces its real -32011. New capture_offer_test covers all of the above plus the
deadline itself (two cases, one per "slow" checkpoint), asserting real wall-clock
time barely moves even though the fake clock jumped 2 simulated seconds. Full ctest:
54/54 (excluding the pre-existing, unrelated conformance failure noted in the
previous commit).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
samiandClaude Sonnet 5
|
de748cc2fc |
daemon: fix startMode 'later' -32603 and isolate the single-instance lock by runtime dir
Two bugs blocking PROTO's live-veloxd conformance check.
1. tasks.start_mode's CHECK was ('auto','now','queue','manual') — not the
contract's StartMode enum (['now','later','queue']) at all. 'later', a real
documented value (the File Info dialog's Download Later button), hit the CHECK
on every insert and surfaced as an unhandled -32603; 'auto'/'manual' were
never contract values to begin with.
Migration 0003 rebuilds tasks (SQLite can't ALTER a CHECK) with the contract's
values, remapping existing rows by what they actually meant: 'auto' -> 'now'
(eligible for the scheduler immediately), 'manual' -> 'later' (parked, matching
StartMode's own "lands the task in paused" description). store_migrations_test
covers the remap and that 'later' inserts clean while the retired spellings
are rejected.
dispatcher.cpp's on_download_add matched: default (absent startMode) is now
'now' instead of the invented 'auto'; 'later' actually lands the task in
`paused` (pause_reason 'user') instead of a dead 'manual' -> `new` branch that
spec.startMode (typed as the 3-value enum) could never even reach.
TaskRow::start_mode's in-memory default followed suit ('now').
2. main.cpp's single-instance guard bound an abstract socket named
"velox-daemon-<euid>" — one name per user, system-wide. XDG_RUNTIME_DIR
isolation never reached it: a leaked test veloxd held the lock for 4h40m and
locked out every other isolated instance with the same euid (PROTO, EXT, the
orchestrator), real daemon included.
Extracted rpc/single_instance.{hpp,cpp} (was a static in main.cpp, untestable)
and derived the abstract-socket name from a hash of the resolved runtime dir
path instead of euid alone. The real per-user daemon is still unique (its
runtime dir is unique to it); isolated instances pointed at their own runtime
dirs now coexist. main() resolves the runtime dir before acquiring the lock
(was the other way around). New single_instance_test covers same-dir refusal,
different-dir coexistence, and release-on-close.
Verified against real veloxd binaries, not just unit tests: startMode: "later"
via a live download.add lands in `paused`; two veloxd with different runtime
dirs run concurrently, two with the same one and the second refuses with the
runtime dir named in the error. Full ctest: 39/39.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
samiandClaude Sonnet 5
|
a967eca669 |
daemon: persist engine progress/probe to the store — segments:0 no longer leaks
Engine numbers never reached the store: download.get after a correctly-finished
download reported sizeBytes: null, downloadedBytes: 0, speedBps: 0, resumable:
false, segments: 0, segmentDetail: [] — segments: 0 breaks the frozen contract
(TaskSummary.segments is minimum:1, required).
- Scheduler::tick() now probes (EnginePort::probe) before every start(), persisting
sizeBytes/resumable/validators via Tasks::set_probe_result before a byte moves,
then starts the engine with that ProbeResult as probe_hint.
- Scheduler::persist_progress() (new) writes downloadedBytes/speedBps/segments/
segmentDetail from the engine's Progress. Called from progress_snapshot() (the
250ms tick) *and* once more from on_engine_state right before release()/unmap on
every terminal transition, so a task that finishes between two ticks — the common
case for anything small or fast — still leaves real numbers instead of the
pre-persistence defaults.
- TaskSummary.segments is sourced from segments.size() when the task has any
(matching what actually lands in segmentDetail, per the schema's "exactly
segments entries"), falling back to the engine's effective_segments (budget
slots *held*, not necessarily physical range count) only pre-segmentation.
- Tasks::set_final_bytes tops up on_finished's byte count as a last-resort
backstop.
- store/segments.{cpp,hpp}: read/write access to the segments table behind
TaskDetail.segmentDetail. Wired into daemon/CMakeLists.txt.
- migrations/0002: speed_bps on tasks and segments; fixes segments.state's CHECK
to include 'pending' (0001 omitted it, so a pre-connect snapshot could never be
written).
- store_migrations_test's forward-only loop faked "released version N" by setting
the user_version pragma alone, with no real schema underneath — never exercised
until 0002 existed. Fixed to actually build the db through migrations 1..N first.
Verified against real veloxd + tools/testserver (not just unit tests):
download.list/download.get correct immediately after completion and after a
daemon restart, with saveTo.allowedRoots pointed at an isolated dir.
Observed but not fixed (CORE, not this lane, noted in deferrals.md): Progress.
speed_bps reads back 0 for the whole lifetime of a live throttled download in the
same E2E check, despite downloadedBytes visibly advancing. DAEMON passes it
through unmodified; filed rather than worked around.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
samiandClaude Sonnet 5
|
824fa481bb |
daemon: event.* fan-out (D5) + category.list/queue.list (D3) — GUI-ready
The two items aimed at pointing GUI at a real veloxd instead of mockd.
rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.
sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
the store's own prior row for previousState (authoritative
regardless of engine/scheduler timing), writes the error columns,
and — when a hub is supplied — publishes event.task.state with
{taskId, state, previousState, summary, error}. Wired into every
transition: scheduler-driven (admission -> probing, resume ->
connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
(EnginePort::progress(), a new interface method backed by
DownloadHandle::progress()), plus a store side-effect
(Tasks::update_progress) so download.list/get stay current between
state transitions. Returns rows; does NOT publish itself — batching
into one array message is the caller's job, per the schema's
x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
event.task.progress per tick, only when there's something to say.
dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").
store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).
Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.
Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.
deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|
||
|
samiandClaude Sonnet 5
|
6787784936 |
daemon: store/ query layer — tasks + settings (prep for the download.add vertical slice)
The read/write surface the dispatcher handlers need, so download.add persists and download.list / download.get project real rows when the engine lands. - store/tasks — TaskRow (1:1 with the schema), insert / get / remove / set_state / count, and list(filter, sort, offset, limit) that does all the WHERE / ORDER BY / LIMIT in SQL (M1 DoD: a 1000-row list never materialised client-side). Filter covers states / category / queue / case-insensitive filename+url substring / date range; sort is a whitelisted column + direction with NULLs last, default newest-first; the enum spellings in a state IN (...) come from proto::to_string, never from user text. to_summary() projects a row onto proto::TaskSummary including the flattened error block when the task failed / retry_wait / auto-paused. - store/settings — key -> JSON-text with a built-in default table mirroring Settings.schema.json / ADR 0012; get_raw / set_raw / overrides plus typed get_int / get_string / get_string_array for the governor config and saveTo.allowedRoots. Full settings.get/set wire projection lands with those handlers. - veloxd_store now links velox::proto + nlohmann_json for the projection. Test veloxd.store_tasks: insert/get round trip, PK duplicate rejected, the error-block projection, list total+paging+sort+every filter, set_state pause_reason clear-on-unpause, remove, and settings default vs override. ASan+UBSan and TSan clean; 34 daemon/cli tests green. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig |