lane/gui
154
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c2eef96175 |
gui: floating drop target, clipboard global shortcut, theming, UI watchdog
Continues the build order past Options/Scheduler/Speed Limiter/Batch/ Grabber/tray. - DropTargetWidget: frameless always-on-top drop target (docs/03-gui-spec.md §5), position persisted, accepts a dropped http(s) URL or link text and opens FileInfoDialog directly (skipping Add URL, since the URL is already known). Shown/hidden from general.showDropTarget, live via event.settings.changed, same pattern MainWindow already used for general.minimizeToTray. - Clipboard, explicit path #2 (docs/06-risks-and-spikes.md R2): GlobalShortcut wraps org.freedesktop.portal.GlobalShortcuts (CreateSession -> BindShortcuts -> Activated), triggering the same Add URL flow. Guarded end-to-end on `if(TARGET Qt6::DBus)` / VELOX_GUI_HAVE_DBUS so a build without the component degrades to "feature skipped," not broken (gui/docs/pkg-qa-requests-m1.md R4). Best-effort by design per the risk doc: fails silent, never advertised. Verified live against the real portal (a real Wayland session, not just offscreen): `CreateSession` refuses every caller with "An app id is required" — reproduced identically via a bare `busctl` call with no Qt involved at all, so this is the portal requiring a sandboxed caller identity, not something fixable from an unconfined process. Recorded as a partial Spike S2 answer in docs/06-risks-and-spikes.md: this explicit path likely doesn't work for Velox as a traditionally-packaged app on stock GNOME, only if/when it ships confined. Also fixed a real leak this verification caught: QDBusInterface's introspection cache reads as a LeakSanitizer leak the first time anything touches D-Bus (tst_rtl went red under ASan) — switched to QDBusMessage::createMethodCall, which needs no introspection. - Theming (docs/03-gui-spec.md §7): gui/resources/qss/{idm-like,dark}.qss, each with a documented palette block up top (QSS itself has no variable syntax), applied by ThemeManager and kept live via QStyleHints::colorSchemeChanged. util/Theme.hpp gives the handful of inline C++ styles (status dot, offline banner, the eleven identical error-label styles across dialogs) named constants instead of a twelfth copy of the same hex. - UiThreadWatchdog: the M1 DoD's 200 ms debug-build watchdog. A background std::thread pings the UI thread every 50 ms via a queued invokeMethod and warns once (not per-poll) if a ping goes unanswered past 200 ms; no QThread, no Qt event loop of its own, so the watchdog itself can never be what blocks the thread it watches. No-op in a release build. Proven both ways in tst_uithreadwatchdog: fires on a genuinely blocked UI thread (synchronous sleep, no processEvents) and stays silent on a responsive one. Full non-conformance suite (55 tests across every lane, `ctest -LE conformance`) passes clean at this point, including the whole gui label under ASan+UBSan. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NSCdCWFXBTSBBK3MzWtJiC |
||
|
|
1fd2e0a0db |
gui: Options — Capture tab, saveTo.allowedRoots, lock coverage to the schema
Options was missing 8 of the 43 real settings.* keys: capture.enabled, monitoredExtensions, monitoredMimeTypes, minSizeBytes, excludedHosts, bypassModifier, autoStartTypes, and saveTo.allowedRoots. The capture.* keys were dropped in an earlier pass on the mistaken read that the spec's "File Types" tab meant per-category extension lists (which do live on Category, not settings.*) — they're real settings.* keys for a real daemon feature (the extension's auto-capture policy), so the tab exists now, named "Capture" to match what it actually configures rather than the spec's label. New tst_optionsdialog case (allKeysMatchesTheSchemaExactly) loads Settings.schema.json itself at test time and diffs its property set against OptionsDialog::allKeys() — this drifted silently once already, so the regression is now a build-time gate an unused import or a future key addition would trip, not something that needs re-discovering by hand again. Verified against a real veloxd (not just mockd): settings.get across all 43 keys, a settings.set/get round trip on a scalar (connection.timeoutSec) and on array-valued keys in the shapes OptionsDialog::currentValues() actually produces (capture.monitoredExtensions, proxy.bypassHosts, saveTo.allowedRoots), and event.settings.changed fanning out to a second subscribed client — all round-tripped and restored to their original values afterward. The real OptionsDialog widget also loads and renders correctly against that same daemon's live defaults with no crash under ASan+UBSan. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NSCdCWFXBTSBBK3MzWtJiC |
||
|
|
755d85964e |
gui: M1 DoD harness — scroll-60fps, rss-flat, unhappy-path
gui/docs/pkg-qa-requests-m1.md R3, filed by the previous session: three GUI M1 DoD items (10k rows at 60 fps, flat RSS over 10 minutes, --slow/--flaky/--drop-connection recovery) had nowhere to run in CI. This is the harness — `gui/tests/dod/run.sh <gate> [--json <path>]`, exactly the path/invocation contract tests/integration/README.md already specified — plus `gui/tests/dod/dod_harness.cpp`, the Qt/RpcClient-driven binary that actually runs each gate against a real mockd run.sh starts and tears down itself. - scroll-60fps: an eased scripted scroll over the whole loaded table, timing each step's synchronous repaint; p99 against a 16.6 ms budget (auto-scaled 4x under a sanitized build — ASan/UBSan overhead, not a loosened bar, see the harness's isSanitizedBuild()). - rss-flat: samples this process's own VmRSS at 1 Hz across the run, discards a warm-up window, checks post-warm-up growth against a stated 20 MiB slack. - unhappy-path: three phases (slow/flaky/drop-connection), each its own mockd instance; passes when the client reaches and holds Connected with no crash or hang. A watchdog (the harness's own QTimer, backstopped by run.sh's external `timeout`) turns a genuine hang into a bounded non-zero exit rather than needing the CI caller to timeout(1) around it. Every gate honours the exit-code and --json contract PKG/QA's pre-drafted CI job expects unchanged (one addition needed: the build step must also build the `gui-dod-harness` target, noted in the R3 update). No leaked mockd processes on any exit path (`trap cleanup EXIT INT TERM`); no writes outside a tempdir except the caller's own --json path. Verified live end-to-end (not just unit-level): all three gates run against a real mockd under the exact `ASAN_OPTIONS=detect_leaks=1:halt_on_error=1` `.github/workflows/ci.yml`'s sanitizers job already sets, all pass, and scroll-60fps was forced red once on purpose (VELOX_DOD_FRAME_BUDGET_MS=1) to prove the fail path and exit code actually work. Building this is also what surfaced the two RpcClient bugs fixed in the previous commit, and one real gap in mockd itself — --drop-connection never worked over the Unix socket transport (only WebSocket) — filed as gui/docs/proto-requests-m1.md since tools/mockd is PROTO's file. gui/docs/pkg-qa-requests-m1.md R3 and R4 (an unrelated, non-blocking Qt6::DBus CMake hygiene note filed while wiring the clipboard global-shortcut path) are updated with the concrete findings above. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NSCdCWFXBTSBBK3MzWtJiC |
||
|
|
c6f864ea30 |
gui: fix RpcClient double-free on stop() and the 1000-row list cap
Both found live while building gui/tests/dod's DoD harness, not from reading
the code — see that commit for how.
RpcClient::stop() left conn_ dangling after joining the worker thread: the
thread's own finish() flushes the DeferredDelete stop()'s
connect(&thread_, &QThread::finished, conn_, &QObject::deleteLater) already
posted, so conn_ is gone by the time stop() returns, but nothing cleared the
pointer. Any caller that calls stop() and later lets the client destruct
(the harness's own client.stop() at shutdown; also plain, correct API usage)
hit a double-free in the destructor's leftover `delete conn_`. Caught by
ASan on the very first run that actually exercised the stop-then-destroy
path.
requestInitialList() also called download.list with a hardcoded
`{"limit": 1000}`, silently capping the table at 1000 rows no matter how
many the daemon actually has — download.list.schema.json's own description
says "the GUI pages", not "the GUI takes it all in one call". The
scroll-60fps DoD gate refused to run against mockd --tasks 10000 rather
than "pass" against a 1000-row table, which is what surfaced it.
requestInitialList() now pages (5000 per call, the schema's own max) until
`total` is satisfied, then resets the model once with everything.
Separately: RpcConnection's session.subscribe list never included
event.settings.changed or event.grabber.progress, even though RpcClient has
carried signals for both since the Options/Grabber work — session.subscribe
"replaces the previous selection" and "nothing is delivered until this is
called", so both events were being silently dropped by any real daemon that
enforces the subscription (mockd does; verified live with a second
subscribed client actually receiving event.settings.changed after this
fix, round-tripped through a real veloxd's settings.set). GrabberWizard's
5 s poll fallback is exactly why this went unnoticed until now — it covered
for the missing push the whole time.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NSCdCWFXBTSBBK3MzWtJiC
|
||
|
|
0468b0176a | merge: lane/daemon | ||
|
|
d8b7c128be | merge: lane/proto | ||
|
|
4f6c0cc9d2 |
daemon: D3's remainder — rules.*, queue.reorder, schedule.*, limiter.*, download.update/refreshUrl
rules.list/rules.upsert: new store/rules.{hpp,cpp} (list in priority order; apply()
upserts+removes atomically, generating an id when absent, per the schema's own "never
leaves the table in a half-valid state"). Found and fixed along the way: migration
0001's rules table had no column for Rule.name at all — every rules.list/.upsert call
failed outright ("no such column: name"), unit tests included, since :memory: migrates
through the same path. Migration 0004 adds it.
queue.reorder: new store::Queues::reorder — taskIds must be an exact permutation of
the queue's current membership (compared as sorted sets) or nothing is written and
-32602 names the queue; a valid permutation rewrites every member's queue_position in
one transaction.
schedule.get/schedule.set: a thin wrapper over the queues.schedule column (already
read since D3b, never independently settable). nextRunAt is deliberately left unset —
computing it needs the same local-time, DST-aware window logic
sched/schedule_window.hpp's window_open() only has half of; called out rather than
approximated, and the field is optional.
limiter.get/limiter.set: backed by the same downloads.speedLimitEnabled/
downloads.speedLimitBps settings keys D9 already wired — one bag of truth, not two.
The new part is reaching the engine: EnginePort/TaskActionPort gain
set_global_speed_limit(bps) (0 = unlimited, TokenBucket's own convention), wired to
Engine::rate_limiter().set_global_limit(). Pushed live on every limiter.set *and* on
Scheduler::reload_config() so a limit from a previous run isn't silently unlimited
again after a restart. applyToRunning is accepted but has no lever to pull
differently — a single shared global bucket has no "next task only" variant. Also
found, not chased further: the schema's "globalBps:0 with enabled:true means 'stop
everything'" is the opposite of what TokenBucket does with rate_bps==0 (unlimited) —
a real discrepancy, but the schema says the GUI must not offer that combination.
download.update: "moving saveDir or filename moves the file on disk in the same
operation" — resolved and root-checked like download.add's destination, then the
.veloxpart/.veloxpart.meta pair (or the finished file, if complete) is moved via
rename, falling back to copy+remove across filesystems, only when the resolved
location actually differs. categoryId/queueId(appended to the new queue's run
order)/description/segments/bufferBytes/checksum apply through new
store::Tasks::apply_update.
download.refreshUrl: same async server-layer special-case as download.probe (a real
network round trip, same 30s deadline). Re-probes, flags contentChanged only when
size or validator are both known and actually differ, persists the new URL and probe
result, and swaps the URL on a live engine handle via a newly-widened
EnginePort::refresh_url (now takes headers too, matching DownloadHandle's real
signature — the seam had silently dropped them).
Found and documented, not fixed: the generated parser collapses "field absent" and
"field explicitly null" to the same nullopt for every optional<T> patch field
(DownloadUpdateParamsPatch, Settings) — both schemas document "an explicit null
clears the field" but neither handler can act on it because the wire distinction is
already gone by the time either sees the parsed struct. A generator-level gap
(PROTO's), not something to hand-route around locally.
Verified against real veloxd + tools/testserver: rules create/list, limiter.set
takes effect and reads back, schedule.set/get round-trips, queue.reorder against real
membership (and rejects a non-permutation), download.update renames+recategorizes a
task, download.refreshUrl swaps a paused task's URL and reports contentChanged
correctly. Full ctest: 55/55 (excluding the pre-existing, unrelated conformance
failure noted two commits back).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
|
e30d994d74 |
proto: fix conformance run.sh flakiness, prune the veloxd xfail list
Two run.sh fixes plus the xfail prune, all requested together:
1. VELOX_PAIR_AUTO=1 for the isolated veloxd. Pairing is the D1 dev stub
(EnvAutoApprover) and denies without it, so session.pair never issued
a token and the WS half of the veloxd step could never even connect.
2. WS_PORT was hardcoded to 52080 with no free-port search, so one leaked
mockd made every future run fail EADDRINUSE. free_port() binds :0 and
asks the kernel instead. The EXIT trap's stop() used `pkill -P "$pid"`,
which only reaps direct children — tsx's actual listener is often a
grandchild, which that missed and left holding the port. Every server
(mockd, slow mockd, veloxd) now launches under `setsid`, making it the
leader of its own process group, so stop() does `kill -TERM -"$pid"`
(a process-group kill) and reaches everything it spawned in one shot.
3. Pruned the xfail list now that D2, D4b and most of D3 have landed.
Pruning surfaced two more bugs than expected, both in the test harness
itself, not veloxd — worth recording since they were indistinguishable
from real daemon hangs until isolated:
- errors/session.hello.version-mismatch.json documents that the *server*
closes the connection after replying (correct, intended behavior). The
harness replays every fixture on one shared connection per transport,
so once this fixture ran, every later UDS fixture sent into the dead
socket and just sat there until its own timeout — including ones still
on the xfail list, which applyXfail waved through as "expected -32603"
regardless of the real reason. Fixed with a `closesConnection` fixture
flag: replay() reconnects (fresh session.hello) right after such a
fixture instead of leaving the rest of the run to time out one by one.
This is what was actually behind queue.*/session.*/download.remove
appearing to hang — none of them do; verified individually and via a
raw probe script before finding the real cause.
- category.remove.json (deletes the "firmware" category) sorted before
category.upsert.json (creates it) alphabetically, so it was failing
-32602 "no such category" against a fresh DB — never a daemon bug.
Added it to DESTRUCTIVE so it now replays after every other fixture.
Also fixed while verifying "confirm each really passes": download.addBatch.json's
`defaults.categoryId` was "compressed", a category nothing ever creates —
real veloxd correctly enforces the FK on tasks.category_id, so all three
batch items failed instead of the two expected. Changed to "programs" (a
migration-seeded builtin).
Of the 15 fixtures named for pruning, 10 turned out to cleanly pass and
are gone from the list entirely: download.pause/resume/start/cancel,
download.remove, download.addBatch, queue.upsert/stop, download.probe's
success path (D2, including errors/download.probe.probe-failed.json),
and category.upsert. Two do NOT cleanly pass and are kept, with reasons
rewritten to match what's actually happening now instead of the stale D3
text: download.probe.json (see below) and errors/download.provideAuth.not-found.json,
a real bug — on_download_provideAuth never checks the task exists, so an
unknown taskId gets a normal `{ok:false}` result instead of -32010.
Five more fixtures newly needed xfail entries to reach green, none of
them stubs:
- category.list.json — documented gap (deferrals.md's D3a note): the
categories table has no mimeTypes/sortOrder columns.
- download.probe.json, download.get.json, download.list.json,
session.hello.json — not bugs. Each golden depicts a richer lifecycle
state (a probed/in-progress download, a daemon with media/grabber/
Secret Service implemented) than this harness's bound tasks, which are
always fresh and never started, can produce. Optional/omit-if-absent
fields (effectiveUrl, requiresAuth, capabilities) are correctly absent;
the mismatch is against the golden's illustrative values, not the
contract.
- queue.start.json, category.remove.json — same class: startedTaskIds /
reassignedTaskIds are correctly empty because this run's queue/category
have no real membership.
`ctest -L conformance` is green: 100% (2/2), 81.7s (down from ~240s now
that pairing and the port/reconnect fixes remove the retries and the
5-10s timeouts the connection-death bug was producing).
One thing NOT fixed here, flagged for a follow-up decision rather than
touched mid-task: download.add.json's fixture is `startMode: "now"`
against a real, large (~6GB) Ubuntu ISO on the real internet, with
saveDir hardcoded to /home/sami/Downloads/Programs. Every run against a
real veloxd writes a real multi-GB file into that path — confirmed by
running this repeatedly during verification. Isolating the daemon's XDG
dirs doesn't isolate this. Worth its own change (startMode: "later"
would still exercise the add path without the transfer) but out of scope
for a fixture I wasn't asked to touch beyond what blocked this task.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01SFeUKLbdHizrJjLBeK7ffz
|
||
|
|
6632b75099 |
daemon: capture.offer for real (D7/D8) — rules, category resolution, dedupe, deadline
capture.offer applies capture.enabled/excludedHosts/monitoredExtensions/
monitoredMimeTypes/minSizeBytes from settings, then the rules table (new
store::Rules + CORE's vdm::rules::match_rules/glob_match — DAEMON only converts its
own stored proto::Rule JSON into CORE's plain vocabulary, per that header's own
layering note), resolves the category folder (a rule's explicit categoryId/saveDir,
else store::Categories::guess_by_extension — the same extension guess
download.probe's suggestedCategoryId already used, now shared instead of
duplicated), dedupes against active (non-terminal) tasks by exact URL, and on `take`
calls add_one() — the same path download.add itself uses, so a captured download is
a real, admitted, persisted task, not a special case.
The 750ms deadline (CLAUDE.md §4 / AGENT-DAEMON.md build step 6) is checked
cooperatively between every step via a new rpc::CaptureDataSource seam: the real
implementation wraps store::Settings/Categories/Rules/Tasks; a test fake can jump its
own injected clock forward to simulate "the store was slow just now" with zero real
sleep. This catches the realistic failure mode (several slow steps adding up) though
it cannot preempt a single pathologically stuck call mid-flight — a true preemptive
guarantee would need the same async/background-thread treatment as download.probe,
which isn't safe to do against the same sqlite3 connection (opened SQLITE_OPEN_NOMUTEX,
explicitly not for concurrent use) without a second connection; left as a known,
documented limit of this pass rather than adding that plumbing speculatively.
capture.getRules returns the same settings-backed fields capture.offer itself reads,
so the two can never drift. rulesVersion is a placeholder constant (1) — no persisted
revision counter exists yet, and the extension already re-fetches on
event.settings.changed regardless.
New store/rules.{hpp,cpp}: rules table CRUD (list, and an atomic upsert+remove for
rules.upsert later). store::Categories::guess_by_extension replaces a duplicate copy
that used to live in sched/scheduler.cpp. store::Tasks::has_active_duplicate for the
dedupe check.
Verified against real veloxd + tools/testserver: a monitored-type offer answers in
~5ms and actually creates + downloads the task, correctly categorized; an
unmonitored type, an excluded host, a rule-vetoed host, and a second offer for a
still-active URL all answer ignore with the right reason; a bad category save dir
surfaces its real -32011. New capture_offer_test covers all of the above plus the
deadline itself (two cases, one per "slow" checkpoint), asserting real wall-clock
time barely moves even though the fake clock jumped 2 simulated seconds. Full ctest:
54/54 (excluding the pre-existing, unrelated conformance failure noted in the
previous commit).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
|
e0edf7a084 |
daemon: file capture.offer/getRules in deferrals (D7/D8); close settings.get/set (D9)
Rebased lane/daemon onto main first — fast-forward, no conflicts (lane/daemon was already fully merged; main has since taken lane/gui, lane/pkg-qa, lane/proto). 1. capture.offer and capture.getRules were stubs, buried inside D3's generic stub list with no flag for how load-bearing capture.offer is: it's a CLAUDE.md §4 non-negotiable (750ms deadline, fails open) and AGENT-DAEMON.md build step 6, and against the real daemon it 500s to -32603 every time — the extension captures nothing, not "falls through to Firefox on a slow path." Split into their own D7/D8 rows so they stop being invisible. 2. settings.get/settings.set (D9, closed). Four pointer-to-member tables in dispatcher.cpp (bool / ranged int / plain string / string array), five enum-typed keys handled individually since parse_XXX already validates those — covers all 43 SettingKeys without ~40 repetitive hand blocks. store::Settings::kDefaults grew from 14 entries to 43 (the schema itself carries no "default" keyword anywhere, so these are hand-chosen — conservative for the ones ADR 0012 doesn't speak to; capture.monitoredExtensions defaults to the union of every builtin category's extensions rather than an arbitrary list of its own). settings.set validates every field before writing any of them: numeric min/max (hand-checked — the generated parser only checks JSON type, not schema constraints) and saveTo.* paths via fs::resolve_target/canonicalize_root (-32011) — allowedRoots entries checked as roots in their own right, defaultDir/tempDir checked as paths resolving inside the (possibly just-updated, same call) root list. Reports exactly the keys whose effective value actually changed, publishes event.settings.changed with that list, and calls the scheduler's reload_config() when a connection.* key took effect — live, not on next restart. Verified against real veloxd: all 43 keys round-trip with defaults, a keys subset filters, an out-of-range value rejects the whole call, a bad saveTo.defaultDir is -32011, allowedRoots+defaultDir set together cross-validate against the new roots, event.settings.changed fires over a live subscription. New dispatcher_settings_test covers the same ground without a socket. Full ctest: 53/53 (excluding a pre-existing, unrelated conformance failure — see below). Also found, not fixed (not this lane): tests/conformance/run.sh's live-veloxd leg fails "pairing failed: no token issued" reproducibly, isolated, on main before any of this session's changes — it launches the isolated veloxd without VELOX_PAIR_AUTO=1, so EnvAutoApprover denies every session.pair and the WS leg's session.hello never gets a token. tests/conformance/ is PROTO/QA-owned; flagging rather than editing across the lane boundary. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP |
||
|
|
f4a6cebb3e | merge: lane/proto | ||
|
|
963a76b6be | merge: lane/pkg-qa | ||
|
|
765c1701d4 | merge: lane/gui | ||
|
|
b578e6de1a | merge: lane/daemon | ||
|
|
ff721f4065 | merge: lane/core | ||
|
|
f2ee45f818 |
core: add a state dump to tools/bench load on task timeout
Per-task engine state, downloaded/effective_segments, every segment's own state, and the engine's SegmentBudget snapshot -- printed once, when a task times out, instead of needing to re-run the bench under a debugger or add throwaway instrumentation to find out why. This is what actually diagnosed the SegmentBudget over-admission bug fixed in the previous commit: the dump showed budget.active pinned at max_active_segments while multiple tasks sat starved, which is what pointed straight at confirm_slot()'s missing engine-wide check rather than a per-task target bug. Also updates the vdm_bench_load20 ctest registration's comment: the straggler under --preset tsan that motivated running it at reduced concurrency (docs/adr/0016's postscript) is now suspected to have been the same SegmentBudget bug (docs/adr/0017), not the TSan-timing artifact first guessed -- not reverified at the DoD's full shape under TSan in this change, so the reduced-concurrency registration stays for now. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
|
322a20efa5 |
core: fix SegmentBudget over-admission and add a wait-list wakeup
Root cause of the M7 RSS gap (core/docs/m7-baseline.md: ~70 MB measured against a 60 MB target): SegmentBudget::confirm_slot() only checked a task's own held count against its own target -- never the engine-wide active_ sum. reallocate_locked()'s two-pass fairness allocation does bound sum(target) <= max_active_ at the moment it computes a plan, but that bound says nothing about sum(held): a task can be legitimately holding more than its own just-lowered target for a while (yield is deferred to a segment boundary, never mid-segment -- ADR 0011 A1), and another task's target can correctly rise to claim that capacity before the first task has physically released it. Both confirm_slot() calls could then succeed against their own, individually-correct targets while sum(held) exceeded max_active_ -- tools/bench heap-profile caught this directly: budget.active reading 56-86 against a total of 32. confirm_slot() now also checks active_ < max_active_, unconditionally, as a backstop that doesn't depend on any task's target bookkeeping being in sync with what every other task holds. That creates a liveness question the original design never answered: a task denied only by this new check has a target that's already correct, so it never changes again and reallocate_locked()'s plain "fire a callback when a task's target changes" mechanism never revisits it. Task gained a waiting_for_slot flag, set on exactly this denial; release_slot()/deregister_task() (the only two places that free real capacity) now hand a freed slot directly to the highest-priority waiting task via wake_one_waiter_locked(), if reallocate_locked()'s own plan didn't already produce a callback for anyone. download_task.cpp's fill_slots_locked() needed a matching fix: a woken task's stalled segments (SegState::stalled -- backed off mid-retry, its own release_slot() already called) have no live worker and never surface through Segmenter::assign_slot(), which only hands out unassigned or fresh ranges. fill_slots_locked() now restarts any stalled segment with no live worker directly (bounded by slot_target, same as its assign_slot() loop) before looking for new work; a segment it doesn't get to keeps its own scheduled retry_worker() timer as a second chance. Also fixes a real TSan-caught data race this work surfaced: SegWorker:: speed_bps was written only by its own segment's curl callback and, before Progress.speed_bps's polled-path fix, only ever read from that same thread -- safe without synchronization. snapshot_progress() reading it from whatever thread calls DownloadHandle::progress() broke that invariant (workers_mu's shared_lock protects the workers map's structure, not an individual SegWorker's fields). Now std::atomic<double> with relaxed ordering -- an informational EMA, nothing synchronizes real state on it -- rather than adding a lock to the write side. core/tests/segment/budget_test.cpp adds two tests reproducing the actual gap (budget_active_never_exceeds_max_active_segments_under_concurrent_load, budget_wait_list_wakes_a_task_whose_target_never_changed) plus a sanity baseline (budget_release_wakes_a_denied_waiter), and introduces AsyncFakeTask + TestTimer for the one existing test that drives the budget from multiple concurrent threads -- mirroring production's real dispatch (register_task()'s on_target lambda posts through host.schedule(), download_task.cpp, never a synchronous call) rather than adding reentrancy-guarding machinery to SegmentBudget itself to compensate for a synchronous test double being unlike production. See docs/adr/0017 for the full writeup, including what an earlier version of this fix got wrong chasing a same-thread reentrancy hazard that doesn't actually exist in production. core/docs/m7-baseline.md updated: the RSS number now clears the DoD line (45.41 MiB via heap-profile), root-caused rather than just re-measured. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
|
7ea04fa79c |
core: add tools/bench heap-profile (no massif/heaptrack in this environment)
For the M7 RSS gap (core/docs/m7-baseline.md: ~70 MB measured vs ADR 0012's ~45-50 MB estimate) -- "find where before tuning anything". No root here to install massif or heaptrack, so this is a small in-process equivalent: operator new/delete (already overridden for alloc-check's counter) now also track, per call site (one return address via __builtin_return_address(0), cheap enough to run for a whole scenario), live (not-yet-freed) bytes. Runs the same 20-task concurrent scenario as `load`, waits for peak RSS to stop growing, then prints the top sites by live bytes -- resolved via one batched addr2line invocation against /proc/self/exe (dladdr alone only resolves dynamic-symbol-table entries, which misses most of this codebase's internal-linkage call sites), with dladdr as a per-site fallback. Also adds the nothrow operator new/delete overloads alongside the existing plain ones: without them, anything that allocates via the nothrow form (e.g. std::stable_sort's std::get_temporary_buffer, hit once while investigating the RSS gap) falls through to ASan's own default nothrow new while still being freed through this file's plain delete override -- an alloc-dealloc-mismatch ASan correctly flags as a real ABI-level bug in an allocator override that claims to intercept "everything". Using this tool, sum(effective_segments) across all 20 tasks vs. Engine::segment_budget().budget() (now cross-checked and printed together) showed the budget hands out well more than max_active_segments -- the actual root cause, in core/src/segment/budget.cpp, not covered by this commit. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
|
ef58796d22 |
core: fix Progress.speed_bps reading 0 on the polled path
DAEMON reported Progress.speed_bps reading 0 for the whole life of a live throttled download while downloaded bytes visibly advanced. DAEMON reads progress by polling DownloadHandle::progress() (engine_port_core.hpp), not the on_progress push callback. DownloadTaskState::snapshot_progress() -- the body behind progress() -- never set speed_bps, per-segment speed_bps, or eta_seconds at all; only the event-driven emit_progress_if_due() (which drives the on_progress callback) computed them, from the same live SegWorker::speed_bps EMA seg_data() maintains. snapshot_progress() now reads that same per-worker speed while building its segment list, so a segment with no live worker (idle, paused, complete, failed) correctly reports 0 and a segment with an active transfer reports its real EMA, matching emit_progress_if_due()'s math including the eta_seconds derivation. engine_polled_progress_reports_nonzero_speed reproduces the bug (fails without the fix, confirmed) by polling .progress() -- the same path DAEMON uses -- during a throttled download and asserting speed_bps > 0 once real progress has accumulated. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
|
cf9e226e61 |
daemon: D1 — checked, not attempted; document why
libdbus-1-dev / libsystemd-dev have no headers installed in this build environment (only the runtime .so's — apt-cache policy confirms libdbus-1-dev is available but not installed). A real org.freedesktop.Notifications-backed PairingApprover needs one of those linked into veloxd, which is a new build dependency for daemon/CMakeLists.txt and, since packaging manifests would need to know about it too, a decision to surface rather than reach for silently mid-session. PairingApprover::approve() is also still synchronous by shape — its own doc comment already says the real approver "will run async and is not this shape." The async pattern this session built for download.probe (rpc::TaskActionPort + the server-layer deferred-reply special-case in uds_server.cpp/ws_server.cpp) is the right shape to reuse once there's a real implementation to justify reshaping the interface; doing that with nothing behind it yet would just be churn. Left EnvAutoApprover in place rather than hand-roll a D-Bus wire client to route around the missing headers — a broken pairing approver is a worse outcome than an honest, already-documented stub. Findings recorded in deferrals.md for whoever picks this up once the dependency is available and approved. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP |
||
|
|
c89158ea09 |
daemon: D3 (partial) — category.upsert/remove, queue.upsert, download.remove/addBatch/provideAuth
Closes a bounded, high-value slice of the remaining D3 stubs. settings.*/rules.*/ limiter.*/schedule.*/grabber.*/media.*/capture.*/queue.reorder/download.refreshUrl/ download.update stay deferred — reasons noted individually in deferrals.md (settings.* specifically: a real, large field<->key<->JSON-type mapping table across ~43 fields / ~50 SettingKeys, not started rather than rushed). category.upsert/remove: store/categories.hpp gains get/upsert/remove. upsert generates an id when absent and always ignores the payload's `builtin` (preserved from the existing row on replace, false on create); saveDir goes through the same fs::resolve_target canonicalize-and-root-check as download.add. remove refuses a builtin at both layers (dispatcher's -32602 pre-check; the store's own "DELETE ... AND builtin = 0" as defense in depth) and reassigns member tasks to reassignTo (default "general") inside one transaction before deleting the row. queue.upsert: store/queues.hpp gains get/upsert (set_state already existed from D4b). Same create-generates-id pattern; a create always starts 'stopped', a replace keeps the queue's current run state (upsert edits config, not run state — that's queue.start/stop). Also fixed in passing: on_complete has been a real column since migration 0001 but Queues::list/get never projected it onto Queue.onComplete. download.remove: cancels with discard_partial=true (always drops the .veloxpart pair — unlike download.cancel, which keeps them, the row is gone either way), deletes the finished file only when deleteFile is true and the task was complete (best-effort), deletes the row (segments cascade via the FK), and publishes event.task.removed (closing the last open note under D5). download.addBatch: on_download_add's body is now a shared add_one(), called once per item after merging each item's unset fields against params.defaults. download.provideAuth: forwards to EnginePort::provide_auth via a new TaskActionPort::provide_auth. `remember`/persisting to the Secret Service is accepted but not acted on — nothing in this build talks to libsecret yet. Verified against real veloxd + tools/testserver: category create/replace/ remove-with-reassignment, queue create/replace-keeps-run-state, a batch add sharing defaults.saveDir, and download.remove with deleteFile actually deleting the file and the task then 404ing download.get with -32010. Full ctest: 39/39. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP |
||
|
|
4f6fb1029d |
daemon: D2 — download.probe, genuinely async on both transports
download.probe was a stub (-32603). It needs an HTTP round trip on the engine's probe
pool (up to the schema's 30s x-deadlineMs), which cannot fit VeloxDispatcher's
synchronous on_download_probe -> HandlerResult<T> return without blocking the RPC
loop for the duration — a hard no per CLAUDE.md ("never block the RPC loop") and
AGENT-DAEMON.md build step 1.
uds_server.cpp and ws_server.cpp special-case "download.probe" before the generic
dispatch(), exactly the way they already special-case session.hello/session.subscribe:
parse the params, call the port, and queue the reply whenever the callback fires
(dropped silently if the connection is gone by then).
rpc::TaskActionPort gains probe_now(DownloadProbeParams, callback) — kept in proto/std
terms, no vdm::net::* in the signature, so veloxd_rpc never needs core/include's vdm
headers just to declare this. sched::Scheduler::probe_now is the implementation:
builds a vdm::net::ProbeRequest, runs it on the engine's probe pool, marshals the
engine-thread callback back onto the loop (deps_.post_to_loop, same as every other
engine callback here), maps a probe failure to -32013 ProbeFailed (data.httpStatus set
when there was an HTTP response), and fills suggestedCategoryId/suggestedSaveDir with a
plain extension match against the categories table — not the real rules engine, which
is still D3; noted in a comment.
Verified against real veloxd + tools/testserver, not just unit tests: a real probe
answers in ~5ms with size/resumable/etag/redirect chain; a bad host maps to -32013;
and — the actual point of the async design — a connection running a 10s slow-loris
probe does not block a second connection's download.list, which answers in ~1ms while
the probe is still outstanding.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
|
55f0c6099d |
daemon: D4b — download.pause/resume/start/cancel and queue.start/stop drive the scheduler
download.pause/resume/start/cancel and queue.start/stop were stubs; now they call into the scheduler and take effect immediately, not on the next 1s tick — pausing, resuming or cancelling a live transfer can't wait, and per ADR 0013 §3 the governor never touches a user-owned pause on its own. New rpc::TaskActionPort interface (owned by rpc/, implemented by sched::Scheduler) is what dispatcher.hpp depends on instead of sched/scheduler.hpp directly. Needed because veloxd_sched already links veloxd_rpc (for EventHub); dispatcher.hpp pulling in sched/scheduler.hpp directly would make it a real circular library dependency, breaking anything that links veloxd_rpc alone (cli's tests, as it turned out — hit and fixed during this change). Scheduler::user_pause/user_resume/user_start/user_cancel + pause_queue follow tick()'s existing to_pause pattern: call the engine (async, no synchronous effect) and transition the store eagerly so download.get/list are correct the instant the RPC call returns. Fixed a real bug surfaced while building this: transition() always overwrote pause_reason to NULL when the engine's own delayed pause-ack callback (on_state to paused, no error) arrived after whoever actually initiated the pause had already written the real reason — now it preserves the stored reason when the callback supplies none, instead of clobbering it. Covered by a regression check in sched_scheduler_test. store/queues gets get() and set_state() (was list()-only) for queue.start/stop. Verified against real veloxd + tools/testserver, not just unit tests: pausing a live single-segment throttled transfer freezes downloadedBytes, resume continues it from that point, cancel stops it; a bad taskId comes back in BulkTaskResult.failed with -32010, not a crash; queue.stop(pauseRunning:true) pauses the queue's running task immediately and queue.start resumes admission. Known gap: download.start's contract "a task in 'queued' jumps its queue" (priority bump) is not implemented — admission is still plain FIFO by created_at. Noted in deferrals.md. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP |
||
|
|
de83ee3cce |
gui: Options, Scheduler, Speed Limiter, Batch, Grabber, and the tray icon
Continues the build order past the Add-URL/File-Info/Progress dialog flow.
- OptionsDialog: General/Save To/Connection/Downloads/Proxy/Sounds tabs, every
control bound to a real settings.* key (contracts/schema/types/Settings.
schema.json). The spec's File Types and Site Logins tabs have no settings.*
backing (categories go through category.upsert, credentials through the
Secret Service) so they don't exist here — a tab either binds to a real key
or isn't shipped. diffChanged() sends only what actually changed, matching
settings.set's "changed[] names exactly what took effect" contract.
- SchedulerDialog: per-queue schedule (schedule.get/set) plus maxConcurrent/
onComplete (queue.upsert), Start Now/Stop. Queue.schema.json already carries
the schedule so queue.list alone seeds the window.
- SpeedLimiterDialog: the live global limiter (limiter.get/set) — a different
thing from Options' downloads.speedLimit* default. buildParams() enforces
the schema's "0 with enabled true must not be offered".
- BatchDialog: clipboard-blob and {start..end}-wildcard tabs sharing one
category/queue/start-mode footer into download.addBatch.
- GrabberWizard: 4-step QWizard (project label -> start URL/depth/filters ->
file-type filter -> review), grabber.start feeding a poll+event.grabber.
progress-driven review page, Finish = grabber.harvest for the checked files.
- TrayIcon: active-count tooltip, Show/Add URL/Pause All/Resume All/Speed
Limiter submenu/Quit. Quit only closes the GUI — there is no RPC to stop
veloxd itself, filed as a new gap in daemon-requests-m1.md. MainWindow now
also hides to tray instead of closing when general.minimizeToTray is set.
Every dialog's non-widget logic (diffChanged, buildSchedule, buildParams,
parseUrlBlob/expandWildcard/buildAddBatchParams, buildFileTypes/
buildStartParams/buildHarvestParams) is a static pure function with its own
test, same shape as FileInfoDialog::buildSpec from the previous round.
Verified end-to-end against a running mockd under ASan+UBSan: all five
surfaces render real data (settings.get values, queue.list's two seeded
queues, limiter.get, a live grabber.start/status crawl returning 3 files) with
no sanitizer reports. gui-check (non-ASan) and dev (ASan+UBSan) presets both
build the whole repo clean; all gui-labeled ctest targets pass.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NSCdCWFXBTSBBK3MzWtJiC
|
||
|
|
39c69f3871 |
gui: rpc plumbing for settings.changed and grabber.progress
Protocol.hpp gains method-name constants for every RPC the next batch of dialogs needs (settings.*, limiter.*, schedule.*, queue.*, download.addBatch, grabber.*) and RpcClient re-broadcasts the two events nothing consumed yet: event.settings.changed and event.grabber.progress. No behaviour change on its own — OptionsDialog, SchedulerDialog and GrabberWizard are what actually call these, landing next. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NSCdCWFXBTSBBK3MzWtJiC |
||
|
|
efe76c319d |
pkg: nightly integration run — real veloxd + testserver, 50 concurrent
Adds tests/integration/nightly_run.py, wired as the nightly-integration job in ci.yml (schedule + workflow_dispatch). Real veloxd + tools/testserver, 50 concurrent downloads mixing flaky-reset/throttled/no-range/plain, asserting: every completed file's SHA-256 against testserver's own /sha256/ route (never trusting veloxd's own success claim), every task reaching a terminal state inside the timeout, and veloxd's own open-FD count settling back to baseline. veloxd runs isolated (XDG_RUNTIME_DIR/XDG_DATA_HOME/XDG_CONFIG_HOME under a fresh mkdtemp — not the session scratch dir, whose path overflows AF_UNIX's sun_path). saveTo.allowedRoots is seeded directly into velox.db after a migrations-only warm-up start, since settings.set returns -32603 'not implemented in this build' on the veloxd this job builds (verified live). Every task gets its own filename override on download.add: tasks sharing (mode, size) share a URL, and without distinct filenames they raced each other's rename on the first real run (48/50 'passed' with io_errors and checksum mismatches on the collided tasks) before this fix. Every assertion was forced red once on purpose and the transcript recorded in tests/integration/README.md, per this repo's history of green checks that didn't look where the bug was. Every spawned child (veloxd, testserver.py) gets PR_SET_PDEATHSIG plus its own process group, so a hard-killed harness can't strand a daemon the way a prior run did (4h40m under systemd --user, because SIGKILL never reaches a finally: block). Proven by kill -9'ing a running harness mid-download and confirming both children exit with it. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01RBPR7iM3YPyxrjWsVtZDPJ |
||
|
|
de748cc2fc |
daemon: fix startMode 'later' -32603 and isolate the single-instance lock by runtime dir
Two bugs blocking PROTO's live-veloxd conformance check.
1. tasks.start_mode's CHECK was ('auto','now','queue','manual') — not the
contract's StartMode enum (['now','later','queue']) at all. 'later', a real
documented value (the File Info dialog's Download Later button), hit the CHECK
on every insert and surfaced as an unhandled -32603; 'auto'/'manual' were
never contract values to begin with.
Migration 0003 rebuilds tasks (SQLite can't ALTER a CHECK) with the contract's
values, remapping existing rows by what they actually meant: 'auto' -> 'now'
(eligible for the scheduler immediately), 'manual' -> 'later' (parked, matching
StartMode's own "lands the task in paused" description). store_migrations_test
covers the remap and that 'later' inserts clean while the retired spellings
are rejected.
dispatcher.cpp's on_download_add matched: default (absent startMode) is now
'now' instead of the invented 'auto'; 'later' actually lands the task in
`paused` (pause_reason 'user') instead of a dead 'manual' -> `new` branch that
spec.startMode (typed as the 3-value enum) could never even reach.
TaskRow::start_mode's in-memory default followed suit ('now').
2. main.cpp's single-instance guard bound an abstract socket named
"velox-daemon-<euid>" — one name per user, system-wide. XDG_RUNTIME_DIR
isolation never reached it: a leaked test veloxd held the lock for 4h40m and
locked out every other isolated instance with the same euid (PROTO, EXT, the
orchestrator), real daemon included.
Extracted rpc/single_instance.{hpp,cpp} (was a static in main.cpp, untestable)
and derived the abstract-socket name from a hash of the resolved runtime dir
path instead of euid alone. The real per-user daemon is still unique (its
runtime dir is unique to it); isolated instances pointed at their own runtime
dirs now coexist. main() resolves the runtime dir before acquiring the lock
(was the other way around). New single_instance_test covers same-dir refusal,
different-dir coexistence, and release-on-close.
Verified against real veloxd binaries, not just unit tests: startMode: "later"
via a live download.add lands in `paused`; two veloxd with different runtime
dirs run concurrently, two with the same one and the second refuses with the
runtime dir named in the error. Full ctest: 39/39.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
|
67aefbce4b | merge: lane/ext | ||
|
|
4df4d7ad08 | merge: lane/gui | ||
|
|
f73a57a9ad | merge: lane/daemon | ||
|
|
c710481018 | merge: lane/core | ||
|
|
d4ad48d494 |
core: stage 9 — rules/ (filename sanitization, collision policy, rule matching)
Pure functions only, per AGENT-CORE.md's build order: no I/O, no JSON, no SQL,
no notion of the wire Rule type — DAEMON decodes its own stored/wire
representation into these plain structs and calls in.
- rules/filename.hpp: sanitize_filename() turns a raw candidate (from
net::parse_content_disposition or net::url_filename — neither is
filesystem-safe by design; both headers say so and point here) into one
safe to create on ext4/APFS/NTFS: strips separators and control bytes,
folds NTFS-illegal characters, neutralizes reserved Windows device names,
clamps length on a UTF-8 boundary. Total on hostile input; never empty.
Not the path-traversal security boundary — that's daemon/fs/safepath,
downstream of this and the one that actually matters adversarially.
- rules/collision.hpp: resolve_collision() finds the next free name
Explorer/Finder-style ("name (1).ext", ...) given an existence predicate,
or returns the desired name unchanged under an overwrite policy. Never
fabricates a guaranteed-unique name past its attempt bound — hands back
the last candidate tried rather than hiding a persistent collision.
- rules/match.hpp: match_rules() is the evaluation half of
contracts/schema/types/Rule.schema.json — priority order, first rule
whose present match clauses (extensions/mimeTypes/host & url glob/size
bounds) all hold, wins; a size clause never matches speculatively before
the probe fills in size_bytes. glob_match() is the iterative (not
recursive — bounded work on an all-'*' pattern) matcher both host_pattern
and url_pattern use.
Every header compiles standalone; tests (39 cases) pass under ASan+UBSan and
TSan. core/include/vdm/README.md documents the new public surface.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
|
||
|
|
b60d4e6f5b |
core: add tools/bench (throughput/load/alloc-check) and record the M7 baseline
Three subcommands in one binary, driving vdm::Engine directly (docs/04 §8): - throughput: a single download against a fast local origin (support/local_server.hpp, busybox httpd), reporting Mbps/CPU%/RSS. Gates on --require-mbps/--max-cpu-pct only when passed, so the ctest smoke registration stays a correctness check, not a hardware-dependent perf gate -- the real 1-Gbit-link sign-off is a manual/CI job (see the file's header comment). - load: N concurrent tasks against tools/testserver's `throttled` mode (support/testserver_client.hpp), reporting peak RSS via getrusage(). Paced externally rather than through the engine's own rate::RateLimiter or busybox: the limiter's pause/resume path allocates on every throttle event (would contaminate alloc-check's measurement) and under heavy segment contention was found to starve individual tasks indefinitely (see docs/adr/0016, added here); busybox couldn't sustain the DoD's ~160 concurrent connections (20 tasks * default_segments=8) reliably. The ctest registration runs at reduced concurrency under sanitizer presets -- see the CMakeLists.txt comment and the ADR's postscript. - alloc-check: operator new/delete overridden process-wide, sampling the allocation count across a steady mid-transfer window against a paced tools/testserver origin. Caught a real bug in the same change (see the http_client.cpp commit) and, by dropping its Engine mid-download to end cleanly, also surfaced the quiesce() use-after-free (see that commit). core/docs/m7-baseline.md records actual measured numbers against the M1/M7 DoD lines, including where they don't clear yet (RSS ~70 MB vs a 60 MB target; throughput/CPU only measured on loopback, no 1 Gbit link available here) rather than rounding them away. docs/adr/0016 documents a rate::RateLimiter fairness gap found building the load subcommand: a single shared TokenBucket under heavy segment contention has no fairness ordering across its peek/commit race and can starve a waiter well past what its configured rate implies. Filed as a follow-up (it's a core/src/rate design question, not a tools/bench one) rather than fixed here, along with a related TSan-only load-test straggler that could not be root-caused in this environment. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
|
6163898c14 |
core: stop allocating an empty deque on every worker-loop iteration
HttpClient::Impl::drain_commands() default-constructed a std::deque<Command> every call, on every iteration of the worker thread's event loop (once per curl_multi_poll wake -- i.e. once per socket-readiness event on the transfer hot path), then swapped the (usually empty) command queue into it. In libstdc++, an empty std::deque still allocates its map array on construction, so this was a real allocation on the hot path regardless of whether any command (add/pause/resume/cancel) was actually pending -- which is the common case, since those are rare next to data arriving. Found via tools/bench's alloc-check, which is built in this change and exists specifically to catch this class of bug (AGENT-CORE.md: "no allocation in the curl write callback... checked in review and by a bench assertion"): before this fix it reported thousands of allocations/sec under a sustained transfer; after, single digits. Fixed by checking `w.queue.empty()` under the lock before touching `local` at all, so the deque is only constructed when there's actually something to swap into it. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
|
ef816c21fb |
core: fix quiesce() racing an in-flight write callback (ASan use-after-free)
DownloadTaskState::quiesce() (Engine shutdown / ~Engine, via quiesce_task()) cancelled every live worker's transfer, then immediately cleared `workers` on the calling thread. transfer.cancel() only *requests* the HttpClient worker thread stop the transfer -- it does not wait for that to happen. If that thread was mid write-callback (seg_data -> WriteBuffer::append -> SparseFile::write_at), clearing the map destroyed the SegWorker (and its ring buffer) it was still writing through: a heap-use-after-free, caught by ASan via tools/bench alloc-check, which by design drops its Engine while a download is still active mid-sample. Every other exit path (verify/fail/auto_pause/demote, via begin_drain_locked) already gets this right: cancel, then let each worker remove and flush itself through seg_finished once HttpClient actually confirms the transfer stopped, on the correct thread. quiesce() now does the same instead of tearing the map down itself -- wait on a condition variable, notified from seg_finished right after it erases, until `workers` is empty. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
|
c99d1d9701 |
core: drain-aware hostile-mode handling (etag/416/lying-ranges/mismatch)
Resumes work left mid-session on the M1 hostile-mode matrix. download_task.cpp: - Replace the old cancel-then-clear teardown (cancel_all_transfers_locked / start_assembly_locked) with a single begin_drain_locked()/PendingAction mechanism: cancel every live worker, remember what to do (verify / fail / auto_pause / demote), and let whichever worker's seg_finished finds the worker map empty carry it out. Every sibling still flushes its buffer on the way out, so no buffered-but-unflushed tail is lost when a download finishes or fails while other segments are still mid-transfer. - A 200 where 206 was expected (wrong_status) now checks the response's ETag/Last-Modified against the probe's: a real mismatch asks the user (server_file_changed, "ask, never silently corrupt" -- docs/04 §5); a match means the server just stopped honouring Range for this connection, so demote to one segment and keep going without a round trip (docs/04 §7). - 416 mid-download (stale range metadata) now surfaces as a decision instead of retrying the same now-invalid range to exhaustion. - do_decide's abort path surfaces the actual reason a decision was asked for (last_error) instead of hardcoding server_file_changed, which was mislabeling a 416 abort. engine_test.cpp adds the four hostile modes where a bug means silent corruption rather than a visible failure: etag-changes, 416-always, lies-about-accept-ranges, content-length-mismatch. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
|
5b03eff926 |
proto: wire veloxd into the conformance suite's canonical entry point
run.sh's TS runner only ever started mockd; veloxd existed but nothing in
the ctest -L conformance path touched it, so mockd's always-valid fixtures
were the only thing ts/replay.ts ever saw. That let a real bug through:
veloxd's download.get can return segments: 0, which TaskSummary.segments
forbids (minimum 1, required) — nothing caught it.
Add a 3b step to run.sh (still the one canonical entry, per ADR 0014):
builds veloxd, starts it isolated (its own XDG_RUNTIME_DIR/XDG_DATA_HOME/
XDG_CONFIG_HOME), seeds saveTo.allowedRoots/defaultDir directly into the
isolated velox.db (settings.set is itself a stub, and the default
~/Downloads root doesn't isolate download.add's writes), then replays
every fixture against it over both transports.
Most handlers are still stubs (daemon/docs/deferrals.md D1-D4b). Fixtures
that hit them get an expected-failure entry in the new veloxd-xfail.json,
loaded by replay.ts's new --xfail flag. This is a maintained allowlist,
not a snapshot: a listed fixture that unexpectedly *passes* is flipped
back to a failure (applyXfail), so the list can only shrink as DAEMON
lands handlers, never rot into a list nobody rechecks. download.get's
segments: 0 is deliberately *not* on it — that's the regression this
step exists to catch.
Also hardened setupBindings: a server that can't even complete fixture
binding used to take the whole runner down with an uncaught exception
before a single fixture was checked. It's now a reported Outcome instead,
so the run still produces a coherent report. That robustness fix earned
its keep immediately: veloxd's download.add crashes on startMode
"later" (a valid, documented StartMode — "the File Info dialog's
Download Later button") with a SQLite CHECK constraint violation, because
migrations/0001_initial.sql's start_mode CHECK never had 'later' added to
it (and includes 'manual'/'auto', neither a contract value). That's a
second, more severe bug this wiring found, unrelated to segments: 0 and
currently blocking most of the veloxd run — filed for DAEMON in
tests/conformance/README.md, not fixed here (out of lane). capture.offer
and capture.getRules are also stubs but missing from deferrals.md's
D-list; xfailed with a note asking DAEMON to add the row.
Verified live once against a real, isolated veloxd before this session's
sandbox became persistently contended for veloxd's single-instance lock
(UID-scoped, not namespaced by XDG_RUNTIME_DIR — daemon/src/main.cpp;
documented as a caveat in the README): it built, started isolated, seeded
settings, connected over both transports, and surfaced the startMode bug
above as a real, non-xfailed failure — confirming the whole pipeline
including --xfail end to end. segments: 0 is confirmed by direct reading
of daemon/src/store/tasks.{hpp,cpp} (TaskRow::eff_segments defaults to 0,
copied verbatim into TaskSummary.segments) rather than by a second live
run reaching that specific fixture, since setup itself fails first on the
startMode bug above. mockd path re-verified green after these changes
(200/200, up from 196/196 — the new setup/$taskId outcomes are visible
and passing).
Recommendation for PKG: don't flip this required yet. The existing
`conformance` ctest entry is already a required check, and right now the
startMode bug fails most of the veloxd run, not just the one expected
segments: 0 case — merging as-is would block every lane's PRs on two
DAEMON bugs at once, one of them unrelated to what this task set out to
catch. Required once DAEMON lands a fix for startMode "later" at minimum;
segments: 0 can stay red for a while by design, same as any other tracked
regression.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01SFeUKLbdHizrJjLBeK7ffz
|
||
|
|
03b6253b5a |
ext: content/ media detection (build order step 7)
Two independent signals feed one panel, per docs/05 §3: - capture/media.ts (background): webRequest-based, recognizes .m3u8/.mpd URLs and the HLS/DASH content types, deduped per (tab, url) so an HLS live-refresh doesn't re-fire. media-bridge.ts relays a hit to the tab's content script over runtime.sendMessage, and separately answers the content script's media.listVariants/media.addVariant calls by forwarding them to the background page's transport. - content/media-observer.ts: watches the page's own <video> elements (present now, added later, or with src changed) for the same extension signal, independent of what the network sniffer saw. Either firing opens content/video-panel.ts's "Download this video ▾" panel (built with createElement, matching the popup's innerHTML-free approach), which lists variants from media.listVariants and greys out any variant.drm or a wholly drmProtected manifest with "Protected content" rather than attempting it. The extension still never parses a manifest itself — that stays in the daemon, one language, one place. content/index.ts is the manifest-registered entry (content_scripts in manifest.json, added in the previous commit); build.mjs builds it as an IIFE rather than ESM, since a manifest content script has no "type": "module" declaration and an emitted top-level export would be a syntax error there. tsconfig.json adds DOM.Iterable for NodeList iteration. docs/05-extension-spec.md gets a short addendum (§8) documenting the popup/options bridge and this media-detection split, since neither was in the original design write-up. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Ed8KEmAW48v4YHdxLtqsMB |
||
|
|
55932a2e11 |
ext: popup, options, and the panel bridge (build order step 6)
Popup and Options are separate documents from the background page and can't reach its live VeloxTransport directly, so background/bridge.ts relays it over one browser.runtime.connect port per document (call/subscribe/getStatus/reconnect/pair/unpair/setOverride in; result/event/status/pairError out). shared/panel-client.ts is the client side both surfaces use. Popup (src/popup/): status dot + text, active-downloads list driven by event.task.progress/added/state/removed (repaints ride the event's own <=4 Hz cap rather than adding a second timer), pause/resume buttons, "Start it" wired to a reconnect request. State lives in a DOM-free store.ts for unit testing; rendering uses createElement, not innerHTML (web-ext lint flags the latter). Options (src/options/): transport override select, pairing (code entry + pair/unpair, backed by two new WebSocketTransport methods, pairWithCode/unpair), and the daemon's capture policy mirrored via capture.getRules. The capture-policy form is editable only when the active transport is native messaging (uds) -- settings.set and rules.upsert are privileged, uds-only methods per shared/protocol METHODS, so WebSocket can't write them no matter what the page shows; CLAUDE.md section 2 rules out working around that locally. The bridge's status payload adds a kind field (which transport is live) for this to key off. Default category is the one piece of state that's genuinely the extension's own, not the daemon's, and lives in browser.storage.local via options/prefs.ts. Pure decisions (statusLine, pairingAvailable, captureRulesEditable) are split into view.ts for unit testing without a DOM. manifest.json registers the popup action and options_ui page (and, in the same edit, the content_scripts entry the next commit's media detection needs -- split by file, not by manifest line). build.mjs gains popup/options as further esbuild entry points, plus copying their static HTML/CSS into dist/. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Ed8KEmAW48v4YHdxLtqsMB |
||
|
|
25c171f742 |
ext: pairing-restart test + AMO permission justification
tests/transport/storage.test.ts covers the storage half of "the pairing token survives a browser restart" (round-trip, unpair-clears, corrupted override falls back to auto). websocket.test.ts adds the transport half: a fresh WebSocketTransport instance over the same backing store reuses the persisted token with no re-pairing, plus pairWithCode/unpair coverage. "Wrong token rejected and rate-limited" was already covered (websocket.test.ts's NotPaired/RateLimited cases). docs/amo-permissions.md is the submission-ready permission justification for AMO's Notes to Reviewer field, covering every permission in manifest.json plus what was deliberately not requested and how cookie/ header data is handled. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Ed8KEmAW48v4YHdxLtqsMB |
||
|
|
2cb1959bff |
ext: no-download-logic ESLint gate in the extension-lint CI job
CLAUDE.md §3's rule was prose only for extension/ (GUI already has gui_no_download_logic as a ctest). eslint.config.mjs adds a no-restricted-syntax/no-restricted-globals rule banning fetch/XHR/Request, ReadableStream.getReader, Range/Content-Range header construction, and IndexedDB in src/**/*.ts. Verified red on a planted violation (fetch + Range header + stream reader) and green on ordinary code; that check is now a permanent regression test (tests/lint/no-download-logic.test.ts) rather than a one-off manual run. Wired into the existing extension-lint job in .github/workflows/ci.yml, ahead of web-ext lint. Generated protocol code (src/shared/protocol/**) is excluded from lint entirely — it must never be hand-edited, so flagging it as fixable would be a lie. Answers gui/docs/ext-requests-m1.md. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Ed8KEmAW48v4YHdxLtqsMB |
||
|
|
a967eca669 |
daemon: persist engine progress/probe to the store — segments:0 no longer leaks
Engine numbers never reached the store: download.get after a correctly-finished
download reported sizeBytes: null, downloadedBytes: 0, speedBps: 0, resumable:
false, segments: 0, segmentDetail: [] — segments: 0 breaks the frozen contract
(TaskSummary.segments is minimum:1, required).
- Scheduler::tick() now probes (EnginePort::probe) before every start(), persisting
sizeBytes/resumable/validators via Tasks::set_probe_result before a byte moves,
then starts the engine with that ProbeResult as probe_hint.
- Scheduler::persist_progress() (new) writes downloadedBytes/speedBps/segments/
segmentDetail from the engine's Progress. Called from progress_snapshot() (the
250ms tick) *and* once more from on_engine_state right before release()/unmap on
every terminal transition, so a task that finishes between two ticks — the common
case for anything small or fast — still leaves real numbers instead of the
pre-persistence defaults.
- TaskSummary.segments is sourced from segments.size() when the task has any
(matching what actually lands in segmentDetail, per the schema's "exactly
segments entries"), falling back to the engine's effective_segments (budget
slots *held*, not necessarily physical range count) only pre-segmentation.
- Tasks::set_final_bytes tops up on_finished's byte count as a last-resort
backstop.
- store/segments.{cpp,hpp}: read/write access to the segments table behind
TaskDetail.segmentDetail. Wired into daemon/CMakeLists.txt.
- migrations/0002: speed_bps on tasks and segments; fixes segments.state's CHECK
to include 'pending' (0001 omitted it, so a pre-connect snapshot could never be
written).
- store_migrations_test's forward-only loop faked "released version N" by setting
the user_version pragma alone, with no real schema underneath — never exercised
until 0002 existed. Fixed to actually build the db through migrations 1..N first.
Verified against real veloxd + tools/testserver (not just unit tests):
download.list/download.get correct immediately after completion and after a
daemon restart, with saveTo.allowedRoots pointed at an isolated dir.
Observed but not fixed (CORE, not this lane, noted in deferrals.md): Progress.
speed_bps reads back 0 for the whole lifetime of a live throttled download in the
same E2E check, despite downloadedBytes visibly advancing. DAEMON passes it
through unmodified; filed rather than worked around.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
|
a71d904a1f |
gui: finish Add URL -> File Info -> Progress dialog flow
Wires up the three dialogs from build order step 5 (docs/03-gui-spec.md §§2-3) and the MainWindow slots that were declared but never implemented: - AddUrlDialog: clipboard prefill is the explicit path docs/06 R2 calls for (no passive monitoring, not advertised). - FileInfoDialog: async download.probe never blocks the UI; ends by calling download.add itself (Now / Later / Add to Queue). buildSpec() is a pure static so the optional-field-omission logic is unit-testable without touching a widget. - ProgressDialog: non-modal, WA_DeleteOnClose, driven by the taskProgress/ taskStateChanged signals RpcClient already re-broadcasts; download.get seeds state once for a dialog opened mid-transfer. Hosts SegmentBarsWidget and SpeedGraphWidget. MainWindow: openAddUrlDialog/openPropertiesForSelection/showTableContextMenu now have bodies; category.list/queue.list responses are cached so File Info can populate its category combo and queue menu without a second round trip. The row context menu covers what already exists (Resume/Pause/Stop/Delete/ Properties) and deliberately leaves out Open/Open With/Move-Rename/ Redownload/Add to Queue — those need dialogs later build-order steps haven't reached yet. util/Format.hpp: pulled the bytes/rate/eta formatting out of MainWindow and DownloadTableModel once the dialogs wanted the same strings a third time. Verified end-to-end against a running mockd (category.list/queue.list, download.probe, download.add, download.get, and live event.task.progress/ event.task.state) under ASan+UBSan: all three dialogs render correctly against real fixture data and the flow runs clean with no leaks or sanitizer reports. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NSCdCWFXBTSBBK3MzWtJiC |
||
|
|
41bce91770 |
gui: add SegmentBarsWidget and SpeedGraphWidget
Per-connection progress bars and the 60 s rolling speed graph the progress dialog needs (docs/03-gui-spec.md §3). Both reuse row/widget state across ticks instead of rebuilding, matching the discipline DownloadTableModel already uses for progress patches. SpeedGraphWidget keeps a fixed ring buffer and one reused QPainterPath — no allocation in paintEvent or addSample. Fixed a real bug found while writing tst_speedgraphwidget: the elapsed timer was started in the constructor, so the very first sample after construction would silently wait up to 1 s to be recorded instead of landing immediately. Tested against mockd (both offline via QTest/offscreen, and manually against a running mockd instance through ProgressDialog once that lands). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01NSCdCWFXBTSBBK3MzWtJiC |
||
|
|
0a38867579 | merge: GUI daemon-requests, verified against veloxd | ||
|
|
0af4a5c4fc |
gui: verify against the real veloxd; file one daemon gap
Pointed the existing RPC client at a real veloxd instance (isolated
HOME/XDG_RUNTIME_DIR/XDG_DATA_HOME — never touched the real ~/Downloads),
added a real download.add against tools/testserver, and watched it render
live end to end with no GUI code changes:
- handshake, subscribe, category.list/queue.list all match what DAEMON
reported
- the full event.task.state sequence and batched event.task.progress
both applied correctly by DownloadTableModel
- the written file's SHA-256 matches the server's reference
Filed gui/docs/daemon-requests-m1.md: TaskSummary.sizeBytes is never
populated by this daemon build, even in download.get after the task
completes with the exact byte count already on disk. Not a GUI bug —
ProgressDelegate and the model already do the documented right thing
when size is unknown (fall back to plain text, no bar) — but it means
every task renders without a percentage against the real daemon today.
mockd always supplies sizeBytes so this doesn't block current GUI work;
flagging before the M1 GUI<->daemon integration pass.
mockd stays the primary harness for the unhappy paths
(--slow/--flaky/--drop-connection) that a real daemon won't misbehave on
command for.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016Ne28kx4VreeBWZv82Nksd
|
||
|
|
ba29fcb5bd | merge: event fan-out, category.list, queue.list | ||
|
|
824fa481bb |
daemon: event.* fan-out (D5) + category.list/queue.list (D3) — GUI-ready
The two items aimed at pointing GUI at a real veloxd instead of mockd.
rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.
sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
the store's own prior row for previousState (authoritative
regardless of engine/scheduler timing), writes the error columns,
and — when a hub is supplied — publishes event.task.state with
{taskId, state, previousState, summary, error}. Wired into every
transition: scheduler-driven (admission -> probing, resume ->
connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
(EnginePort::progress(), a new interface method backed by
DownloadHandle::progress()), plus a store side-effect
(Tasks::update_progress) so download.list/get stay current between
state transitions. Returns rows; does NOT publish itself — batching
into one array message is the caller's job, per the schema's
x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
event.task.progress per tick, only when there's something to say.
dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").
store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).
Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.
Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.
deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|
||
|
|
72344134cc | merge: engine wired into veloxd — vertical slice closes | ||
|
|
08d7ee9263 |
daemon: wire the engine into veloxd — the vertical slice runs end to end
CORE stage 8 merged, so vdm::Engine is linkable. This closes D4a and narrows D4b: `velox add <url>` now actually downloads. - sched/engine_port_core.hpp — the real EnginePort: forwards to a live vdm::Engine, keeps the DownloadHandle per task for pause/resume/ cancel/provide_auth/decide/refresh_url, drives set_task_order / set_max_active_segments / set_host_segment_cap via engine.segment_budget(). CORE confirmed the admission model: DAEMON decides when to start(); the engine's own download_task calls register_task/set_want internally — DAEMON never touches per-task budget calls. EnginePort gains release(TaskId) so the port drops a handle when the task goes terminal. - rpc/event_loop — EventLoop::post(fn): thread-safe, runs fn on the loop thread next iteration. The marshaller for engine-thread callbacks. - main.cpp — constructs vdm::Engine + EnginePortCore + Scheduler (post_to_loop = loop.post). At startup: reconcile_after_restart() (ADR 0013 §5), reload_config(), tick(). A 1 s timerfd on the loop re-runs tick() (schedule windows, missed nudges); download.add nudges via dispatcher.set_on_mutation. End-to-end verified against tools/testserver: `velox add http://127.0.0.1:.../file/512K` -> task queued -> scheduler admits -> engine downloads 524288 bytes -> complete, file on disk. First byte-path all the way through the project. safepath-adversarial.md: re-verified per its own note — CORE landed O_NOFOLLOW on the target open (core/src/io/sparse_file.cpp), so the leaf-symlink TOCTOU is now closed; residual is down to one intermediate-dir gap (documented post-M1 chase). 36 daemon/cli tests green; scheduler + uds_roundtrip TSan-clean. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig |