4e87730ce9964f5558bb675659563ace435425f6
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  samiandClaude Sonnet 5
|
eb72aa522c |
daemon: build velox-nmhost, systemd socket activation + units, velox(1) man page
Closes build order items 7 (the systemd half) and 9 (D11 in deferrals.md). velox-nmhost (nmhost/src/main.cpp, 185 lines): a poll()-driven byte pump between Firefox's native-messaging framing on stdio (4-byte native-byte-order length prefix) and veloxd's own NDJSON framing on the Unix socket. Reframes each direction, no JSON parsing, no retry/backoff (the extension relaunches a fresh host on its own reconnect), exits the moment either side closes. Deliberately dependency-free — no veloxd_* library, no nlohmann_json — since it runs unconfined outside Firefox's sandbox regardless of packaging format. Two real bugs found and fixed while getting the integration test to actually pass rather than hang, both exactly the class of bug a "trivial pump" invites: 1. Never set the pumped fds non-blocking, so the "drain what's available" read loop blocked on its own second read() instead of returning to poll(). 2. stdin and stdout are two different descriptors (0 and 1), not one — an early draft polled POLLOUT on fd 0, which is opened read-only, so EOF and writability were never both observable through the same pollfd entry. packaging/nativehost/com.velox.host.json + its own README.md supersede AGENT-DAEMON.md's stale "four locations" line: spike S1 / ADR 0003 found only three manifest locations are real (~/.mozilla/native-messaging-hosts/ for BOTH deb/tarball and snap Firefox, /usr/lib/mozilla/... for deb/tarball only, the flatpak sandbox path) — the fourth, ~/snap/firefox/common/.mozilla/..., is not read by snap Firefox at all. The README spells out the per-user-manifest / postinst enumeration implication for PKG/QA (postinst runs once as root; the two ~/-relative locations are per-user) and flags that docs/07-packaging.md's own install-layout line only shows the one root-owned path. Socket activation: rpc/systemd_activation.cpp is a from-scratch sd_listen_fds() (env vars only — LISTEN_PID/LISTEN_FDS, fd 3 — no libsystemd link) that UdsServer::start() checks first, skipping its own create/bind/chmod/listen when systemd already bound the socket. packaging/systemd/velox.socket + velox.service are the unit pair, verified both by systemd-analyze verify and by an actual fork/dup2/execve simulation of the activation handshake — a real session.hello round-tripped over the handed-off fd with no bind() ever called inside the daemon for that run. velox.service deliberately skips ProtectSystem=/ProtectHome=/ReadWritePaths=: saveTo.allowedRoots is user-configurable to anywhere on the filesystem, and a sandbox here would turn a legitimately-configured save location into an opaque EROFS/EACCES instead of the daemon's own clear -32011. cli/man/velox.1 documents the CLI as it actually exists today (add/ls/pause/resume/rm, --json) — the queue/settings subcommands AGENT-DAEMON.md's build order originally sketched aren't implemented in cli/src/main.cpp yet, so the page doesn't claim they are. Checked warning-free with groff -mandoc -ww -z. Full ctest: 57/57 (excluding the pre-existing, unrelated conformance failure noted in earlier commits). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP |
||
|
samiandClaude Sonnet 5
|
4f6c0cc9d2 |
daemon: D3's remainder — rules.*, queue.reorder, schedule.*, limiter.*, download.update/refreshUrl
rules.list/rules.upsert: new store/rules.{hpp,cpp} (list in priority order; apply()
upserts+removes atomically, generating an id when absent, per the schema's own "never
leaves the table in a half-valid state"). Found and fixed along the way: migration
0001's rules table had no column for Rule.name at all — every rules.list/.upsert call
failed outright ("no such column: name"), unit tests included, since :memory: migrates
through the same path. Migration 0004 adds it.
queue.reorder: new store::Queues::reorder — taskIds must be an exact permutation of
the queue's current membership (compared as sorted sets) or nothing is written and
-32602 names the queue; a valid permutation rewrites every member's queue_position in
one transaction.
schedule.get/schedule.set: a thin wrapper over the queues.schedule column (already
read since D3b, never independently settable). nextRunAt is deliberately left unset —
computing it needs the same local-time, DST-aware window logic
sched/schedule_window.hpp's window_open() only has half of; called out rather than
approximated, and the field is optional.
limiter.get/limiter.set: backed by the same downloads.speedLimitEnabled/
downloads.speedLimitBps settings keys D9 already wired — one bag of truth, not two.
The new part is reaching the engine: EnginePort/TaskActionPort gain
set_global_speed_limit(bps) (0 = unlimited, TokenBucket's own convention), wired to
Engine::rate_limiter().set_global_limit(). Pushed live on every limiter.set *and* on
Scheduler::reload_config() so a limit from a previous run isn't silently unlimited
again after a restart. applyToRunning is accepted but has no lever to pull
differently — a single shared global bucket has no "next task only" variant. Also
found, not chased further: the schema's "globalBps:0 with enabled:true means 'stop
everything'" is the opposite of what TokenBucket does with rate_bps==0 (unlimited) —
a real discrepancy, but the schema says the GUI must not offer that combination.
download.update: "moving saveDir or filename moves the file on disk in the same
operation" — resolved and root-checked like download.add's destination, then the
.veloxpart/.veloxpart.meta pair (or the finished file, if complete) is moved via
rename, falling back to copy+remove across filesystems, only when the resolved
location actually differs. categoryId/queueId(appended to the new queue's run
order)/description/segments/bufferBytes/checksum apply through new
store::Tasks::apply_update.
download.refreshUrl: same async server-layer special-case as download.probe (a real
network round trip, same 30s deadline). Re-probes, flags contentChanged only when
size or validator are both known and actually differ, persists the new URL and probe
result, and swaps the URL on a live engine handle via a newly-widened
EnginePort::refresh_url (now takes headers too, matching DownloadHandle's real
signature — the seam had silently dropped them).
Found and documented, not fixed: the generated parser collapses "field absent" and
"field explicitly null" to the same nullopt for every optional<T> patch field
(DownloadUpdateParamsPatch, Settings) — both schemas document "an explicit null
clears the field" but neither handler can act on it because the wire distinction is
already gone by the time either sees the parsed struct. A generator-level gap
(PROTO's), not something to hand-route around locally.
Verified against real veloxd + tools/testserver: rules create/list, limiter.set
takes effect and reads back, schedule.set/get round-trips, queue.reorder against real
membership (and rejects a non-permutation), download.update renames+recategorizes a
task, download.refreshUrl swaps a paused task's URL and reports contentChanged
correctly. Full ctest: 55/55 (excluding the pre-existing, unrelated conformance
failure noted two commits back).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
samiandClaude Sonnet 5
|
4f6fb1029d |
daemon: D2 — download.probe, genuinely async on both transports
download.probe was a stub (-32603). It needs an HTTP round trip on the engine's probe
pool (up to the schema's 30s x-deadlineMs), which cannot fit VeloxDispatcher's
synchronous on_download_probe -> HandlerResult<T> return without blocking the RPC
loop for the duration — a hard no per CLAUDE.md ("never block the RPC loop") and
AGENT-DAEMON.md build step 1.
uds_server.cpp and ws_server.cpp special-case "download.probe" before the generic
dispatch(), exactly the way they already special-case session.hello/session.subscribe:
parse the params, call the port, and queue the reply whenever the callback fires
(dropped silently if the connection is gone by then).
rpc::TaskActionPort gains probe_now(DownloadProbeParams, callback) — kept in proto/std
terms, no vdm::net::* in the signature, so veloxd_rpc never needs core/include's vdm
headers just to declare this. sched::Scheduler::probe_now is the implementation:
builds a vdm::net::ProbeRequest, runs it on the engine's probe pool, marshals the
engine-thread callback back onto the loop (deps_.post_to_loop, same as every other
engine callback here), maps a probe failure to -32013 ProbeFailed (data.httpStatus set
when there was an HTTP response), and fills suggestedCategoryId/suggestedSaveDir with a
plain extension match against the categories table — not the real rules engine, which
is still D3; noted in a comment.
Verified against real veloxd + tools/testserver, not just unit tests: a real probe
answers in ~5ms with size/resumable/etag/redirect chain; a bad host maps to -32013;
and — the actual point of the async design — a connection running a 10s slow-loris
probe does not block a second connection's download.list, which answers in ~1ms while
the probe is still outstanding.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
|
||
|
samiandClaude Sonnet 5
|
824fa481bb |
daemon: event.* fan-out (D5) + category.list/queue.list (D3) — GUI-ready
The two items aimed at pointing GUI at a real veloxd instead of mockd.
rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.
sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
the store's own prior row for previousState (authoritative
regardless of engine/scheduler timing), writes the error columns,
and — when a hub is supplied — publishes event.task.state with
{taskId, state, previousState, summary, error}. Wired into every
transition: scheduler-driven (admission -> probing, resume ->
connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
(EnginePort::progress(), a new interface method backed by
DownloadHandle::progress()), plus a store side-effect
(Tasks::update_progress) so download.list/get stay current between
state transitions. Returns rows; does NOT publish itself — batching
into one array message is the caller's job, per the schema's
x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
event.task.progress per tick, only when there's something to say.
dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").
store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).
Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.
Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.
deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|
||
|
samiandClaude Sonnet 5
|
e60d6669d8 |
daemon: rpc/ — Unix-socket transport + generated dispatch wiring (build step 1)
First real code in daemon/. veloxd now listens on
$XDG_RUNTIME_DIR/velox/velox.sock (0600, SO_PEERCRED same-UID check),
frames NDJSON, and routes every method through the generated
velox::proto::dispatch(). The CLI and GUI have a server to talk to.
Modules:
- rpc/ndjson.hpp — newline-delimited framing, 8 MiB frame cap, CRLF-
tolerant, partial-tail buffering. Header-only, tested.
- rpc/event_loop — single-threaded poll(2) reactor; never blocks the
loop. stop()/wake() are async-signal-safe (eventfd).
- rpc/runtime_dir — $XDG_RUNTIME_DIR/velox resolution, 0700, owner-checked;
refuses an insecure fallback rather than using /tmp.
- rpc/uds_server — listener + non-blocking per-conn read/write with
backpressure; handles session.hello (protocol-major
check -> -32001, sessionId, transport=uds) and
session.subscribe in the server layer; routes the
rest through dispatch().
- rpc/dispatcher — VeloxDispatcher : proto::Dispatcher, all 39 methods.
download.list answers an empty table; the rest return
"not implemented" (-> -32603) until the store lands.
- main.cpp — abstract-namespace single-instance lock, signal ->
clean shutdown, socket unlinked on exit.
Tests (ASan+UBSan and TSan clean):
- veloxd.ndjson — framing edge cases
- veloxd.uds_roundtrip — real socket: hello ok / version mismatch / empty
list / -32601 / -32700 / pipelined requests, and a
guard on the -32603 collapse documented in P1.
Known gap, filed not worked around: daemon/docs/proto-requests-m1.md P1 —
the generated Dispatcher has no error channel below -32603, so handlers
cannot yet return -32010/-32011/-32013 with their data payloads. The
server layer handles -32001/-32002/-32003 around dispatch(); genuine
in-handler errors collapse to -32603 until PROTO gives handlers a real
error return. Three error fixtures are non-conformant until then.
Not in this drop: rpc/ws_server (next; needs the store for hashed pairing
tokens), store/, sched/, cli/. WS reuses this event loop.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
|