daemon: event.* fan-out (D5) + category.list/queue.list (D3) — GUI-ready

The two items aimed at pointing GUI at a real veloxd instead of mockd.

rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.

sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
  the store's own prior row for previousState (authoritative
  regardless of engine/scheduler timing), writes the error columns,
  and — when a hub is supplied — publishes event.task.state with
  {taskId, state, previousState, summary, error}. Wired into every
  transition: scheduler-driven (admission -> probing, resume ->
  connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
  (EnginePort::progress(), a new interface method backed by
  DownloadHandle::progress()), plus a store side-effect
  (Tasks::update_progress) so download.list/get stay current between
  state transitions. Returns rows; does NOT publish itself — batching
  into one array message is the caller's job, per the schema's
  x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
  tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
  event.task.progress per tick, only when there's something to say.

dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").

store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).

Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.

Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.

deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig

daemon/src/sched/engine_port.hpp

@@ -14,6 +14,7 @@
 
 #include <cstdint>
 #include <functional>
+#include <optional>
 #include <string>
 #include <vector>
 
@@ -40,6 +41,10 @@ public:
     // The daemon is done with this task (it went terminal). Drop the handle. Idempotent.
     virtual void release(vdm::TaskId) = 0;
 
+    // A synchronous, lock-guarded snapshot (DownloadHandle::progress()). nullopt if the
+    // id is unknown (already released, or never started).
+    virtual std::optional<vdm::task::Progress> progress(vdm::TaskId) const = 0;
+
     // ADR 0011 admission surface. Values are DAEMON's; enforcement is the engine's.
     virtual void set_task_order(const std::vector<vdm::TaskId>& order) = 0;
     virtual void set_max_active_segments(std::uint32_t n) = 0;

daemon/src/sched/engine_port_core.hpp

@@ -44,6 +44,11 @@ public:
         if (auto* h = find(id)) h->refresh_url(url);
     }
     void release(vdm::TaskId id) override { handles_.erase(id); }
+    std::optional<vdm::task::Progress> progress(vdm::TaskId id) const override {
+        auto it = handles_.find(id);
+        if (it == handles_.end()) return std::nullopt;
+        return it->second.progress();
+    }
 
     void set_task_order(const std::vector<vdm::TaskId>& order) override {
         engine_.segment_budget().set_task_order(order);

daemon/src/sched/fake_engine_port.hpp

@@ -5,6 +5,7 @@
 
 #include <cstdint>
 #include <string>
+#include <unordered_map>
 #include <vector>
 
 #include "sched/engine_port.hpp"
@@ -42,6 +43,13 @@ public:
     void decide(vdm::TaskId, vdm::task::Decision) override {}
     void refresh_url(vdm::TaskId, const std::string&) override {}
     void release(vdm::TaskId id) override { released.push_back(id); }
+    std::optional<vdm::task::Progress> progress(vdm::TaskId id) const override {
+        auto it = fake_progress.find(id.value);
+        return it == fake_progress.end() ? std::nullopt : std::optional(it->second);
+    }
+
+    // Tests set this to control what progress(id) returns.
+    std::unordered_map<std::uint64_t, vdm::task::Progress> fake_progress;
     void set_task_order(const std::vector<vdm::TaskId>& order) override { orders.push_back(order); }
     void set_max_active_segments(std::uint32_t n) override { max_active_segments.push_back(n); }
     void set_host_segment_cap(const std::string& h, std::uint32_t c) override {

daemon/src/sched/scheduler.cpp

@@ -102,8 +102,9 @@ std::vector<proto::TaskState> non_terminal_states() {
 
 }  // namespace
 
-Scheduler::Scheduler(store::Db& db, EnginePort& engine, Governor governor, Deps deps)
-    : db_(db), engine_(engine), governor_(std::move(governor)), deps_(std::move(deps)) {
+Scheduler::Scheduler(store::Db& db, EnginePort& engine, Governor governor, rpc::EventHub* hub,
+                     Deps deps)
+    : db_(db), engine_(engine), governor_(std::move(governor)), hub_(hub), deps_(std::move(deps)) {
     if (!deps_.local_now) deps_.local_now = local_now_default;
     if (!deps_.post_to_loop) deps_.post_to_loop = [](std::function<void()> f) { f(); };
 }
@@ -237,7 +238,7 @@ store::DbResult<void> Scheduler::tick() {
 
         vdm::task::DownloadCallbacks cbs;
         const std::string id_copy = wire_id;
-        cbs.on_state = [this, id_copy](vdm::task::EngineState, vdm::task::EngineState to,
+        cbs.on_state = [this, id_copy](vdm::task::EngineState from, vdm::task::EngineState to,
                                        const std::optional<vdm::ErrorInfo>& err) {
             std::optional<TaskErrorFields> ef;
             if (err) {
@@ -247,20 +248,22 @@ store::DbResult<void> Scheduler::tick() {
                 if (err->http_status != 0) ef->http_status = err->http_status;
                 ef->retryable = err->retryable;
             }
+            const std::string from_name = engine_state_name(from);
             const std::string to_name = engine_state_name(to);
-            deps_.post_to_loop(
-                [this, id_copy, to_name, ef]() { on_engine_state(id_copy, to_name, ef); });
+            deps_.post_to_loop([this, id_copy, from_name, to_name, ef]() {
+                on_engine_state(id_copy, from_name, to_name, ef);
+            });
         };
 
         const vdm::TaskId engine_id = engine_.start(spec, std::move(cbs));
         map(wire_id, engine_id);
-        (void)tasks.set_state(wire_id, "probing", std::nullopt);
+        transition(wire_id, "probing", std::nullopt, std::nullopt);
     }
 
     for (const auto& wire_id : d.to_resume) {
         if (auto eid = engine_id_of(wire_id)) {
             engine_.resume(*eid);
-            (void)tasks.set_state(wire_id, "connecting", std::nullopt);
+            transition(wire_id, "connecting", std::nullopt, std::nullopt);
         }
     }
 
@@ -269,7 +272,7 @@ store::DbResult<void> Scheduler::tick() {
                                 ? pause_reason_str(d.pause_reasons.at(wire_id))
                                 : "user";
         if (auto eid = engine_id_of(wire_id)) engine_.pause(*eid);
-        (void)tasks.set_state(wire_id, "paused", std::string(reason));
+        transition(wire_id, "paused", std::string(reason), std::nullopt);
     }
 
     std::vector<vdm::TaskId> order;
@@ -281,14 +284,18 @@ store::DbResult<void> Scheduler::tick() {
     return {};
 }
 
-void Scheduler::on_engine_state(const std::string& wire_id, std::string_view engine_state,
-                                const std::optional<TaskErrorFields>& err) {
+void Scheduler::transition(const std::string& wire_id, std::string_view to_state,
+                           std::optional<std::string> pause_reason,
+                           const std::optional<TaskErrorFields>& err) {
     store::Tasks tasks(db_);
-    // pause_reason: an engine-initiated pause carries an error => 'auto' (ADR 0013 §2);
-    // otherwise set_state clears the column.
-    std::optional<std::string> reason;
-    if (engine_state == "paused" && err) reason = "auto";
-    (void)tasks.set_state(wire_id, engine_state, reason);
+    const auto before = tasks.get(wire_id);
+    const std::string previous = (before && before->has_value()) ? (**before).state : std::string();
+
+    // An engine-initiated pause carries an error => 'auto' (ADR 0013 §2), overriding
+    // whatever the caller passed (a scheduler-driven pause never carries an error here).
+    std::optional<std::string> reason = pause_reason;
+    if (to_state == "paused" && err) reason = "auto";
+    (void)tasks.set_state(wire_id, to_state, reason);
 
     if (err) {
         auto st = db_.prepare(
@@ -306,7 +313,30 @@ void Scheduler::on_engine_state(const std::string& wire_id, std::string_view eng
         }
     }
 
-    if (engine_state == "complete" || engine_state == "failed" || engine_state == "cancelled") {
+    if (!hub_) return;
+    auto after = tasks.get(wire_id);
+    if (!after || !after->has_value()) return;
+    const proto::TaskSummary summary = store::to_summary(**after);
+
+    nlohmann::json params{
+        {"taskId", wire_id},
+        {"state", std::string(to_state)},
+        {"previousState", previous.empty() ? nlohmann::json(nullptr) : nlohmann::json(previous)},
+        {"summary", summary},
+        {"error", summary.error.has_value() ? nlohmann::json(*summary.error) : nlohmann::json(nullptr)},
+    };
+    hub_->publish(proto::Event::TaskState, proto::make_notification(proto::Event::TaskState, params),
+                 wire_id);
+}
+
+void Scheduler::on_engine_state(const std::string& wire_id, std::string_view from_state,
+                                std::string_view to_state,
+                                const std::optional<TaskErrorFields>& err) {
+    (void)from_state;  // transition() reads the store's own current state as previousState,
+                       // which is authoritative regardless of engine/store timing
+    transition(wire_id, to_state, std::nullopt, err);
+
+    if (to_state == "complete" || to_state == "failed" || to_state == "cancelled") {
         if (auto eid = engine_id_of(wire_id)) {
             engine_.release(*eid);
             unmap_engine(*eid);
@@ -314,4 +344,30 @@ void Scheduler::on_engine_state(const std::string& wire_id, std::string_view eng
     }
 }
 
+std::vector<Scheduler::ProgressRow> Scheduler::progress_snapshot() {
+    std::vector<ProgressRow> out;
+    if (to_engine_.empty()) return out;
+
+    store::Tasks tasks(db_);
+    out.reserve(to_engine_.size());
+    for (const auto& [wire_id, engine_id] : to_engine_) {
+        const auto p = engine_.progress(engine_id);
+        if (!p) continue;
+
+        ProgressRow row;
+        row.task_id = wire_id;
+        row.downloaded_bytes = p->downloaded;
+        row.speed_bps = p->speed_bps;
+        row.eta_seconds = p->eta_seconds;
+        for (const auto& s : p->segments)
+            row.segments.push_back({s.index, s.completed, s.speed_bps});
+        out.push_back(std::move(row));
+
+        (void)tasks.update_progress(wire_id, static_cast<std::int64_t>(p->downloaded),
+                                    static_cast<std::int64_t>(p->effective_segments),
+                                    static_cast<std::int64_t>(p->effective_buffer_bytes));
+    }
+    return out;
+}
+
 }  // namespace velox::daemon::sched

daemon/src/sched/scheduler.hpp

@@ -4,13 +4,17 @@
 // wire-UUID <-> vdm::TaskId map and is the only thing that calls EnginePort::start /
 // pause / resume / set_task_order.
 //
-// Threading: tick(), reload_config(), reconcile_after_restart() and the on_engine_*
-// callbacks all run on ONE thread (the RPC loop). Engine callbacks arrive on engine
-// threads, so the real wiring passes a `post_to_loop` that marshals them here; the
-// default runs them inline (tests, single-threaded).
+// Threading: tick(), reload_config(), reconcile_after_restart(), progress_snapshot() and
+// the on_engine_* callbacks all run on ONE thread (the RPC loop). Engine callbacks arrive
+// on engine threads, so the real wiring passes a `post_to_loop` that marshals them here;
+// the default runs them inline (tests, single-threaded).
 //
-// Not yet wired into veloxd — that plus the real EnginePort land when velox::core's
-// stage-8 bodies reach main (daemon/docs/deferrals.md D4).
+// event.task.state (D5): on_engine_state publishes it when a hub is supplied — the same
+// callback that keeps the store row current also keeps subscribed clients current.
+// event.task.progress is NOT published here: it must be batched into one array message at
+// <=4 Hz (event.task.progress.schema.json x-maxRateHz), so the caller collects
+// progress_snapshot() on its own 250 ms timer and does one hub_.publish() with the whole
+// array, never one per task.
 
 #include <cstdint>
 #include <ctime>
@@ -18,7 +22,9 @@
 #include <optional>
 #include <string>
 #include <unordered_map>
+#include <vector>
 
+#include "rpc/event_hub.hpp"
 #include "sched/engine_port.hpp"
 #include "sched/governor.hpp"
 #include "store/sqlite.hpp"
@@ -46,7 +52,10 @@ public:
         std::function<void(std::function<void()>)> post_to_loop;
     };
 
-    Scheduler(store::Db& db, EnginePort& engine, Governor governor, Deps deps = {});
+    // `hub` is optional so unit tests can build a Scheduler with no event fan-out at all;
+    // production wiring always supplies one.
+    Scheduler(store::Db& db, EnginePort& engine, Governor governor, rpc::EventHub* hub = nullptr,
+             Deps deps = {});
 
     // ADR 0013 §5: on daemon start, every task whose persisted state is a CORE-owned one
     // (probing..verifying) is rewritten to `queued`; `paused` keeps its pauseReason. The
@@ -62,11 +71,29 @@ public:
     // (start / resume / pause via the engine, update task state rows, push set_task_order).
     store::DbResult<void> tick();
 
-    // Engine lifecycle callback -> store projection, so the next tick sees ground truth.
-    // Keyed by wire UUID (known when the callback is built, before start() returns the
-    // TaskId). Also the hook the event.task.state fan-out will use — D5.
-    void on_engine_state(const std::string& wire_id, std::string_view engine_state,
-                         const std::optional<TaskErrorFields>& err);
+    // Engine lifecycle callback -> store projection, so the next tick sees ground truth,
+    // and (when a hub was supplied) the event.task.state publish. Keyed by wire UUID
+    // (known when the callback is built, before start() returns the TaskId).
+    void on_engine_state(const std::string& wire_id, std::string_view from_state,
+                         std::string_view to_state, const std::optional<TaskErrorFields>& err);
+
+    // One row per task the engine is currently tracking, for the caller's
+    // event.task.progress batch. Also writes downloaded_bytes / eff_segments /
+    // eff_buffer_bytes back to the store so download.list / download.get stay current
+    // between state transitions.
+    struct ProgressRow {
+        std::string task_id;
+        std::uint64_t downloaded_bytes;
+        std::uint64_t speed_bps;
+        std::optional<std::uint32_t> eta_seconds;
+        struct Segment {
+            std::uint32_t index;
+            std::uint64_t downloaded_bytes;
+            std::uint64_t speed_bps;
+        };
+        std::vector<Segment> segments;
+    };
+    std::vector<ProgressRow> progress_snapshot();
 
     // Diagnostics / tests.
     std::optional<std::string> wire_id_of(vdm::TaskId id) const;
@@ -76,9 +103,17 @@ private:
     void map(const std::string& wire_id, vdm::TaskId engine_id);
     void unmap_engine(vdm::TaskId engine_id);
 
+    // The one place a task's state row changes and (if a hub is set) event.task.state
+    // publishes. previousState is read from the store's own current row, not passed in —
+    // authoritative regardless of engine/scheduler timing.
+    void transition(const std::string& wire_id, std::string_view to_state,
+                    std::optional<std::string> pause_reason,
+                    const std::optional<TaskErrorFields>& err);
+
     store::Db& db_;
     EnginePort& engine_;
     Governor governor_;
+    rpc::EventHub* hub_;
     Deps deps_;
 
     std::unordered_map<std::string, vdm::TaskId> to_engine_;