sami/vdm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

daemon: event.* fan-out (D5) + category.list/queue.list (D3) — GUI-ready

Browse Source 
The two items aimed at pointing GUI at a real veloxd instead of mockd.

rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.

sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
  the store's own prior row for previousState (authoritative
  regardless of engine/scheduler timing), writes the error columns,
  and — when a hub is supplied — publishes event.task.state with
  {taskId, state, previousState, summary, error}. Wired into every
  transition: scheduler-driven (admission -> probing, resume ->
  connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
  (EnginePort::progress(), a new interface method backed by
  DownloadHandle::progress()), plus a store side-effect
  (Tasks::update_progress) so download.list/get stay current between
  state transitions. Returns rows; does NOT publish itself — batching
  into one array message is the caller's job, per the schema's
  x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
  tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
  event.task.progress per tick, only when there's something to say.

dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").

store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).

Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.

Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.

deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
This commit is contained in:
sami
2026-09-11 07:19:11 +04:00
co-authored by Claude Sonnet 5
parent 72344134cc
commit 824fa481bb
29 changed files with 775 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
daemon/src/sched/scheduler.cpp
+72 -16
View File
@@ -102,8 +102,9 @@ std::vector<proto::TaskState> non_terminal_states() {
} // namespace
Scheduler::Scheduler(store::Db& db, EnginePort& engine, Governor governor, Deps deps)
: db_(db), engine_(engine), governor_(std::move(governor)), deps_(std::move(deps)) {
Scheduler::Scheduler(store::Db& db, EnginePort& engine, Governor governor, rpc::EventHub* hub,
Deps deps)
: db_(db), engine_(engine), governor_(std::move(governor)), hub_(hub), deps_(std::move(deps)) {
if (!deps_.local_now) deps_.local_now = local_now_default;
if (!deps_.post_to_loop) deps_.post_to_loop = [](std::function<void()> f) { f(); };
}
@@ -237,7 +238,7 @@ store::DbResult<void> Scheduler::tick() {
vdm::task::DownloadCallbacks cbs;
const std::string id_copy = wire_id;
cbs.on_state = [this, id_copy](vdm::task::EngineState, vdm::task::EngineState to,
cbs.on_state = [this, id_copy](vdm::task::EngineState from, vdm::task::EngineState to,
const std::optional<vdm::ErrorInfo>& err) {
std::optional<TaskErrorFields> ef;
if (err) {
@@ -247,20 +248,22 @@ store::DbResult<void> Scheduler::tick() {
if (err->http_status != 0) ef->http_status = err->http_status;
ef->retryable = err->retryable;
}
const std::string from_name = engine_state_name(from);
const std::string to_name = engine_state_name(to);
deps_.post_to_loop(
[this, id_copy, to_name, ef]() { on_engine_state(id_copy, to_name, ef); });
deps_.post_to_loop([this, id_copy, from_name, to_name, ef]() {
on_engine_state(id_copy, from_name, to_name, ef);
});
};
const vdm::TaskId engine_id = engine_.start(spec, std::move(cbs));
map(wire_id, engine_id);
(void)tasks.set_state(wire_id, "probing", std::nullopt);
transition(wire_id, "probing", std::nullopt, std::nullopt);
}
for (const auto& wire_id : d.to_resume) {
if (auto eid = engine_id_of(wire_id)) {
engine_.resume(*eid);
(void)tasks.set_state(wire_id, "connecting", std::nullopt);
transition(wire_id, "connecting", std::nullopt, std::nullopt);
}
}
@@ -269,7 +272,7 @@ store::DbResult<void> Scheduler::tick() {
? pause_reason_str(d.pause_reasons.at(wire_id))
: "user";
if (auto eid = engine_id_of(wire_id)) engine_.pause(*eid);
(void)tasks.set_state(wire_id, "paused", std::string(reason));
transition(wire_id, "paused", std::string(reason), std::nullopt);
}
std::vector<vdm::TaskId> order;
@@ -281,14 +284,18 @@ store::DbResult<void> Scheduler::tick() {
return {};
}
void Scheduler::on_engine_state(const std::string& wire_id, std::string_view engine_state,
const std::optional<TaskErrorFields>& err) {
void Scheduler::transition(const std::string& wire_id, std::string_view to_state,
std::optional<std::string> pause_reason,
const std::optional<TaskErrorFields>& err) {
store::Tasks tasks(db_);
// pause_reason: an engine-initiated pause carries an error => 'auto' (ADR 0013 §2);
// otherwise set_state clears the column.
std::optional<std::string> reason;
if (engine_state == "paused" && err) reason = "auto";
(void)tasks.set_state(wire_id, engine_state, reason);
const auto before = tasks.get(wire_id);
const std::string previous = (before && before->has_value()) ? (**before).state : std::string();
// An engine-initiated pause carries an error => 'auto' (ADR 0013 §2), overriding
// whatever the caller passed (a scheduler-driven pause never carries an error here).
std::optional<std::string> reason = pause_reason;
if (to_state == "paused" && err) reason = "auto";
(void)tasks.set_state(wire_id, to_state, reason);
if (err) {
auto st = db_.prepare(
@@ -306,7 +313,30 @@ void Scheduler::on_engine_state(const std::string& wire_id, std::string_view eng
}
}
if (engine_state == "complete" || engine_state == "failed" || engine_state == "cancelled") {
if (!hub_) return;
auto after = tasks.get(wire_id);
if (!after || !after->has_value()) return;
const proto::TaskSummary summary = store::to_summary(**after);
nlohmann::json params{
{"taskId", wire_id},
{"state", std::string(to_state)},
{"previousState", previous.empty() ? nlohmann::json(nullptr) : nlohmann::json(previous)},
{"summary", summary},
{"error", summary.error.has_value() ? nlohmann::json(*summary.error) : nlohmann::json(nullptr)},
};
hub_->publish(proto::Event::TaskState, proto::make_notification(proto::Event::TaskState, params),
wire_id);
}
void Scheduler::on_engine_state(const std::string& wire_id, std::string_view from_state,
std::string_view to_state,
const std::optional<TaskErrorFields>& err) {
(void)from_state; // transition() reads the store's own current state as previousState,
// which is authoritative regardless of engine/store timing
transition(wire_id, to_state, std::nullopt, err);
if (to_state == "complete" || to_state == "failed" || to_state == "cancelled") {
if (auto eid = engine_id_of(wire_id)) {
engine_.release(*eid);
unmap_engine(*eid);
@@ -314,4 +344,30 @@ void Scheduler::on_engine_state(const std::string& wire_id, std::string_view eng
}
}
std::vector<Scheduler::ProgressRow> Scheduler::progress_snapshot() {
std::vector<ProgressRow> out;
if (to_engine_.empty()) return out;
store::Tasks tasks(db_);
out.reserve(to_engine_.size());
for (const auto& [wire_id, engine_id] : to_engine_) {
const auto p = engine_.progress(engine_id);
if (!p) continue;
ProgressRow row;
row.task_id = wire_id;
row.downloaded_bytes = p->downloaded;
row.speed_bps = p->speed_bps;
row.eta_seconds = p->eta_seconds;
for (const auto& s : p->segments)
row.segments.push_back({s.index, s.completed, s.speed_bps});
out.push_back(std::move(row));
(void)tasks.update_progress(wire_id, static_cast<std::int64_t>(p->downloaded),
static_cast<std::int64_t>(p->effective_segments),
static_cast<std::int64_t>(p->effective_buffer_bytes));
}
return out;
}
} // namespace velox::daemon::sched