sami/vdm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

daemon: event.* fan-out (D5) + category.list/queue.list (D3) — GUI-ready

Browse Source 
The two items aimed at pointing GUI at a real veloxd instead of mockd.

rpc/event_hub — per-subscription fan-out shared by both transports.
subscribe() registers a connection with no interest; set_filter()
(session.subscribe, replaces not adds) turns on event kinds and an
optional per-task id filter; publish() delivers a pre-built
notification to every matching subscriber. session.subscribe on both
UdsServer and WsServer now does the real thing — registers/updates a
subscription, tears it down in close_conn.

sched/scheduler — the on_engine_state hook now actually publishes:
- transition() is the one place a task's row changes state; it reads
  the store's own prior row for previousState (authoritative
  regardless of engine/scheduler timing), writes the error columns,
  and — when a hub is supplied — publishes event.task.state with
  {taskId, state, previousState, summary, error}. Wired into every
  transition: scheduler-driven (admission -> probing, resume ->
  connecting, pause) and engine-reported (on_engine_state).
- progress_snapshot(): one row per task the engine is tracking
  (EnginePort::progress(), a new interface method backed by
  DownloadHandle::progress()), plus a store side-effect
  (Tasks::update_progress) so download.list/get stay current between
  state transitions. Returns rows; does NOT publish itself — batching
  into one array message is the caller's job, per the schema's
  x-maxRateHz: 4 and AGENT-DAEMON.md item 5 ("one message per task per
  tick burns a core"). main.cpp's 250 ms timerfd is that caller: one
  event.task.progress per tick, only when there's something to say.

dispatcher::on_download_add now publishes event.task.added (schema:
"summary is always present so a client can insert the row without a
follow-up download.get").

store/categories, store/queues — the two D3 handlers GUI's panels
call. category.list projects the six seeded built-ins; queue.list
derives taskIds from tasks.queue_id/queue_position (Queue's own schema
note: a queue's stored row never carries membership, download.update
/ queue.reorder do).

Verified live end to end against tools/testserver: a subscribed client
sees event.task.added on add, then the full event.task.state sequence
(queued -> probing -> connecting -> downloading -> assembling ->
verifying -> complete) with correct previousState at every step, and
real category.list / queue.list results.

Tests: event_hub (filter-by-kind, filter-by-task-id, replace-not-add,
unsubscribe), store_categories_queues, plus new sched_scheduler cases
for event.task.state publishing and progress_snapshot's store
side-effect. 38 daemon/cli tests green; sched_scheduler / event_hub /
ws_server / uds_roundtrip TSan-clean.

deferrals.md: D5 mostly closed (event.task.removed and the
still-unpublished events wait on their owning D3 handlers); D3 down to
the remaining download.* verbs, rules/settings/limiter/schedule,
queue mutation, category mutation, grabber, media.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Upd9WhG9oppieig5nRDLig
This commit is contained in:
sami
2026-09-11 07:19:11 +04:00
co-authored by Claude Sonnet 5
parent 72344134cc
commit 824fa481bb
29 changed files with 775 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
daemon/src/rpc/dispatcher.cpp
+26 -2
View File
@@ -6,6 +6,8 @@
#include <nlohmann/json.hpp>
#include "fs/safepath.hpp"
#include "store/categories.hpp"
#include "store/queues.hpp"
#include "store/settings.hpp"
#include "store/tasks.hpp"
#include "util/time.hpp"
@@ -168,6 +170,14 @@ VeloxDispatcher::on_download_add(const proto::DownloadSpec& spec) {
return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
"download.add: " + ins.error().message});
// event.task.added carries the summary so a client inserts the row without a
// follow-up download.get (schema note on the event).
hub_.publish(proto::Event::TaskAdded,
proto::make_notification(proto::Event::TaskAdded,
nlohmann::json{{"taskId", row.task_id},
{"summary", store::to_summary(row)}}),
row.task_id);
if (on_mutation_) on_mutation_();
proto::DownloadAddResult r;
@@ -188,7 +198,14 @@ VeloxDispatcher::on_capture_offer(const proto::CaptureOfferParams&) {
}
proto::HandlerResult<proto::CategoryListResult>
VeloxDispatcher::on_category_list(const proto::CategoryListParams&) {
return not_implemented<proto::CategoryListResult>("category.list");
store::Categories categories(db_);
auto items = categories.list();
if (!items)
return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
"category.list: " + items.error().message});
proto::CategoryListResult r;
r.items = std::move(*items);
return r;
}
proto::HandlerResult<proto::CategoryRemoveResult>
VeloxDispatcher::on_category_remove(const proto::CategoryRemoveParams&) {
@@ -289,7 +306,14 @@ VeloxDispatcher::on_media_listVariants(const proto::MediaListVariantsParams&) {
}
proto::HandlerResult<proto::QueueListResult>
VeloxDispatcher::on_queue_list(const proto::QueueListParams&) {
return not_implemented<proto::QueueListResult>("queue.list");
store::Queues queues(db_);
auto items = queues.list();
if (!items)
return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
"queue.list: " + items.error().message});
proto::QueueListResult r;
r.items = std::move(*items);
return r;
}
proto::HandlerResult<proto::QueueReorderResult>
VeloxDispatcher::on_queue_reorder(const proto::QueueReorderParams&) {
daemon/src/rpc/dispatcher.hpp
+3 -1
View File
@@ -14,6 +14,7 @@
#include <functional>
#include "rpc/event_hub.hpp"
#include "store/sqlite.hpp"
#include "velox_proto.hpp"
@@ -21,7 +22,7 @@ namespace velox::daemon::rpc {
class VeloxDispatcher final : public velox::proto::Dispatcher {
public:
explicit VeloxDispatcher(velox::daemon::store::Db& db) : db_(db) {}
VeloxDispatcher(velox::daemon::store::Db& db, EventHub& hub) : db_(db), hub_(hub) {}
// Called after a handler mutates task state (download.add for now). main.cpp wires it
// to nudge the scheduler; unset in tests.
@@ -107,6 +108,7 @@ public:
private:
velox::daemon::store::Db& db_;
EventHub& hub_;
std::function<void()> on_mutation_;
};
daemon/src/rpc/event_hub.cpp
+55
View File
@@ -0,0 +1,55 @@
#include "rpc/event_hub.hpp"
#include <algorithm>
#include <nlohmann/json.hpp>
namespace velox::daemon::rpc {
namespace proto = velox::proto;
EventHub::SubId EventHub::subscribe(Sink sink) {
std::lock_guard<std::mutex> lk(mu_);
const SubId id = next_++;
subs_.emplace(id, Sub{std::move(sink), {}, std::nullopt});
return id;
}
void EventHub::set_filter(SubId id, std::vector<proto::Event> events,
std::optional<std::vector<std::string>> task_ids) {
std::lock_guard<std::mutex> lk(mu_);
if (auto it = subs_.find(id); it != subs_.end()) {
it->second.events = std::move(events);
it->second.task_ids = std::move(task_ids);
}
}
void EventHub::unsubscribe(SubId id) {
std::lock_guard<std::mutex> lk(mu_);
subs_.erase(id);
}
void EventHub::publish(proto::Event kind, const nlohmann::json& notification,
std::string_view task_id) {
// Copy the sinks to call out to while holding the lock only long enough to build the
// list — a sink runs arbitrary connection code (framing + a write syscall) and must
// not run with mu_ held.
std::vector<Sink> targets;
{
std::lock_guard<std::mutex> lk(mu_);
targets.reserve(subs_.size());
for (const auto& [id, sub] : subs_) {
(void)id;
if (std::find(sub.events.begin(), sub.events.end(), kind) == sub.events.end())
continue;
if (!task_id.empty() && sub.task_ids &&
std::find(sub.task_ids->begin(), sub.task_ids->end(), task_id) ==
sub.task_ids->end())
continue;
targets.push_back(sub.sink);
}
}
for (const auto& sink : targets) sink(notification);
}
} // namespace velox::daemon::rpc
daemon/src/rpc/event_hub.hpp
+63
View File
@@ -0,0 +1,63 @@
#pragma once
// Per-subscription event fan-out, shared by both transports. A connection subscribes once
// (session.subscribe) with the event kinds and optional task-id filter it wants; publish()
// delivers a pre-built notification to every subscription that asked for that kind and
// passes the per-task filter.
//
// event.task.progress is the one call site that matters for load: it is batched by the
// caller (Scheduler::progress_snapshot + one publish) into a single array message at
// <=4 Hz, never one publish per task — that batching happens before this class ever sees
// it (AGENT-DAEMON.md item 5, event.task.progress.schema.json x-maxRateHz).
#include <cstdint>
#include <functional>
#include <mutex>
#include <optional>
#include <string>
#include <string_view>
#include <unordered_map>
#include <vector>
#include <nlohmann/json_fwd.hpp>
#include "velox_proto.hpp"
namespace velox::daemon::rpc {
class EventHub {
public:
using SubId = std::uint64_t;
using Sink = std::function<void(const nlohmann::json&)>;
// Register a connection with no interest yet; session.subscribe calls set_filter to
// actually turn events on. Returns the id to unsubscribe with on disconnect.
SubId subscribe(Sink sink);
// Replaces the subscription's event set and task filter (session.subscribe replaces,
// never adds — matches the method's own description).
void set_filter(SubId id, std::vector<velox::proto::Event> events,
std::optional<std::vector<std::string>> task_ids);
void unsubscribe(SubId id);
// `notification` is a complete {jsonrpc, method, params} object
// (velox::proto::make_notification). `task_id` is matched against each subscription's
// filter when set; empty means "not task-scoped" and reaches every subscriber of
// `kind` regardless of their filter.
void publish(velox::proto::Event kind, const nlohmann::json& notification,
std::string_view task_id = {});
private:
struct Sub {
Sink sink;
std::vector<velox::proto::Event> events;
std::optional<std::vector<std::string>> task_ids;
};
std::mutex mu_;
std::unordered_map<SubId, Sub> subs_;
SubId next_ = 1;
};
} // namespace velox::daemon::rpc
daemon/src/rpc/uds_server.cpp
+17 -5
View File
@@ -56,8 +56,9 @@ json rpc_error(const json& id, proto::ErrorCode code, std::string_view msg, json
} // namespace
UdsServer::UdsServer(EventLoop& loop, proto::Dispatcher& dispatcher, std::string socket_path)
: loop_(loop), dispatcher_(dispatcher), path_(std::move(socket_path)) {}
UdsServer::UdsServer(EventLoop& loop, proto::Dispatcher& dispatcher, EventHub& hub,
std::string socket_path)
: loop_(loop), dispatcher_(dispatcher), hub_(hub), path_(std::move(socket_path)) {}
UdsServer::~UdsServer() {
for (auto& [fd, c] : conns_) {
@@ -251,11 +252,21 @@ bool UdsServer::handle_session_method(Conn& c, const std::string& method, const
json{{"path", p.error().path}});
return true;
}
// Event fan-out is not wired yet; accept the subscription and echo it back so a
// client can already register its interest without erroring.
if (!c.sub_id) {
const int fd = c.fd;
c.sub_id = hub_.subscribe([this, fd](const json& n) {
if (const auto it = conns_.find(fd); it != conns_.end()) queue_reply(*it->second, n);
});
}
std::vector<proto::Event> events;
proto::SessionSubscribeResult r;
r.ok = true;
for (const auto& ev : p->events) r.events.emplace_back(proto::to_string(ev));
for (const auto& ev : p->events) {
const auto name = proto::to_string(ev);
r.events.emplace_back(name);
if (auto e = proto::event_from_string(name)) events.push_back(*e);
}
hub_.set_filter(*c.sub_id, std::move(events), p->taskIds);
reply = proto::make_result(id, r);
return true;
}
@@ -300,6 +311,7 @@ void UdsServer::flush(Conn& c) {
void UdsServer::close_conn(int fd) {
if (const auto it = conns_.find(fd); it != conns_.end()) {
if (it->second->sub_id) hub_.unsubscribe(*it->second->sub_id);
loop_.del_fd(fd);
::close(fd);
conns_.erase(it);
daemon/src/rpc/uds_server.hpp
+6 -1
View File
@@ -13,12 +13,14 @@
#include <cstdint>
#include <memory>
#include <string>
#include <optional>
#include <system_error>
#include <unordered_map>
#include <vector>
#include <nlohmann/json_fwd.hpp>
#include "rpc/event_hub.hpp"
#include "rpc/ndjson.hpp"
#include "velox_proto.hpp"
@@ -28,7 +30,8 @@ class EventLoop;
class UdsServer {
public:
UdsServer(EventLoop& loop, velox::proto::Dispatcher& dispatcher, std::string socket_path);
UdsServer(EventLoop& loop, velox::proto::Dispatcher& dispatcher, EventHub& hub,
std::string socket_path);
~UdsServer();
UdsServer(const UdsServer&) = delete;
@@ -51,6 +54,7 @@ private:
bool close_after_flush = false;
bool hello_ok = false;
std::string session_id;
std::optional<EventHub::SubId> sub_id;
};
void on_listener_readable();
@@ -68,6 +72,7 @@ private:
EventLoop& loop_;
velox::proto::Dispatcher& dispatcher_;
EventHub& hub_;
std::string path_;
int listen_fd_ = -1;
bool bound_ = false; // path_ is ours to unlink on destruction
daemon/src/rpc/ws_server.cpp
+16 -2
View File
@@ -60,11 +60,12 @@ json rpc_error(const json& id, proto::ErrorCode code, std::string_view msg, json
} // namespace
WsServer::WsServer(EventLoop& loop, proto::Dispatcher& dispatcher, store::Db& db,
PairingApprover& approver, RuntimeDir runtime)
PairingApprover& approver, EventHub& hub, RuntimeDir runtime)
: loop_(loop),
dispatcher_(dispatcher),
db_(db),
approver_(approver),
hub_(hub),
runtime_(std::move(runtime)) {}
WsServer::~WsServer() {
@@ -364,9 +365,21 @@ bool WsServer::handle_session_ws(Conn& c, const std::string& method, const json&
json{{"path", p.error().path}});
return true;
}
if (!c.sub_id) {
const int fd = c.fd;
c.sub_id = hub_.subscribe([this, fd](const json& n) {
if (const auto it = conns_.find(fd); it != conns_.end()) send_text(*it->second, n);
});
}
std::vector<proto::Event> events;
proto::SessionSubscribeResult r;
r.ok = true;
for (const auto& ev : p->events) r.events.emplace_back(proto::to_string(ev));
for (const auto& ev : p->events) {
const auto name = proto::to_string(ev);
r.events.emplace_back(name);
if (auto e = proto::event_from_string(name)) events.push_back(*e);
}
hub_.set_filter(*c.sub_id, std::move(events), p->taskIds);
reply = proto::make_result(id, r);
return true;
}
@@ -423,6 +436,7 @@ void WsServer::flush(Conn& c) {
void WsServer::close_conn(int fd) {
if (const auto it = conns_.find(fd); it != conns_.end()) {
if (it->second->sub_id) hub_.unsubscribe(*it->second->sub_id);
loop_.del_fd(fd);
::close(fd);
conns_.erase(it);
daemon/src/rpc/ws_server.hpp
+5 -1
View File
@@ -8,12 +8,14 @@
#include <cstdint>
#include <memory>
#include <optional>
#include <string>
#include <system_error>
#include <unordered_map>
#include <nlohmann/json_fwd.hpp>
#include "rpc/event_hub.hpp"
#include "rpc/pairing.hpp"
#include "rpc/runtime_dir.hpp"
#include "rpc/ws_frame.hpp"
@@ -30,7 +32,7 @@ class EventLoop;
class WsServer {
public:
WsServer(EventLoop& loop, velox::proto::Dispatcher& dispatcher, velox::daemon::store::Db& db,
PairingApprover& approver, RuntimeDir runtime);
PairingApprover& approver, EventHub& hub, RuntimeDir runtime);
~WsServer();
WsServer(const WsServer&) = delete;
@@ -60,6 +62,7 @@ private:
bool authed = false;
std::string pairing_id;
std::string session_id;
std::optional<EventHub::SubId> sub_id;
};
void on_listener_readable();
@@ -80,6 +83,7 @@ private:
velox::proto::Dispatcher& dispatcher_;
velox::daemon::store::Db& db_;
PairingApprover& approver_;
EventHub& hub_;
RuntimeDir runtime_;
PairingRateLimiter rate_limiter_;