daemon: D3's remainder — rules.*, queue.reorder, schedule.*, limiter.*, download.update/refreshUrl

rules.list/rules.upsert: new store/rules.{hpp,cpp} (list in priority order; apply() upserts+removes atomically, generating an id when absent, per the schema's own "never leaves the table in a half-valid state"). Found and fixed along the way: migration 0001's rules table had no column for Rule.name at all — every rules.list/.upsert call failed outright ("no such column: name"), unit tests included, since :memory: migrates through the same path. Migration 0004 adds it. queue.reorder: new store::Queues::reorder — taskIds must be an exact permutation of the queue's current membership (compared as sorted sets) or nothing is written and -32602 names the queue; a valid permutation rewrites every member's queue_position in one transaction. schedule.get/schedule.set: a thin wrapper over the queues.schedule column (already read since D3b, never independently settable). nextRunAt is deliberately left unset — computing it needs the same local-time, DST-aware window logic sched/schedule_window.hpp's window_open() only has half of; called out rather than approximated, and the field is optional. limiter.get/limiter.set: backed by the same downloads.speedLimitEnabled/ downloads.speedLimitBps settings keys D9 already wired — one bag of truth, not two. The new part is reaching the engine: EnginePort/TaskActionPort gain set_global_speed_limit(bps) (0 = unlimited, TokenBucket's own convention), wired to Engine::rate_limiter().set_global_limit(). Pushed live on every limiter.set *and* on Scheduler::reload_config() so a limit from a previous run isn't silently unlimited again after a restart. applyToRunning is accepted but has no lever to pull differently — a single shared global bucket has no "next task only" variant. Also found, not chased further: the schema's "globalBps:0 with enabled:true means 'stop everything'" is the opposite of what TokenBucket does with rate_bps==0 (unlimited) — a real discrepancy, but the schema says the GUI must not offer that combination. download.update: "moving saveDir or filename moves the file on disk in the same operation" — resolved and root-checked like download.add's destination, then the .veloxpart/.veloxpart.meta pair (or the finished file, if complete) is moved via rename, falling back to copy+remove across filesystems, only when the resolved location actually differs. categoryId/queueId(appended to the new queue's run order)/description/segments/bufferBytes/checksum apply through new store::Tasks::apply_update. download.refreshUrl: same async server-layer special-case as download.probe (a real network round trip, same 30s deadline). Re-probes, flags contentChanged only when size or validator are both known and actually differ, persists the new URL and probe result, and swaps the URL on a live engine handle via a newly-widened EnginePort::refresh_url (now takes headers too, matching DownloadHandle's real signature — the seam had silently dropped them). Found and documented, not fixed: the generated parser collapses "field absent" and "field explicitly null" to the same nullopt for every optional<T> patch field (DownloadUpdateParamsPatch, Settings) — both schemas document "an explicit null clears the field" but neither handler can act on it because the wire distinction is already gone by the time either sees the parsed struct. A generator-level gap (PROTO's), not something to hand-route around locally. Verified against real veloxd + tools/testserver: rules create/list, limiter.set takes effect and reads back, schedule.set/get round-trips, queue.reorder against real membership (and rejects a non-permutation), download.update renames+recategorizes a task, download.refreshUrl swaps a paused task's URL and reports contentChanged correctly. Full ctest: 55/55 (excluding the pre-existing, unrelated conformance failure noted two commits back). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
2026-09-12 14:26:18 +04:00
parent 6632b75099
commit 4f6c0cc9d2
20 changed files with 948 additions and 19 deletions
@@ -819,8 +819,120 @@ VeloxDispatcher::on_download_start(const proto::DownloadStartParams& params) {
    return result;
 }
 proto::HandlerResult<proto::TaskSummary>
-VeloxDispatcher::on_download_update(const proto::DownloadUpdateParams&) {
-    return not_implemented<proto::TaskSummary>("download.update");
+VeloxDispatcher::on_download_update(const proto::DownloadUpdateParams& params) {
+    store::Tasks tasks(db_);
+    auto got = tasks.get(params.taskId);
+    if (!got)
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "download.update: " + got.error().message});
+    if (!got->has_value())
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::TaskNotFound, "no such task",
+                                                   nlohmann::json{{"taskId", params.taskId}}});
+    const store::TaskRow& row = **got;
+    const auto& patch = params.patch;
+
+    if (patch.segments && (*patch.segments < 1 || *patch.segments > 32)) {
+        return std::unexpected(proto::HandlerError{
+            proto::ErrorCode::InvalidParams, "segments must be between 1 and 32",
+            nlohmann::json{{"value", *patch.segments}}});
+    }
+    if (patch.bufferBytes && (*patch.bufferBytes < 65536 || *patch.bufferBytes > 16777216)) {
+        return std::unexpected(proto::HandlerError{
+            proto::ErrorCode::InvalidParams, "bufferBytes must be between 65536 and 16777216",
+            nlohmann::json{{"value", *patch.bufferBytes}}});
+    }
+
+    store::Tasks::UpdatePatch update;
+
+    // "Moving saveDir or filename moves the file on disk in the same operation" (the
+    // schema's own words). Resolved and root-checked exactly like download.add's own
+    // destination; only actually touches the filesystem if the resolved location differs
+    // from where the task already is.
+    if (patch.filename || patch.saveDir) {
+        store::Settings settings(db_);
+        std::string dir = expand_tilde(patch.saveDir.value_or(row.save_dir));
+        std::string leaf = patch.filename.value_or(row.filename);
+        std::vector<std::string> roots;
+        for (const auto& r : settings.get_string_array("saveTo.allowedRoots"))
+            if (auto c = fs::canonicalize_root(r)) roots.push_back(*c);
+
+        auto target = fs::resolve_target(dir, leaf, roots);
+        if (!target) {
+            return std::unexpected(proto::HandlerError{
+                proto::ErrorCode::InvalidPath, target.error().message,
+                nlohmann::json{{"path", patch.saveDir.value_or(dir)}}});
+        }
+
+        if (target->dir != row.save_dir || target->leaf != row.filename) {
+            namespace fsn = std::filesystem;
+            const std::string old_base = row.save_dir + "/" + row.filename;
+            const std::string new_base = target->dir + "/" + target->leaf;
+
+            // move_one: rename, falling back to copy+remove across filesystems (EXDEV).
+            // Missing source (nothing to move at this path yet) is not an error.
+            auto move_one = [](const std::string& from, const std::string& to) -> bool {
+                std::error_code ec;
+                if (!fsn::exists(from, ec)) return true;
+                fsn::rename(from, to, ec);
+                if (!ec) return true;
+                ec.clear();
+                fsn::copy_file(from, to, fsn::copy_options::overwrite_existing, ec);
+                if (ec) return false;
+                fsn::remove(from, ec);
+                return true;
+            };
+
+            const bool moved = row.state == "complete"
+                                   ? move_one(old_base, new_base)
+                                   : (move_one(old_base + ".veloxpart", new_base + ".veloxpart") &&
+                                      move_one(old_base + ".veloxpart.meta", new_base + ".veloxpart.meta"));
+            if (!moved) {
+                return std::unexpected(proto::HandlerError{
+                    proto::ErrorCode::InternalError, "could not move the file to its new location",
+                    nlohmann::json{{"path", new_base}}});
+            }
+            update.save_dir = target->dir;
+            update.filename = target->leaf;
+        }
+    }
+
+    if (patch.categoryId) update.category_id = patch.categoryId;
+    if (patch.queueId) {
+        update.queue_id = patch.queueId;
+        // Appended to the end of the new queue's run order.
+        auto page = tasks.list(
+            [&] {
+                proto::TaskFilter f;
+                f.queueId = *patch.queueId;
+                return f;
+            }(),
+            std::nullopt, 0, 10000);
+        std::int64_t next_pos = 0;
+        if (page)
+            for (const auto& r : page->rows)
+                if (r.queue_position) next_pos = std::max<std::int64_t>(next_pos, *r.queue_position + 1);
+        update.queue_position = next_pos;
+    }
+    if (patch.description) update.description = patch.description;
+    if (patch.segments) update.req_segments = patch.segments;
+    if (patch.bufferBytes) update.req_buffer_bytes = patch.bufferBytes;
+    if (patch.checksum) {
+        update.checksum_algo = std::string(proto::to_string(patch.checksum->algorithm));
+        update.checksum_value = patch.checksum->value;
+    }
+
+    if (auto r = tasks.apply_update(params.taskId, update); !r) {
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "download.update: " + r.error().message});
+    }
+
+    auto after = tasks.get(params.taskId);
+    if (!after || !after->has_value()) {
+        return std::unexpected(
+            proto::HandlerError{proto::ErrorCode::InternalError, "download.update: row vanished"});
+    }
+    if (on_mutation_) on_mutation_();
+    return store::to_summary(**after);
 }
 proto::HandlerResult<proto::GrabberHarvestResult>
 VeloxDispatcher::on_grabber_harvest(const proto::GrabberHarvestParams&) {
@@ -835,10 +947,24 @@ VeloxDispatcher::on_grabber_status(const proto::GrabberStatusParams&) {
    return not_implemented<proto::GrabberStatusResult>("grabber.status");
 }
 proto::HandlerResult<proto::Limiter> VeloxDispatcher::on_limiter_get(const proto::LimiterGetParams&) {
-    return not_implemented<proto::Limiter>("limiter.get");
+    store::Settings settings(db_);
+    proto::Limiter r;
+    r.enabled = settings.get_bool("downloads.speedLimitEnabled");
+    r.globalBps = settings.get_int("downloads.speedLimitBps");
+    return r;
 }
-proto::HandlerResult<proto::Limiter> VeloxDispatcher::on_limiter_set(const proto::Limiter&) {
-    return not_implemented<proto::Limiter>("limiter.set");
+proto::HandlerResult<proto::Limiter> VeloxDispatcher::on_limiter_set(const proto::Limiter& params) {
+    store::Settings settings(db_);
+    (void)settings.set_raw("downloads.speedLimitEnabled", params.enabled ? "true" : "false");
+    (void)settings.set_raw("downloads.speedLimitBps", std::to_string(params.globalBps));
+
+    // The bucket's own convention: 0 == unlimited. `enabled: false` means "no limit"
+    // regardless of what globalBps happens to hold (the GUI is expected to keep it around
+    // for when the user re-enables it, not to reset it to 0).
+    if (actions_) actions_->set_global_speed_limit(params.enabled ? static_cast<std::uint64_t>(params.globalBps) : 0);
+
+    if (on_mutation_) on_mutation_();
+    return params;
 }
 proto::HandlerResult<proto::MediaAddVariantResult>
 VeloxDispatcher::on_media_addVariant(const proto::MediaAddVariantParams&) {
@@ -860,8 +986,33 @@ VeloxDispatcher::on_queue_list(const proto::QueueListParams&) {
    return r;
 }
 proto::HandlerResult<proto::QueueReorderResult>
-VeloxDispatcher::on_queue_reorder(const proto::QueueReorderParams&) {
-    return not_implemented<proto::QueueReorderResult>("queue.reorder");
+VeloxDispatcher::on_queue_reorder(const proto::QueueReorderParams& params) {
+    store::Queues queues(db_);
+    auto exists = queues.get(params.queueId);
+    if (!exists)
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "queue.reorder: " + exists.error().message});
+    if (!exists->has_value())
+        return std::unexpected(proto::HandlerError{
+            proto::ErrorCode::InvalidParams, "no such queue",
+            nlohmann::json{{"queueId", params.queueId}}});
+
+    auto applied = queues.reorder(params.queueId, params.taskIds);
+    if (!applied)
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "queue.reorder: " + applied.error().message});
+    if (!*applied) {
+        return std::unexpected(proto::HandlerError{
+            proto::ErrorCode::InvalidParams,
+            "taskIds must be an exact permutation of the queue's current membership",
+            nlohmann::json{{"queueId", params.queueId}}});
+    }
+
+    auto after = queues.get(params.queueId);
+    proto::QueueReorderResult r;
+    if (after && after->has_value()) r.queue = **after;
+    if (on_mutation_) on_mutation_();
+    return r;
 }
 proto::HandlerResult<proto::QueueStartResult>
 VeloxDispatcher::on_queue_start(const proto::QueueStartParams& params) {
@@ -926,19 +1077,87 @@ VeloxDispatcher::on_queue_upsert(const proto::QueueUpsertParams& params) {
 }
 proto::HandlerResult<proto::RulesListResult>
 VeloxDispatcher::on_rules_list(const proto::RulesListParams&) {
-    return not_implemented<proto::RulesListResult>("rules.list");
+    store::Rules rules(db_);
+    auto items = rules.list();
+    if (!items)
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "rules.list: " + items.error().message});
+    proto::RulesListResult r;
+    r.items = std::move(*items);
+    return r;
 }
 proto::HandlerResult<proto::RulesUpsertResult>
-VeloxDispatcher::on_rules_upsert(const proto::RulesUpsertParams&) {
-    return not_implemented<proto::RulesUpsertResult>("rules.upsert");
+VeloxDispatcher::on_rules_upsert(const proto::RulesUpsertParams& params) {
+    store::Rules rules(db_);
+    auto items = rules.apply(params.upsert, params.remove.value_or(std::vector<std::string>{}));
+    if (!items)
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "rules.upsert: " + items.error().message});
+    proto::RulesUpsertResult r;
+    r.items = std::move(*items);
+    return r;
 }
 proto::HandlerResult<proto::ScheduleGetResult>
-VeloxDispatcher::on_schedule_get(const proto::ScheduleGetParams&) {
-    return not_implemented<proto::ScheduleGetResult>("schedule.get");
+VeloxDispatcher::on_schedule_get(const proto::ScheduleGetParams& params) {
+    store::Queues queues(db_);
+    proto::ScheduleGetResult r;
+    if (params.queueId) {
+        auto q = queues.get(*params.queueId);
+        if (!q)
+            return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                       "schedule.get: " + q.error().message});
+        if (q->has_value()) {
+            proto::ScheduleGetResultItemsItem item;
+            item.queueId = (*q)->queueId;
+            item.schedule = (*q)->schedule;
+            r.items.push_back(std::move(item));
+        }
+        // No such queue: an empty items list rather than an error — same "the caller asked
+        // about something specific and there's nothing to say about it" shape download.get
+        // uses -32010 for, but schedule.get's own x-errors lists only -32003, so an empty
+        // result (not "every schedule", just this one queue's, which doesn't exist) is the
+        // faithful answer within what the schema actually allows.
+        return r;
+    }
+
+    auto all = queues.list();
+    if (!all)
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "schedule.get: " + all.error().message});
+    for (const auto& q : *all) {
+        proto::ScheduleGetResultItemsItem item;
+        item.queueId = q.queueId;
+        item.schedule = q.schedule;
+        r.items.push_back(std::move(item));
+    }
+    return r;
 }
 proto::HandlerResult<proto::ScheduleSetResult>
-VeloxDispatcher::on_schedule_set(const proto::ScheduleSetParams&) {
-    return not_implemented<proto::ScheduleSetResult>("schedule.set");
+VeloxDispatcher::on_schedule_set(const proto::ScheduleSetParams& params) {
+    store::Queues queues(db_);
+    auto exists = queues.get(params.queueId);
+    if (!exists)
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "schedule.set: " + exists.error().message});
+    if (!exists->has_value())
+        return std::unexpected(proto::HandlerError{
+            proto::ErrorCode::InvalidParams, "no such queue",
+            nlohmann::json{{"queueId", params.queueId}}});
+
+    auto applied = queues.set_schedule(params.queueId, params.schedule);
+    if (!applied)
+        return std::unexpected(proto::HandlerError{proto::ErrorCode::InternalError,
+                                                   "schedule.set: " + applied.error().message});
+
+    proto::ScheduleSetResult r;
+    r.queueId = params.queueId;
+    r.schedule = params.schedule;
+    // nextRunAt is left unset: computing it correctly needs the same local-time,
+    // DST-aware window logic sched/schedule_window.hpp's window_open() already has half
+    // of (open-right-now, not next-transition) — worth building as its own pass rather
+    // than an approximate guess here. The field is optional; nullopt is a legal answer.
+    if (on_mutation_) on_mutation_();
+    return r;
 }
 proto::HandlerResult<proto::SettingsGetResult>
 VeloxDispatcher::on_settings_get(const proto::SettingsGetParams& params) {
@@ -67,6 +67,19 @@ public:
        std::function<void(velox::proto::HandlerResult<velox::proto::DownloadProbeResult>)>
            done) = 0;

+    // download.refreshUrl ("IDM's 'Refresh Download Address'"): same async reasoning and
+    // the same server-layer special-case as probe_now — a real network round trip, up to
+    // the schema's own 30s x-deadlineMs. Re-probes the new URL, compares size/validator
+    // against what the task already has on record (contentChanged), persists the new URL
+    // and probe result, and — if the task holds a live engine handle — swaps its URL
+    // in-flight without losing progress.
+    virtual void refresh_url(
+        const std::string& wire_id, const std::string& url,
+        const std::optional<velox::proto::Headers>& headers,
+        const std::optional<std::vector<velox::proto::Cookie>>& cookies,
+        std::function<void(velox::proto::HandlerResult<velox::proto::DownloadRefreshUrlResult>)>
+            done) = 0;
+
    // settings.set of a connection.* key: re-read connection.maxConcurrentDownloads /
    // maxActiveSegments and the per-host cap table, push the new caps to the engine and
    // the governor (Scheduler::reload_config's own doc comment names this exact trigger).
@@ -74,6 +87,10 @@ public:
    // sched::Scheduler's own already-public reload_config() (returns DbResult<void>,
    // consumed by main.cpp and a unit test — kept as-is rather than reshaped to fit here).
    virtual void apply_settings_reload() = 0;
+
+    // limiter.set: pushed straight to the engine's shared global token bucket. 0 means
+    // unlimited (the bucket's own convention).
+    virtual void set_global_speed_limit(std::uint64_t bps) = 0;
 };

 }  // namespace velox::daemon::rpc
@@ -207,6 +207,10 @@ void UdsServer::handle_line(Conn& c, const std::string& line) {
        handle_download_probe(c, req);
        return;
    }
+    if (method == "download.refreshUrl") {
+        handle_download_refreshUrl(c, req);
+        return;
+    }

    // Everything else: the generated router. It returns a null json for a notification
    // that needs no reply.
@@ -244,6 +248,37 @@ void UdsServer::handle_download_probe(Conn& c, const json& request) {
        });
 }

+void UdsServer::handle_download_refreshUrl(Conn& c, const json& request) {
+    const json id = request.contains("id") ? request.at("id") : json(nullptr);
+    const json params_json = request.contains("params") ? request.at("params") : json::object();
+
+    auto parsed = proto::parse<proto::DownloadRefreshUrlParams>(params_json, "params");
+    if (!parsed) {
+        queue_reply(c, rpc_error(id, proto::ErrorCode::InvalidParams, parsed.error().message,
+                                 json{{"path", parsed.error().path}}));
+        return;
+    }
+    if (!actions_) {
+        queue_reply(c, rpc_error(id, proto::ErrorCode::InternalError,
+                                 "not implemented in this build: download.refreshUrl"));
+        return;
+    }
+
+    const int fd = c.fd;
+    actions_->refresh_url(
+        parsed->taskId, parsed->url, parsed->headers, parsed->cookies,
+        [this, fd, id](proto::HandlerResult<proto::DownloadRefreshUrlResult> r) {
+            auto it = conns_.find(fd);
+            if (it == conns_.end()) return;  // client gone while the probe was outstanding
+            if (r) {
+                queue_reply(*it->second, proto::make_result(id, *r));
+            } else {
+                queue_reply(*it->second,
+                           rpc_error(id, r.error().code, r.error().message, r.error().data));
+            }
+        });
+}
+
 bool UdsServer::handle_session_method(Conn& c, const std::string& method, const json& request,
                                      json& reply) {
    const json id = request.contains("id") ? request.at("id") : json(nullptr);
@@ -76,6 +76,9 @@ private:
    // Queues the reply itself, later, when actions_->probe_now()'s callback fires; does
    // nothing if the connection is gone by then (client disconnected mid-probe).
    void handle_download_probe(Conn& c, const nlohmann::json& request);
+    // Same reasoning as handle_download_probe — a real network round trip, same 30s
+    // x-deadlineMs.
+    void handle_download_refreshUrl(Conn& c, const nlohmann::json& request);

    void queue_reply(Conn& c, const nlohmann::json& reply);
    void flush(Conn& c);
@@ -267,6 +267,10 @@ void WsServer::handle_rpc(Conn& c, const std::string& text) {
        handle_download_probe(c, req);
        return;
    }
+    if (method == "download.refreshUrl") {
+        handle_download_refreshUrl(c, req);
+        return;
+    }

    json reply = proto::dispatch(dispatcher_, proto::Transport::Ws, req);
    if (!reply.is_null()) send_text(c, reply);
@@ -302,6 +306,37 @@ void WsServer::handle_download_probe(Conn& c, const json& request) {
        });
 }

+void WsServer::handle_download_refreshUrl(Conn& c, const json& request) {
+    const json id = request.contains("id") ? request.at("id") : json(nullptr);
+    const json params_json = request.contains("params") ? request.at("params") : json::object();
+
+    auto parsed = proto::parse<proto::DownloadRefreshUrlParams>(params_json, "params");
+    if (!parsed) {
+        send_text(c, rpc_error(id, proto::ErrorCode::InvalidParams, parsed.error().message,
+                               json{{"path", parsed.error().path}}));
+        return;
+    }
+    if (!actions_) {
+        send_text(c, rpc_error(id, proto::ErrorCode::InternalError,
+                               "not implemented in this build: download.refreshUrl"));
+        return;
+    }
+
+    const int fd = c.fd;
+    actions_->refresh_url(
+        parsed->taskId, parsed->url, parsed->headers, parsed->cookies,
+        [this, fd, id](proto::HandlerResult<proto::DownloadRefreshUrlResult> r) {
+            auto it = conns_.find(fd);
+            if (it == conns_.end()) return;  // client gone while the probe was outstanding
+            if (r) {
+                send_text(*it->second, proto::make_result(id, *r));
+            } else {
+                send_text(*it->second,
+                         rpc_error(id, r.error().code, r.error().message, r.error().data));
+            }
+        });
+}
+
 bool WsServer::handle_session_ws(Conn& c, const std::string& method, const json& request,
                                 json& reply) {
    const json id = request.contains("id") ? request.at("id") : json(nullptr);
@@ -81,6 +81,7 @@ private:
    // See UdsServer::handle_download_probe — same reasoning, same pattern, duplicated per
    // transport because each owns its own Conn/send mechanics.
    void handle_download_probe(Conn& c, const nlohmann::json& request);
+    void handle_download_refreshUrl(Conn& c, const nlohmann::json& request);

    void send_text(Conn& c, const nlohmann::json& value);
    void send_frame(Conn& c, WsOpcode op, std::string_view payload);