daemon: D3's remainder — rules.*, queue.reorder, schedule.*, limiter.*, download.update/refreshUrl
rules.list/rules.upsert: new store/rules.{hpp,cpp} (list in priority order; apply()
upserts+removes atomically, generating an id when absent, per the schema's own "never
leaves the table in a half-valid state"). Found and fixed along the way: migration
0001's rules table had no column for Rule.name at all — every rules.list/.upsert call
failed outright ("no such column: name"), unit tests included, since :memory: migrates
through the same path. Migration 0004 adds it.
queue.reorder: new store::Queues::reorder — taskIds must be an exact permutation of
the queue's current membership (compared as sorted sets) or nothing is written and
-32602 names the queue; a valid permutation rewrites every member's queue_position in
one transaction.
schedule.get/schedule.set: a thin wrapper over the queues.schedule column (already
read since D3b, never independently settable). nextRunAt is deliberately left unset —
computing it needs the same local-time, DST-aware window logic
sched/schedule_window.hpp's window_open() only has half of; called out rather than
approximated, and the field is optional.
limiter.get/limiter.set: backed by the same downloads.speedLimitEnabled/
downloads.speedLimitBps settings keys D9 already wired — one bag of truth, not two.
The new part is reaching the engine: EnginePort/TaskActionPort gain
set_global_speed_limit(bps) (0 = unlimited, TokenBucket's own convention), wired to
Engine::rate_limiter().set_global_limit(). Pushed live on every limiter.set *and* on
Scheduler::reload_config() so a limit from a previous run isn't silently unlimited
again after a restart. applyToRunning is accepted but has no lever to pull
differently — a single shared global bucket has no "next task only" variant. Also
found, not chased further: the schema's "globalBps:0 with enabled:true means 'stop
everything'" is the opposite of what TokenBucket does with rate_bps==0 (unlimited) —
a real discrepancy, but the schema says the GUI must not offer that combination.
download.update: "moving saveDir or filename moves the file on disk in the same
operation" — resolved and root-checked like download.add's destination, then the
.veloxpart/.veloxpart.meta pair (or the finished file, if complete) is moved via
rename, falling back to copy+remove across filesystems, only when the resolved
location actually differs. categoryId/queueId(appended to the new queue's run
order)/description/segments/bufferBytes/checksum apply through new
store::Tasks::apply_update.
download.refreshUrl: same async server-layer special-case as download.probe (a real
network round trip, same 30s deadline). Re-probes, flags contentChanged only when
size or validator are both known and actually differ, persists the new URL and probe
result, and swaps the URL on a live engine handle via a newly-widened
EnginePort::refresh_url (now takes headers too, matching DownloadHandle's real
signature — the seam had silently dropped them).
Found and documented, not fixed: the generated parser collapses "field absent" and
"field explicitly null" to the same nullopt for every optional<T> patch field
(DownloadUpdateParamsPatch, Settings) — both schemas document "an explicit null
clears the field" but neither handler can act on it because the wire distinction is
already gone by the time either sees the parsed struct. A generator-level gap
(PROTO's), not something to hand-route around locally.
Verified against real veloxd + tools/testserver: rules create/list, limiter.set
takes effect and reads back, schedule.set/get round-trips, queue.reorder against real
membership (and rejects a non-permutation), download.update renames+recategorizes a
task, download.refreshUrl swaps a paused task's URL and reports contentChanged
correctly. Full ctest: 55/55 (excluding the pre-existing, unrelated conformance
failure noted two commits back).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01GRDjHGgpYmMoPE2UFbe7pP
This commit is contained in:
@@ -0,0 +1,10 @@
|
||||
-- Migration 0004 — rules gets a name column.
|
||||
--
|
||||
-- 0001's rules table had no column for Rule.name (Rule.schema.json's own optional,
|
||||
-- maxLength-64 label field) — store/rules.cpp discovered this the hard way building
|
||||
-- rules.list/rules.upsert: every rules.list call failed outright ("no such column:
|
||||
-- name") because the SELECT it needs to project onto proto::Rule names a column that was
|
||||
-- never there. A plain ALTER TABLE ADD COLUMN suffices here (no CHECK constraint to
|
||||
-- rebuild around, unlike 0002/0003).
|
||||
|
||||
ALTER TABLE rules ADD COLUMN name TEXT;
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
#include <sqlite3.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstdio>
|
||||
#include <random>
|
||||
#include <string>
|
||||
@@ -85,6 +86,56 @@ DbResult<bool> Queues::set_state(std::string_view queue_id, std::string_view sta
|
||||
return sqlite3_changes(db_.raw()) > 0;
|
||||
}
|
||||
|
||||
DbResult<bool> Queues::set_schedule(std::string_view queue_id,
|
||||
const std::optional<proto::Schedule>& schedule) {
|
||||
auto st = db_.prepare("UPDATE queues SET schedule = ?2 WHERE queue_id = ?1");
|
||||
if (!st) return std::unexpected(st.error());
|
||||
if (auto b = st->bind(1, queue_id); !b) return std::unexpected(b.error());
|
||||
if (auto r = schedule ? st->bind(2, std::string_view(nlohmann::json(*schedule).dump()))
|
||||
: st->bind_null(2);
|
||||
!r)
|
||||
return std::unexpected(r.error());
|
||||
if (auto r = st->step(); !r) return std::unexpected(r.error());
|
||||
return sqlite3_changes(db_.raw()) > 0;
|
||||
}
|
||||
|
||||
DbResult<bool> Queues::reorder(std::string_view queue_id, const std::vector<std::string>& task_ids) {
|
||||
std::vector<std::string> current;
|
||||
{
|
||||
auto st = db_.prepare(
|
||||
"SELECT task_id FROM tasks WHERE queue_id = ?1 ORDER BY queue_position");
|
||||
if (!st) return std::unexpected(st.error());
|
||||
if (auto b = st->bind(1, queue_id); !b) return std::unexpected(b.error());
|
||||
for (;;) {
|
||||
auto row = st->step();
|
||||
if (!row) return std::unexpected(row.error());
|
||||
if (!*row) break;
|
||||
current.push_back(st->column_text(0));
|
||||
}
|
||||
}
|
||||
|
||||
// Exact permutation: same size, same members, order aside.
|
||||
std::vector<std::string> a = current, b = task_ids;
|
||||
std::sort(a.begin(), a.end());
|
||||
std::sort(b.begin(), b.end());
|
||||
if (a != b) return false;
|
||||
|
||||
auto txn = db_.transaction([&]() -> DbResult<void> {
|
||||
for (std::size_t i = 0; i < task_ids.size(); ++i) {
|
||||
auto st = db_.prepare("UPDATE tasks SET queue_position = ?2 WHERE task_id = ?1");
|
||||
if (!st) return std::unexpected(st.error());
|
||||
if (auto bd = st->bind(1, std::string_view(task_ids[i])); !bd)
|
||||
return std::unexpected(bd.error());
|
||||
if (auto bd = st->bind(2, static_cast<std::int64_t>(i)); !bd)
|
||||
return std::unexpected(bd.error());
|
||||
if (auto r = st->step(); !r) return std::unexpected(r.error());
|
||||
}
|
||||
return {};
|
||||
});
|
||||
if (!txn) return std::unexpected(txn.error());
|
||||
return true;
|
||||
}
|
||||
|
||||
DbResult<proto::Queue> Queues::upsert(proto::Queue queue) {
|
||||
if (queue.queueId.empty()) {
|
||||
std::random_device rd;
|
||||
|
||||
@@ -27,6 +27,18 @@ public:
|
||||
// id doesn't exist.
|
||||
DbResult<bool> set_state(std::string_view queue_id, std::string_view state);
|
||||
|
||||
// schedule.set: nullopt clears it (NULL = manual control, per the schema's own
|
||||
// words). false if the queue doesn't exist.
|
||||
DbResult<bool> set_schedule(std::string_view queue_id,
|
||||
const std::optional<velox::proto::Schedule>& schedule);
|
||||
|
||||
// queue.reorder: `task_ids` must be an exact permutation of the queue's current
|
||||
// membership (the schema's own words — "anything else is -32602 rather than a
|
||||
// partial reorder, so a stale drag from an out-of-date view cannot quietly reshuffle
|
||||
// the queue"). false (no write at all) if it isn't; true and queue_position rewritten
|
||||
// to match `task_ids`'s order if it is.
|
||||
DbResult<bool> reorder(std::string_view queue_id, const std::vector<std::string>& task_ids);
|
||||
|
||||
// "Omit queueId to create" (queue.upsert's own words) — an empty id generates one.
|
||||
// taskIds is ignored (membership changes only through download.update / queue.reorder,
|
||||
// per the schema's own note); a create defaults to 'stopped', a replace keeps the
|
||||
|
||||
@@ -318,6 +318,88 @@ DbResult<std::int64_t> Tasks::count() {
|
||||
return (*row) ? st->column_int(0) : 0;
|
||||
}
|
||||
|
||||
DbResult<bool> Tasks::apply_update(std::string_view task_id, const UpdatePatch& patch) {
|
||||
// One UPDATE per present field: simplest thing that's obviously correct for a
|
||||
// single-row edit with ~9 independent optional fields, and it means a field the
|
||||
// caller didn't touch is never rewritten with its own unchanged value (matters for
|
||||
// no-op-detection callers, though download.update doesn't currently need that).
|
||||
bool touched_any = false;
|
||||
auto run = [&](const char* sql, auto&& binder) -> DbResult<void> {
|
||||
auto st = db_.prepare(sql);
|
||||
if (!st) return std::unexpected(st.error());
|
||||
if (auto r = st->bind(1, task_id); !r) return std::unexpected(r.error());
|
||||
if (auto r = binder(*st); !r) return std::unexpected(r.error());
|
||||
if (auto r = st->step(); !r) return std::unexpected(r.error());
|
||||
touched_any = true;
|
||||
return {};
|
||||
};
|
||||
|
||||
if (patch.save_dir && patch.filename) {
|
||||
if (auto r = run("UPDATE tasks SET save_dir = ?2, filename = ?3 WHERE task_id = ?1",
|
||||
[&](Stmt& s) {
|
||||
if (auto b = s.bind(2, std::string_view(*patch.save_dir)); !b) return b;
|
||||
return s.bind(3, std::string_view(*patch.filename));
|
||||
});
|
||||
!r)
|
||||
return std::unexpected(r.error());
|
||||
}
|
||||
if (patch.category_id) {
|
||||
if (auto r = run("UPDATE tasks SET category_id = ?2 WHERE task_id = ?1",
|
||||
[&](Stmt& s) { return s.bind(2, std::string_view(*patch.category_id)); });
|
||||
!r)
|
||||
return std::unexpected(r.error());
|
||||
}
|
||||
if (patch.queue_id) {
|
||||
if (auto r = run("UPDATE tasks SET queue_id = ?2, queue_position = ?3 WHERE task_id = ?1",
|
||||
[&](Stmt& s) {
|
||||
if (auto b = s.bind(2, std::string_view(*patch.queue_id)); !b) return b;
|
||||
return patch.queue_position ? s.bind(3, *patch.queue_position)
|
||||
: s.bind_null(3);
|
||||
});
|
||||
!r)
|
||||
return std::unexpected(r.error());
|
||||
}
|
||||
if (patch.description) {
|
||||
if (auto r = run("UPDATE tasks SET description = ?2 WHERE task_id = ?1",
|
||||
[&](Stmt& s) { return s.bind(2, std::string_view(*patch.description)); });
|
||||
!r)
|
||||
return std::unexpected(r.error());
|
||||
}
|
||||
if (patch.req_segments) {
|
||||
if (auto r = run("UPDATE tasks SET req_segments = ?2 WHERE task_id = ?1",
|
||||
[&](Stmt& s) { return s.bind(2, *patch.req_segments); });
|
||||
!r)
|
||||
return std::unexpected(r.error());
|
||||
}
|
||||
if (patch.req_buffer_bytes) {
|
||||
if (auto r = run("UPDATE tasks SET req_buffer_bytes = ?2 WHERE task_id = ?1",
|
||||
[&](Stmt& s) { return s.bind(2, *patch.req_buffer_bytes); });
|
||||
!r)
|
||||
return std::unexpected(r.error());
|
||||
}
|
||||
if (patch.checksum_algo && patch.checksum_value) {
|
||||
if (auto r = run(
|
||||
"UPDATE tasks SET checksum_algo = ?2, checksum_value = ?3 WHERE task_id = ?1",
|
||||
[&](Stmt& s) {
|
||||
if (auto b = s.bind(2, std::string_view(*patch.checksum_algo)); !b) return b;
|
||||
return s.bind(3, std::string_view(*patch.checksum_value));
|
||||
});
|
||||
!r)
|
||||
return std::unexpected(r.error());
|
||||
}
|
||||
|
||||
return touched_any;
|
||||
}
|
||||
|
||||
DbResult<bool> Tasks::set_url(std::string_view task_id, std::string_view url) {
|
||||
auto st = db_.prepare("UPDATE tasks SET url = ?2 WHERE task_id = ?1");
|
||||
if (!st) return std::unexpected(st.error());
|
||||
if (auto r = st->bind(1, task_id); !r) return std::unexpected(r.error());
|
||||
if (auto r = st->bind(2, url); !r) return std::unexpected(r.error());
|
||||
if (auto r = st->step(); !r) return std::unexpected(r.error());
|
||||
return sqlite3_changes(db_.raw()) > 0;
|
||||
}
|
||||
|
||||
DbResult<bool> Tasks::has_active_duplicate(std::string_view url) {
|
||||
auto st = db_.prepare(
|
||||
"SELECT 1 FROM tasks WHERE url = ?1 "
|
||||
|
||||
@@ -85,6 +85,34 @@ public:
|
||||
// is almost always a mistake, e.g. two tabs triggering the same link).
|
||||
DbResult<bool> has_active_duplicate(std::string_view url);
|
||||
|
||||
// download.update's patch, already resolved by the caller (new save_dir/filename
|
||||
// canonicalized and root-checked, any file already moved on disk — this only writes
|
||||
// the row). Every field is applied when present; queue_position is written alongside
|
||||
// queue_id (nullopt leaves the existing position alone — the caller decides what
|
||||
// "moved into a queue" should set it to). Note: the generated parser collapses "field
|
||||
// absent" and "field explicitly null" to the same nullopt (DownloadUpdateParamsPatch
|
||||
// has no way to tell them apart on the wire as generated), so this — like the RPC
|
||||
// layer above it — can only ever set category_id/queue_id/description/checksum, never
|
||||
// clear them back to NULL through this call.
|
||||
struct UpdatePatch {
|
||||
std::optional<std::string> save_dir;
|
||||
std::optional<std::string> filename;
|
||||
std::optional<std::string> category_id;
|
||||
std::optional<std::string> queue_id;
|
||||
std::optional<std::int64_t> queue_position;
|
||||
std::optional<std::string> description;
|
||||
std::optional<std::int64_t> req_segments;
|
||||
std::optional<std::int64_t> req_buffer_bytes;
|
||||
std::optional<std::string> checksum_algo;
|
||||
std::optional<std::string> checksum_value;
|
||||
};
|
||||
DbResult<bool> apply_update(std::string_view task_id, const UpdatePatch& patch);
|
||||
|
||||
// download.refreshUrl: point the task at a freshly-issued URL. Separate from
|
||||
// apply_update/set_probe_result since neither owns the base `url` column — refreshUrl
|
||||
// is the one caller that changes it after creation.
|
||||
DbResult<bool> set_url(std::string_view task_id, std::string_view url);
|
||||
|
||||
// Byte-counter update from an engine progress tick — cheaper than a full row rewrite,
|
||||
// and keeps download.list / download.get current between state transitions.
|
||||
DbResult<bool> update_progress(std::string_view task_id, std::int64_t downloaded_bytes,
|
||||
|
||||
Reference in New Issue
Block a user