core: post the engine API for DAEMON review (pre-stage-7)
The download entry point the AGENT-CORE brief asked for on day one and that slipped. DAEMON has an RPC surface and a store and, until this is agreed, nothing in velox::core to call. vdm/task/download.hpp — DownloadSpec (the resolved subset DAEMON hands in: absolute save_path, verbatim browser headers, requested segments/buffer, optional probe_hint / checksum / auth, allow_resume), EngineState (the CORE-owned subset of the wire TaskState), Progress / SegmentProgress, DownloadCallbacks (on_progress <=4 Hz, on_state for every transition incl. auto-pauses, on_auth_required, on_decision_needed, on_finished last), DownloadHandle (pause/resume/cancel — idempotent per the ADR 0013 signature — plus provide_auth / decide / refresh_url, and synchronous state()/progress() snapshots). vdm/engine.hpp — Engine: start(spec, callbacks) -> handle, segment_budget() (DAEMON's sched/ admission surface, ADR 0011), live connection.* setters, a standalone probe() on the pool outside the segment budget. core/docs/engine-api-m1.md — the review doc: field semantics, the state machine, threading/lifetime rules (which thread callbacks arrive on, what is legal from inside one, handle/engine lifetime), the shared-`paused` idempotency contract as a signature, and five open questions for DAEMON. Value types compile and are covered by api_compiles_test; Engine / DownloadHandle bodies land in stage 8, built against whatever DAEMON signs off here. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
This commit is contained in:
Claude Sonnet 5
@@ -1,8 +1,12 @@
|
||||
# `libveloxcore` — public API
|
||||
|
||||
**Status: M1 in progress.** Only `util/` is landed. The download-facing API
|
||||
(`DownloadSpec`, `DownloadTask`, probe, typed callbacks) arrives with later stages and
|
||||
is reviewed by DAEMON before M2 (AGENT-CORE DoD).
|
||||
**Status: M1 in progress.** `util/`, `net/` (http_client, probe, url, content_disposition),
|
||||
`io/` (sparse_file, write_buffer), `meta/veloxpart`, and `segment/` (segmenter, budget)
|
||||
are landed. The **download entry point** — `vdm::Engine`, `vdm::task::DownloadSpec` /
|
||||
`DownloadHandle` / `DownloadCallbacks` — is sketched in `vdm/engine.hpp` and
|
||||
`vdm/task/download.hpp` and **out for DAEMON review**: see
|
||||
[`core/docs/engine-api-m1.md`](../../docs/engine-api-m1.md). Bodies land in CORE stage 8;
|
||||
build against the value types now.
|
||||
|
||||
Layering (CLAUDE.md §3): this library knows nothing about JSON, SQL, Qt, or RPC. Input is
|
||||
a spec value; output is bytes on disk plus typed callbacks. DAEMON projects engine state
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
// vdm/engine.hpp — the download engine's single entry point. REVIEW SKETCH (stage 7
|
||||
// pre-work); bodies land in stage 8. See core/docs/engine-api-m1.md.
|
||||
//
|
||||
// The engine owns the HTTP client, the probe pool, the segment budget, and the disk I/O.
|
||||
// Its input is a DownloadSpec; its output is bytes at save_path plus typed callbacks. No
|
||||
// JSON, no SQL, no Qt, no RPC — DAEMON projects the callbacks onto the wire contract.
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_ENGINE_HPP
|
||||
#define VDM_ENGINE_HPP
|
||||
|
||||
#include <cstdint>
|
||||
#include <memory>
|
||||
|
||||
#include "vdm/segment/budget.hpp"
|
||||
#include "vdm/task/download.hpp"
|
||||
|
||||
namespace vdm {
|
||||
|
||||
class Engine {
|
||||
public:
|
||||
struct Config {
|
||||
// Defaults used when a DownloadSpec leaves the field unset. Live-adjustable via
|
||||
// the setters below (they take effect on the next segment (re)assignment, not by
|
||||
// resizing an in-flight buffer).
|
||||
std::uint32_t default_segments = 8; // connection.maxSegmentsPerDownload
|
||||
std::uint64_t default_buffer_bytes = 1u << 20; // connection.bufferBytes (1 MiB)
|
||||
std::uint64_t min_segment_bytes = 1u << 20; // never split below this
|
||||
std::uint64_t max_total_buffer_bytes = 128ull << 20; // connection.maxTotalBufferBytes
|
||||
std::uint32_t max_active_segments = 32; // connection.maxActiveSegments
|
||||
std::uint32_t probe_pool_size = 4; // ADR 0011 §5, outside the budget
|
||||
long default_max_retries = 10; // per segment
|
||||
std::uint32_t http_workers = 0; // 0 => hardware-derived (<=4)
|
||||
};
|
||||
|
||||
Engine(); // default Config
|
||||
explicit Engine(Config cfg);
|
||||
~Engine(); // cancels every running task and joins before returning
|
||||
|
||||
Engine(const Engine &) = delete;
|
||||
Engine &operator=(const Engine &) = delete;
|
||||
|
||||
// Start a download. Returns immediately with a handle; the task begins in `probing`
|
||||
// (or `connecting` when spec.probe_hint is supplied). Every failure — bad URL, DNS,
|
||||
// an unwritable save_path — is delivered through callbacks.on_finished, never thrown.
|
||||
[[nodiscard]] task::DownloadHandle start(task::DownloadSpec spec,
|
||||
task::DownloadCallbacks callbacks);
|
||||
|
||||
// The global segment allocator. DAEMON's scheduler drives admission through this
|
||||
// (set_max_active_segments / set_host_segment_cap / set_task_order) and reads
|
||||
// occupancy from it (budget() / segments_active() / starved_tasks() /
|
||||
// on_budget_changed). See ADR 0011.
|
||||
[[nodiscard]] segment::SegmentBudget &segment_budget() noexcept;
|
||||
|
||||
// Live settings (connection.* changes from settings.set). Each affects future work.
|
||||
void set_default_segments(std::uint32_t n);
|
||||
void set_default_buffer_bytes(std::uint64_t bytes);
|
||||
void set_max_total_buffer_bytes(std::uint64_t bytes);
|
||||
void set_probe_pool_size(std::uint32_t n);
|
||||
|
||||
// A standalone probe for the File Info dialog, on the same pool as spec-less probes
|
||||
// (never charged against the segment budget). capture.offer's 750 ms deadline is
|
||||
// DAEMON's to enforce — it should answer `ignore` and probe after, never block on
|
||||
// this.
|
||||
void probe(net::ProbeRequest req, std::function<void(Result<net::ProbeResult>)> done);
|
||||
|
||||
private:
|
||||
struct Impl;
|
||||
std::unique_ptr<Impl> impl_;
|
||||
};
|
||||
|
||||
} // namespace vdm
|
||||
|
||||
#endif // VDM_ENGINE_HPP
|
||||
@@ -0,0 +1,213 @@
|
||||
// vdm/task/download.hpp — the public download API: what DAEMON hands the engine and how
|
||||
// the engine reports back. REVIEW SKETCH (stage 7 pre-work) — value types are final
|
||||
// enough to build against; Engine/DownloadHandle bodies land in stage 8.
|
||||
//
|
||||
// See core/docs/engine-api-m1.md for the threading, lifetime, and pause/resume/cancel
|
||||
// contract that goes with these signatures.
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_TASK_DOWNLOAD_HPP
|
||||
#define VDM_TASK_DOWNLOAD_HPP
|
||||
|
||||
#include <chrono>
|
||||
#include <cstdint>
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "vdm/ids.hpp"
|
||||
#include "vdm/net/http_types.hpp"
|
||||
#include "vdm/net/probe.hpp"
|
||||
#include "vdm/segment/segmenter.hpp"
|
||||
#include "vdm/util/error.hpp"
|
||||
#include "vdm/util/result.hpp"
|
||||
|
||||
namespace vdm {
|
||||
class Engine; // owns and fills DownloadHandle (see vdm/engine.hpp)
|
||||
} // namespace vdm
|
||||
|
||||
namespace vdm::task {
|
||||
|
||||
// --- input ---------------------------------------------------------------------------
|
||||
|
||||
struct Checksum {
|
||||
enum class Algo { md5, sha1, sha256 };
|
||||
Algo algo = Algo::sha256;
|
||||
std::string hex; // lower-case, no separators
|
||||
};
|
||||
|
||||
// Everything the engine needs to run ONE download. DAEMON has already run the rules
|
||||
// engine, canonicalised the path and checked it against the allowed roots, and resolved
|
||||
// the filename — `save_path` is absolute and final. `<save_path>.veloxpart` and
|
||||
// `<save_path>.veloxpart.meta` live beside it during the transfer.
|
||||
struct DownloadSpec {
|
||||
std::string url;
|
||||
std::vector<std::string> mirrors; // alternative URLs for the same bytes
|
||||
|
||||
std::vector<net::HeaderField> headers; // the browser's, verbatim
|
||||
std::vector<net::Cookie> cookies;
|
||||
std::string referrer;
|
||||
std::string user_agent;
|
||||
|
||||
std::string save_path; // absolute; the engine never canonicalises or root-checks
|
||||
|
||||
std::optional<std::uint32_t> segments; // requested 1..32; nullopt => engine default
|
||||
std::optional<std::uint64_t> buffer_bytes; // requested per segment; nullopt => default
|
||||
|
||||
net::ProxyConfig proxy;
|
||||
net::AuthConfig auth; // credentials known up front (e.g. from the Secret Service);
|
||||
// leave scheme == none to be prompted on a 401/407
|
||||
|
||||
std::optional<Checksum> checksum; // verified during `verifying`; mismatch => failed
|
||||
|
||||
// DAEMON usually probed already for the File Info dialog. Pass it to skip a second
|
||||
// probe; the engine still revalidates on resume. nullopt => the engine probes.
|
||||
std::optional<net::ProbeResult> probe_hint;
|
||||
|
||||
bool allow_resume = true; // if a valid .veloxpart.meta sits beside save_path, resume
|
||||
// from it; false starts fresh and overwrites
|
||||
std::optional<long> max_retries; // per-segment; nullopt => engine default (10)
|
||||
};
|
||||
|
||||
// --- lifecycle (the CORE-owned subset of the wire TaskState; ADR 0013 §1) -------------
|
||||
|
||||
enum class EngineState {
|
||||
probing,
|
||||
connecting,
|
||||
downloading,
|
||||
paused, // shared with DAEMON; entered by either side, idempotently
|
||||
retry_wait, // the engine's own backoff timer
|
||||
assembling, // no-op rename in M1; a real mux step for HLS/DASH (M4)
|
||||
verifying, // checksum
|
||||
complete, // terminal
|
||||
failed, // terminal
|
||||
cancelled, // terminal; always DAEMON- or user-initiated
|
||||
};
|
||||
|
||||
[[nodiscard]] constexpr bool is_terminal(EngineState s) noexcept {
|
||||
return s == EngineState::complete || s == EngineState::failed || s == EngineState::cancelled;
|
||||
}
|
||||
|
||||
// --- progress ---------------------------------------------------------------------------
|
||||
|
||||
struct SegmentProgress {
|
||||
std::uint32_t index = 0;
|
||||
std::uint64_t start = 0;
|
||||
std::uint64_t end = 0; // inclusive
|
||||
std::uint64_t completed = 0;
|
||||
std::uint64_t speed_bps = 0;
|
||||
segment::SegState state = segment::SegState::idle;
|
||||
};
|
||||
|
||||
struct Progress {
|
||||
std::uint64_t downloaded = 0;
|
||||
std::optional<std::uint64_t> total; // absent for a chunked source until it ends
|
||||
std::uint64_t speed_bps = 0; // aggregate over the last window
|
||||
std::optional<std::uint32_t> eta_seconds;
|
||||
|
||||
std::uint32_t effective_segments = 0; // slots the budget granted (held)
|
||||
std::uint64_t effective_buffer_bytes = 0; // per segment, after the maxTotal clamp
|
||||
std::vector<SegmentProgress> segments;
|
||||
};
|
||||
|
||||
// --- interaction callbacks ----------------------------------------------------------
|
||||
|
||||
// A 401/407. The task has already auto-paused (state -> paused, error == auth_required).
|
||||
// DAEMON collects credentials and calls handle.provide_auth().
|
||||
struct AuthChallenge {
|
||||
std::string host;
|
||||
std::string realm;
|
||||
enum class Scheme { basic, digest, ntlm, negotiate, unknown };
|
||||
Scheme scheme = Scheme::unknown;
|
||||
};
|
||||
|
||||
// The server's copy changed under us (a 200 where a 206 was expected, or an If-Range /
|
||||
// ETag mismatch on resume — docs/04 §5), or the range metadata went stale (416). The
|
||||
// task has auto-paused. DAEMON asks the user and calls handle.decide().
|
||||
struct DecisionRequest {
|
||||
enum class Kind { server_file_changed, range_metadata_stale };
|
||||
Kind kind = Kind::server_file_changed;
|
||||
std::string detail; // human-readable, for the dialog body
|
||||
};
|
||||
|
||||
enum class Decision {
|
||||
restart, // discard the partial file, download again from scratch
|
||||
keep_partial, // trust what is on disk and continue (the user's risk)
|
||||
abort, // give up: the task goes to `failed`
|
||||
};
|
||||
|
||||
struct DownloadOutcome {
|
||||
std::string final_path;
|
||||
std::uint64_t bytes = 0;
|
||||
std::optional<std::string> sha256_hex; // present when a checksum was requested/derived
|
||||
std::chrono::milliseconds elapsed{0};
|
||||
};
|
||||
|
||||
// All callbacks are optional. See core/docs/engine-api-m1.md for the rules; in short:
|
||||
// they arrive on an engine thread, are serialised per task, must not block, and must not
|
||||
// re-enter THIS task's handle synchronously.
|
||||
struct DownloadCallbacks {
|
||||
// Coalesced to <= 4 Hz per task (matches the wire event.task.progress cadence).
|
||||
std::function<void(const Progress &)> on_progress;
|
||||
|
||||
// Every lifecycle transition, including the auto-pauses above (to == paused with a
|
||||
// populated ErrorInfo) and terminals.
|
||||
std::function<void(EngineState from, EngineState to, const std::optional<ErrorInfo> &)>
|
||||
on_state;
|
||||
|
||||
std::function<void(const AuthChallenge &)> on_auth_required;
|
||||
std::function<void(const DecisionRequest &)> on_decision_needed;
|
||||
|
||||
// Fired exactly once, last. Success carries the outcome; failure carries the mapped
|
||||
// ErrorInfo. After it returns the engine makes no further callbacks for this task and
|
||||
// the handle's control methods become no-ops.
|
||||
std::function<void(Result<DownloadOutcome>)> on_finished;
|
||||
};
|
||||
|
||||
// --- the handle -------------------------------------------------------------------------
|
||||
|
||||
// Copyable (shared state). Every method is safe to call from any thread; each posts to
|
||||
// the engine and returns immediately. Dropping the last handle does NOT cancel the task —
|
||||
// call cancel() for that. Bodies land in stage 8.
|
||||
class DownloadHandle {
|
||||
public:
|
||||
DownloadHandle() = default;
|
||||
|
||||
[[nodiscard]] TaskId id() const noexcept;
|
||||
[[nodiscard]] bool valid() const noexcept { return static_cast<bool>(state_); }
|
||||
|
||||
// Idempotent. pause() on an already-paused or terminal task is a no-op (no error);
|
||||
// likewise resume() on a task that is not paused. The resulting state is observed via
|
||||
// on_state / this->state(), never a return value (ADR 0013 §2).
|
||||
void pause();
|
||||
void resume();
|
||||
// Idempotent, terminal. discard_partial also removes the .veloxpart[.meta] files.
|
||||
void cancel(bool discard_partial = false);
|
||||
|
||||
// Only act while the task is awaiting the matching input (auto-paused for auth /
|
||||
// decision); otherwise a no-op. `remember` asks DAEMON to persist to the Secret
|
||||
// Service — the engine never stores a credential.
|
||||
void provide_auth(std::string username, std::string password, bool remember);
|
||||
void decide(Decision d);
|
||||
|
||||
// IDM's "Refresh Download Address": swap the URL (e.g. a fresh signed URL) on a live
|
||||
// or paused task without losing progress. Empty `headers` keeps the current ones.
|
||||
void refresh_url(std::string url, std::vector<net::HeaderField> headers = {});
|
||||
|
||||
// Synchronous snapshots — cheap, lock-guarded, safe any time.
|
||||
[[nodiscard]] EngineState state() const;
|
||||
[[nodiscard]] Progress progress() const;
|
||||
|
||||
private:
|
||||
friend class vdm::Engine;
|
||||
struct State;
|
||||
explicit DownloadHandle(std::shared_ptr<State> s) : state_(std::move(s)) {}
|
||||
std::shared_ptr<State> state_;
|
||||
};
|
||||
|
||||
} // namespace vdm::task
|
||||
|
||||
#endif // VDM_TASK_DOWNLOAD_HPP
|
||||
Reference in New Issue
Block a user