Files
vdm/core/include/vdm/task/download.hpp
T
samiandClaude Sonnet 5 91636f8a4d core: add the download engine — task machine, DownloadHandle, Engine
Stage 8 of the CORE build order: the bodies behind the DownloadSpec /
callback API reviewed in core/docs/engine-api-m1.md. Wires probe -> segment
workers -> WriteBuffer -> SparseFile -> .veloxpart.meta -> retry/backoff ->
SegmentBudget -> RateLimiter -> callbacks into one event-driven machine.

- Engine (src/engine.cpp): owns HttpClient, Prober, SegmentBudget,
  RateLimiter and one timer jthread (min-heap of scheduled fns). start()
  builds a task and returns a DownloadHandle; ~Impl quiesces every task
  before joining the timer so no callback fires during teardown.

- DownloadTaskState (src/task/download_task.cpp): one `mu` task lock; a
  shared_mutex over the worker map for the curl write path; callbacks
  collected under `mu` and fired after release via a separate deferred
  queue; weak_from_this() in every async hop. State machine over the
  CORE-owned EngineState subset, auto-pause on 401/407 and on a 200 where
  206 was expected, validated resume via If-Range.

- digest (src/task/digest.cpp): OpenSSL EVP hash_file() for the optional
  post-download checksum; links OpenSSL::Crypto PRIVATE.

- Segmenter::release_segment(): hand a paused segment back to the pool
  unassigned so resume's assign_slot() picks it up instead of splitting a
  still-"assigned" range and orphaning its front half.

- DownloadHandle now names the real control block (vdm::task::
  DownloadTaskState, defined only in the engine TU) via a namespace-scope
  fwd decl and a public-but-effectively-engine-only ctor, replacing the
  nested State/friend pair. Every public signature is unchanged; DAEMON
  (vdm-79) confirmed sched/ names only the public API.

Fixes found while building the end-to-end suite (tests/task/engine_test.cpp,
9 cases against tools/testserver, green under ASan/UBSan and TSan):
- a dropped connection lost its unflushed WriteBuffer tail while advance()
  had already counted those bytes as done -> a retry resumed past an
  unwritten hole. Flush on the failure path.
- when the byte counters hit total while other workers were still live,
  teardown dropped their buffered tails. Now: cancel them and let each
  worker's own seg_finished drain it (the `assembling` state), last one
  starts verification -- no cross-thread buffer access.
- seg_head() let a 401 with credentials present abort before libcurl's
  resend; now it proceeds once and acts on the final status.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
2026-09-10 20:19:31 +04:00

224 lines
9.3 KiB
C++

// vdm/task/download.hpp — the public download API: what DAEMON hands the engine and how
// the engine reports back. REVIEW SKETCH (stage 7 pre-work) — value types are final
// enough to build against; Engine/DownloadHandle bodies land in stage 8.
//
// See core/docs/engine-api-m1.md for the threading, lifetime, and pause/resume/cancel
// contract that goes with these signatures.
//
// This header compiles standalone.
#ifndef VDM_TASK_DOWNLOAD_HPP
#define VDM_TASK_DOWNLOAD_HPP
#include <chrono>
#include <cstdint>
#include <functional>
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "vdm/ids.hpp"
#include "vdm/net/http_types.hpp"
#include "vdm/net/probe.hpp"
#include "vdm/segment/segmenter.hpp"
#include "vdm/util/error.hpp"
#include "vdm/util/result.hpp"
namespace vdm {
class Engine; // owns and fills DownloadHandle (see vdm/engine.hpp)
} // namespace vdm
namespace vdm::task {
// The task control block. Opaque: defined only in the engine's translation unit. A handle
// holds a shared_ptr to one; the engine keeps its own copy so the task outlives a caller
// that drops its handle.
struct DownloadTaskState;
// --- input ---------------------------------------------------------------------------
struct Checksum {
enum class Algo { md5, sha1, sha256, sha512 }; // matches the wire Checksum set
Algo algo = Algo::sha256;
std::string hex; // lower-case, no separators
};
// Everything the engine needs to run ONE download. DAEMON has already run the rules
// engine, canonicalised the path, checked it against the allowed roots, resolved the
// filename, and created the parent directory — `save_path` is absolute and final and its
// directory exists. `<save_path>.veloxpart` and `<save_path>.veloxpart.meta` live beside
// it during the transfer; on success the part file is renamed in place. If the directory
// is missing at open time the task fails with Error::path_rejected.
struct DownloadSpec {
std::string url;
std::vector<std::string> mirrors; // alternative URLs for the same bytes
std::vector<net::HeaderField> headers; // the browser's, verbatim
std::vector<net::Cookie> cookies;
std::string referrer;
std::string user_agent;
std::string save_path; // absolute; the engine never canonicalises or root-checks
std::optional<std::uint32_t> segments; // requested 1..32; nullopt => engine default
std::optional<std::uint64_t> buffer_bytes; // requested per segment; nullopt => default
net::ProxyConfig proxy;
net::AuthConfig auth; // credentials known up front (e.g. from the Secret Service);
// leave scheme == none to be prompted on a 401/407
std::optional<Checksum> checksum; // verified during `verifying`; mismatch => failed
// DAEMON usually probed already for the File Info dialog. Pass it to skip a second
// probe; the engine still revalidates on resume. nullopt => the engine probes.
std::optional<net::ProbeResult> probe_hint;
bool allow_resume = true; // if a valid .veloxpart.meta sits beside save_path, resume
// from it; false starts fresh and overwrites
std::optional<long> max_retries; // per-segment; nullopt => engine default (10)
};
// --- lifecycle (the CORE-owned subset of the wire TaskState; ADR 0013 §1) -------------
enum class EngineState {
probing,
connecting,
downloading,
paused, // shared with DAEMON; entered by either side, idempotently
retry_wait, // the engine's own backoff timer
assembling, // no-op rename in M1; a real mux step for HLS/DASH (M4)
verifying, // checksum
complete, // terminal
failed, // terminal
cancelled, // terminal; always DAEMON- or user-initiated
};
[[nodiscard]] constexpr bool is_terminal(EngineState s) noexcept {
return s == EngineState::complete || s == EngineState::failed || s == EngineState::cancelled;
}
// --- progress ---------------------------------------------------------------------------
struct SegmentProgress {
std::uint32_t index = 0;
std::uint64_t start = 0;
std::uint64_t end = 0; // inclusive
std::uint64_t completed = 0;
std::uint64_t speed_bps = 0;
segment::SegState state = segment::SegState::idle;
};
struct Progress {
std::uint64_t downloaded = 0;
std::optional<std::uint64_t> total; // absent for a chunked source until it ends
std::uint64_t speed_bps = 0; // aggregate over the last window
std::optional<std::uint32_t> eta_seconds;
std::uint32_t effective_segments = 0; // slots the budget granted (held)
std::uint64_t effective_buffer_bytes = 0; // per segment, after the maxTotal clamp
std::vector<SegmentProgress> segments;
};
// --- interaction callbacks ----------------------------------------------------------
// A 401/407. The task has already auto-paused (state -> paused, error == auth_required).
// DAEMON collects credentials and calls handle.provide_auth().
struct AuthChallenge {
std::string host;
std::string realm;
enum class Scheme { basic, digest, ntlm, negotiate, unknown };
Scheme scheme = Scheme::unknown;
};
// The server's copy changed under us (a 200 where a 206 was expected, or an If-Range /
// ETag mismatch on resume — docs/04 §5), or the range metadata went stale (416). The
// task has auto-paused. DAEMON asks the user and calls handle.decide().
struct DecisionRequest {
enum class Kind { server_file_changed, range_metadata_stale };
Kind kind = Kind::server_file_changed;
std::string detail; // human-readable, for the dialog body
};
enum class Decision {
restart, // discard the partial file, download again from scratch
keep_partial, // trust what is on disk and continue (the user's risk)
abort, // give up: the task goes to `failed`
};
struct DownloadOutcome {
std::string final_path;
std::uint64_t bytes = 0;
std::optional<std::string> sha256_hex; // present when a checksum was requested/derived
std::chrono::milliseconds elapsed{0};
};
// All callbacks are optional. See core/docs/engine-api-m1.md for the rules; in short:
// they arrive on an engine thread, are serialised per task, must not block, and must not
// re-enter THIS task's handle synchronously.
struct DownloadCallbacks {
// Coalesced to <= 4 Hz per task (matches the wire event.task.progress cadence).
std::function<void(const Progress &)> on_progress;
// Every lifecycle transition, including the auto-pauses above (to == paused with a
// populated ErrorInfo) and terminals.
std::function<void(EngineState from, EngineState to, const std::optional<ErrorInfo> &)>
on_state;
std::function<void(const AuthChallenge &)> on_auth_required;
std::function<void(const DecisionRequest &)> on_decision_needed;
// Fired exactly once, last. Success carries the outcome; failure carries the mapped
// ErrorInfo. After it returns the engine makes no further callbacks for this task and
// the handle's control methods become no-ops.
std::function<void(Result<DownloadOutcome>)> on_finished;
};
// --- the handle -------------------------------------------------------------------------
// Copyable (shared state). Every method is safe to call from any thread; each posts to
// the engine and returns immediately. Dropping the last handle does NOT cancel the task —
// call cancel() for that. Bodies land in stage 8.
class DownloadHandle {
public:
DownloadHandle() = default;
// The engine builds handles; `DownloadTaskState` is incomplete everywhere else, so
// this is effectively engine-only without a friend declaration.
explicit DownloadHandle(std::shared_ptr<DownloadTaskState> s) : state_(std::move(s)) {}
[[nodiscard]] TaskId id() const noexcept;
[[nodiscard]] bool valid() const noexcept { return static_cast<bool>(state_); }
// Idempotent. pause() on an already-paused or terminal task is a no-op (no error);
// likewise resume() on a task that is not paused. The resulting state is observed via
// on_state / this->state(), never a return value (ADR 0013 §2).
void pause();
void resume();
// Idempotent, terminal. discard_partial also removes the .veloxpart[.meta] files.
// Always fires on_state(_, cancelled, nullopt) then on_finished(Err{Error::canceled}),
// in that order. `download.cancel` == cancel(false); `download.remove` == cancel(true)
// (plus DAEMON's own row/file cleanup).
void cancel(bool discard_partial = false);
// Only act while the task is awaiting the matching input (auto-paused for auth /
// decision); otherwise a no-op. `remember` asks DAEMON to persist to the Secret
// Service — the engine never stores a credential.
void provide_auth(std::string username, std::string password, bool remember);
void decide(Decision d);
// IDM's "Refresh Download Address": swap the URL (e.g. a fresh signed URL) on a live
// or paused task without losing progress. Empty `headers` keeps the current ones.
void refresh_url(std::string url, std::vector<net::HeaderField> headers = {});
// Synchronous snapshots — cheap, lock-guarded, safe any time.
[[nodiscard]] EngineState state() const;
[[nodiscard]] Progress progress() const;
private:
std::shared_ptr<DownloadTaskState> state_;
};
} // namespace vdm::task
#endif // VDM_TASK_DOWNLOAD_HPP