core: post the engine API for DAEMON review (pre-stage-7)
The download entry point the AGENT-CORE brief asked for on day one and that slipped. DAEMON has an RPC surface and a store and, until this is agreed, nothing in velox::core to call. vdm/task/download.hpp — DownloadSpec (the resolved subset DAEMON hands in: absolute save_path, verbatim browser headers, requested segments/buffer, optional probe_hint / checksum / auth, allow_resume), EngineState (the CORE-owned subset of the wire TaskState), Progress / SegmentProgress, DownloadCallbacks (on_progress <=4 Hz, on_state for every transition incl. auto-pauses, on_auth_required, on_decision_needed, on_finished last), DownloadHandle (pause/resume/cancel — idempotent per the ADR 0013 signature — plus provide_auth / decide / refresh_url, and synchronous state()/progress() snapshots). vdm/engine.hpp — Engine: start(spec, callbacks) -> handle, segment_budget() (DAEMON's sched/ admission surface, ADR 0011), live connection.* setters, a standalone probe() on the pool outside the segment budget. core/docs/engine-api-m1.md — the review doc: field semantics, the state machine, threading/lifetime rules (which thread callbacks arrive on, what is legal from inside one, handle/engine lifetime), the shared-`paused` idempotency contract as a signature, and five open questions for DAEMON. Value types compile and are covered by api_compiles_test; Engine / DownloadHandle bodies land in stage 8, built against whatever DAEMON signs off here. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
This commit is contained in:
@@ -0,0 +1,158 @@
|
|||||||
|
# CORE → DAEMON — the engine API for review (M1)
|
||||||
|
|
||||||
|
**Status: for review.** This is the `libveloxcore` public surface `veloxd` links and calls
|
||||||
|
to actually run a download. It is the thing the AGENT-CORE brief asked for on day one and
|
||||||
|
that slipped: DAEMON has an RPC surface and a store and, until this is agreed, nothing to
|
||||||
|
call. Sketch headers: `core/include/vdm/engine.hpp`, `core/include/vdm/task/download.hpp`
|
||||||
|
(both compile now; `Engine` / `DownloadHandle` bodies land in CORE stage 8). Nothing here
|
||||||
|
touches `contracts/` — DAEMON projects these callbacks onto `TaskSummary` / `TaskDetail` /
|
||||||
|
`event.*`.
|
||||||
|
|
||||||
|
Please review the field semantics, the threading/lifetime rules, and the
|
||||||
|
pause/resume/cancel contract, and raise anything that would force a `daemon/src/sched/` or
|
||||||
|
RPC-dispatch rewrite later. Open questions are at the bottom.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. `DownloadSpec` — what DAEMON hands the engine
|
||||||
|
|
||||||
|
DAEMON has already run the rules engine, picked the category folder, canonicalised the
|
||||||
|
path and checked it against the allowed roots (`-32011` is DAEMON's error, raised before
|
||||||
|
`start()`), and resolved the filename. The engine's spec is the concrete result.
|
||||||
|
|
||||||
|
| field | who fills it | notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `url`, `mirrors` | DAEMON | `mirrors` are alternative URLs for the *same bytes*; the segmenter's requeue prefers a different host after 3 connection failures (docs/04 §3). |
|
||||||
|
| `headers`, `cookies`, `referrer`, `user_agent` | DAEMON, verbatim from the capture | replayed on every segment request and on the probe. |
|
||||||
|
| `save_path` | DAEMON | **absolute and final.** The engine never canonicalises or root-checks. `<save_path>.veloxpart` and `.veloxpart.meta` sit beside it; on success the part file is renamed in place. |
|
||||||
|
| `segments` | user / `connection.maxSegmentsPerDownload` | requested upper bound 1–32; the engine lowers it to the per-host cap and to 1 for a non-resumable source. Effective value comes back in `Progress.effective_segments`. |
|
||||||
|
| `buffer_bytes` | user / `connection.bufferBytes` | requested per-segment; silently reduced to fit `maxTotalBufferBytes` across all live segments. Effective value in `Progress.effective_buffer_bytes` → your `TaskDetail.effectiveBufferBytes`. |
|
||||||
|
| `proxy`, `auth` | DAEMON | `auth` carries credentials only if known up front (Secret Service). Leave `scheme == none` to get an `on_auth_required` on a 401/407 instead. |
|
||||||
|
| `checksum` | user (`download.add.checksum`) | `{algo, hex}`. Verified during `verifying`; a mismatch is a terminal `failed` with `Error::checksum_mismatch`. |
|
||||||
|
| `probe_hint` | DAEMON | the `ProbeResult` you already got for the File Info dialog. Supplying it skips the engine's own probe — the task starts in `connecting`, not `probing`. The engine still revalidates with `If-Range` on resume. |
|
||||||
|
| `allow_resume` | DAEMON | `true`: if a CRC-valid `.veloxpart.meta` sits beside `save_path`, resume from it. `false`: start fresh, overwrite. Your restart flow (ADR 0013 §5) sets this per task. |
|
||||||
|
| `max_retries` | user / default 10 | per segment, before `failed` with `Error::max_retries_exhausted`. |
|
||||||
|
|
||||||
|
`start()` returns immediately. It never throws and never blocks on the network; a bad URL,
|
||||||
|
DNS failure, or unwritable `save_path` is delivered through `on_finished`.
|
||||||
|
|
||||||
|
## 2. The state machine the engine drives
|
||||||
|
|
||||||
|
`EngineState` is the CORE-owned subset of the wire `TaskState` (ADR 0013 §1):
|
||||||
|
|
||||||
|
```
|
||||||
|
probing ─▶ connecting ⇄ downloading ─▶ assembling ─▶ verifying ─▶ complete
|
||||||
|
│ │ ▲ │ │ (M4 mux; a no-op rename in M1)
|
||||||
|
│ │ └─ retry_wait ┘
|
||||||
|
▼ ▼
|
||||||
|
(any) ──────▶ paused ──(resume)──▶ connecting
|
||||||
|
(any CORE state) ──────────────────▶ failed (terminal, engine-initiated)
|
||||||
|
(any state, on cancel()) ──────────▶ cancelled (terminal, DAEMON-initiated)
|
||||||
|
```
|
||||||
|
|
||||||
|
`new` and `queued` are yours; the engine never emits them. `start()` corresponds to your
|
||||||
|
`queued → probing`. Every transition is reported through `on_state(from, to, error?)`,
|
||||||
|
including the auto-pauses (§4) and the terminals. `previousState` on your
|
||||||
|
`event.task.state` maps straight from the `from` argument.
|
||||||
|
|
||||||
|
## 3. Threading and lifetime
|
||||||
|
|
||||||
|
- **Callbacks run on an engine thread** — a transfer worker, the progress timer, or a
|
||||||
|
dispatch thread — **never** the thread that called `start()` / `pause()` / etc.
|
||||||
|
- **Per task, callbacks are serialised.** You will never get two callbacks for the same
|
||||||
|
handle at once. Across tasks they run concurrently.
|
||||||
|
- **A callback must not block.** It runs on a thread doing real transfer work; a slow
|
||||||
|
callback stalls that work. Hand off to your own queue/loop.
|
||||||
|
- **A callback must not re-enter the same handle synchronously** — no `pause()` from
|
||||||
|
inside `on_state`, etc. Post it. (Calling into a *different* handle, or into
|
||||||
|
`segment_budget()`, is fine.)
|
||||||
|
- **`on_finished` is always the last callback** for a task. After it returns the engine
|
||||||
|
makes no further callbacks for that handle and the handle's control methods are no-ops.
|
||||||
|
- **The handle is copyable and thread-safe.** Dropping the last copy does **not** cancel —
|
||||||
|
the task runs on. Call `cancel()` to stop it. (DAEMON holds the handle for the task's
|
||||||
|
life anyway.)
|
||||||
|
- **`Engine` must outlive every handle.** `~Engine()` cancels all running tasks and joins
|
||||||
|
their workers before returning — expect it to block briefly.
|
||||||
|
- **Logging**: the engine writes through `vdm::set_log_sink()` (a `core/util` global).
|
||||||
|
Install your sink once at startup; the engine never opens a file itself.
|
||||||
|
|
||||||
|
## 4. `paused` is shared, and idempotency is the contract (ADR 0013 §2)
|
||||||
|
|
||||||
|
Both sides put a task in `paused`, for disjoint reasons:
|
||||||
|
|
||||||
|
- **DAEMON-initiated**: `handle.pause()` — user pause, a schedule window closing, a queue
|
||||||
|
stop, `Queue.onComplete`, the admission governor reconciling a lowered
|
||||||
|
`maxActiveSegments`.
|
||||||
|
- **Engine auto-pause**: `on_auth_required` (401/407), `on_decision_needed`
|
||||||
|
(`server_file_changed` / stale range), disk full. The engine transitions to `paused`
|
||||||
|
on its own and fires `on_state(_, paused, ErrorInfo{...})` — the same path as any other
|
||||||
|
transition. `ErrorInfo.code` present ⇒ engine-initiated; absent ⇒ you did it. That is
|
||||||
|
the only discriminator, and it is what your `error`-on-`paused` widening (open item 3 of
|
||||||
|
ADR 0013) carries on the wire.
|
||||||
|
|
||||||
|
**Idempotency, now a signature:**
|
||||||
|
|
||||||
|
| call | already in that state / terminal | otherwise |
|
||||||
|
|---|---|---|
|
||||||
|
| `pause()` | no-op, no error | stop new segment requests, flush + `fdatasync` in-flight buffers, write `.veloxpart.meta`, release the budget slots, `on_state(_, paused, nullopt)`. Bounded by the slowest in-flight flush. |
|
||||||
|
| `resume()` | no-op if not `paused`; no-op if terminal | revalidate with `If-Range`, re-acquire budget slots, `paused → connecting`, resume from the sidecar offsets. |
|
||||||
|
| `cancel(discard_partial)` | no-op if already terminal | stop everything, `on_state(_, cancelled, nullopt)`, `on_finished(Err{cancelled})`. `discard_partial` also unlinks `.veloxpart[.meta]` — wire this to `download.remove {deleteFile}`. |
|
||||||
|
|
||||||
|
`resume()` after an auto-pause for `auth_required` **without** a preceding
|
||||||
|
`provide_auth()` is a no-op — the task stays paused. This is ADR 0013 §3's "resume must
|
||||||
|
not cross reasons", enforced on CORE's side: the scheduler cannot accidentally un-pause a
|
||||||
|
task waiting on credentials.
|
||||||
|
|
||||||
|
- **`provide_auth(user, pass, remember)`** — acts only while the task is auto-paused for
|
||||||
|
auth; supplies the credential for the retry and resumes. `remember` asks *you* to
|
||||||
|
persist to the Secret Service; the engine never stores it. No-op otherwise.
|
||||||
|
- **`decide(Decision)`** — acts only while auto-paused for a decision. `restart` discards
|
||||||
|
the partial and re-downloads; `keep_partial` continues against what is on disk (the
|
||||||
|
user's stated risk); `abort` → `failed`. No-op otherwise.
|
||||||
|
- **`refresh_url(url, headers)`** — IDM's "Refresh Download Address": swap the URL on a
|
||||||
|
live or paused task without losing progress (a fresh signed URL). Maps to
|
||||||
|
`download.refreshUrl`.
|
||||||
|
|
||||||
|
## 5. Progress
|
||||||
|
|
||||||
|
`on_progress` is coalesced to **≤ 4 Hz per task** inside the engine — the same cadence as
|
||||||
|
`event.task.progress`, so your batcher can forward without re-throttling. It carries
|
||||||
|
aggregate `downloaded` / `speed_bps` / `eta_seconds`, the effective segment count and
|
||||||
|
buffer size, and a `SegmentProgress[]` (index, inclusive `[start,end]`, `completed`,
|
||||||
|
per-segment speed, state) for the GUI's segment bars. `total` is absent for a chunked
|
||||||
|
source until the stream ends.
|
||||||
|
|
||||||
|
`handle.state()` and `handle.progress()` are synchronous lock-guarded snapshots for
|
||||||
|
`download.get` / `download.list` — call them any time, including from your RPC thread.
|
||||||
|
|
||||||
|
## 6. What the engine does NOT do
|
||||||
|
|
||||||
|
- No filename resolution, no category matching, no path canonicalisation, no allowed-root
|
||||||
|
check — all DAEMON, before `start()`.
|
||||||
|
- No queueing, scheduling, priority, or "when queue completes" — DAEMON, via
|
||||||
|
`segment_budget().set_task_order()` and by choosing when to call `start()` / `pause()`.
|
||||||
|
- No persistence beyond `.veloxpart.meta`. On a daemon restart the engine knows nothing;
|
||||||
|
you reload from SQLite, rewrite CORE-owned states to `queued`, and re-`start()` with
|
||||||
|
`allow_resume = true` (ADR 0013 §5).
|
||||||
|
- No credential storage. Ever (`CLAUDE.md` §4).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Open questions for DAEMON
|
||||||
|
|
||||||
|
1. **`DownloadSpec.probe_hint`** — do you want to pass the File-Info `ProbeResult` in, or
|
||||||
|
would you rather the engine always probe (one code path, ~1 extra round trip)? The
|
||||||
|
sketch supports both; picking one simplifies stage 8.
|
||||||
|
2. **`cancel(discard_partial)`** vs a separate `handle.remove()` — the wire has
|
||||||
|
`download.cancel` and `download.remove {deleteFile}` as two methods. One call with a
|
||||||
|
flag, or two?
|
||||||
|
3. **`on_decision_needed` granularity** — is `{restart, keep_partial, abort}` the right
|
||||||
|
choice set for `server_file_changed`, or do you also need "retry the same range once
|
||||||
|
more" as a distinct option for the stale-416 case?
|
||||||
|
4. **Progress cadence** — 4 Hz per task matches the wire. With 20 active tasks that is 80
|
||||||
|
callbacks/s on the engine's timer thread. Acceptable, or do you want a single
|
||||||
|
`on_progress_batch(span<Progress>)` so the engine coalesces across tasks too?
|
||||||
|
5. **`refresh_url` while `downloading`** — should an in-flight segment finish on the old
|
||||||
|
URL and only new/retried segments use the new one (less disruption), or should the
|
||||||
|
engine restart all segments on the new URL immediately (guaranteed consistency)? The
|
||||||
|
signed-URL-expiry case wants the latter; a mirror swap wants the former.
|
||||||
@@ -1,8 +1,12 @@
|
|||||||
# `libveloxcore` — public API
|
# `libveloxcore` — public API
|
||||||
|
|
||||||
**Status: M1 in progress.** Only `util/` is landed. The download-facing API
|
**Status: M1 in progress.** `util/`, `net/` (http_client, probe, url, content_disposition),
|
||||||
(`DownloadSpec`, `DownloadTask`, probe, typed callbacks) arrives with later stages and
|
`io/` (sparse_file, write_buffer), `meta/veloxpart`, and `segment/` (segmenter, budget)
|
||||||
is reviewed by DAEMON before M2 (AGENT-CORE DoD).
|
are landed. The **download entry point** — `vdm::Engine`, `vdm::task::DownloadSpec` /
|
||||||
|
`DownloadHandle` / `DownloadCallbacks` — is sketched in `vdm/engine.hpp` and
|
||||||
|
`vdm/task/download.hpp` and **out for DAEMON review**: see
|
||||||
|
[`core/docs/engine-api-m1.md`](../../docs/engine-api-m1.md). Bodies land in CORE stage 8;
|
||||||
|
build against the value types now.
|
||||||
|
|
||||||
Layering (CLAUDE.md §3): this library knows nothing about JSON, SQL, Qt, or RPC. Input is
|
Layering (CLAUDE.md §3): this library knows nothing about JSON, SQL, Qt, or RPC. Input is
|
||||||
a spec value; output is bytes on disk plus typed callbacks. DAEMON projects engine state
|
a spec value; output is bytes on disk plus typed callbacks. DAEMON projects engine state
|
||||||
|
|||||||
@@ -0,0 +1,75 @@
|
|||||||
|
// vdm/engine.hpp — the download engine's single entry point. REVIEW SKETCH (stage 7
|
||||||
|
// pre-work); bodies land in stage 8. See core/docs/engine-api-m1.md.
|
||||||
|
//
|
||||||
|
// The engine owns the HTTP client, the probe pool, the segment budget, and the disk I/O.
|
||||||
|
// Its input is a DownloadSpec; its output is bytes at save_path plus typed callbacks. No
|
||||||
|
// JSON, no SQL, no Qt, no RPC — DAEMON projects the callbacks onto the wire contract.
|
||||||
|
//
|
||||||
|
// This header compiles standalone.
|
||||||
|
|
||||||
|
#ifndef VDM_ENGINE_HPP
|
||||||
|
#define VDM_ENGINE_HPP
|
||||||
|
|
||||||
|
#include <cstdint>
|
||||||
|
#include <memory>
|
||||||
|
|
||||||
|
#include "vdm/segment/budget.hpp"
|
||||||
|
#include "vdm/task/download.hpp"
|
||||||
|
|
||||||
|
namespace vdm {
|
||||||
|
|
||||||
|
class Engine {
|
||||||
|
public:
|
||||||
|
struct Config {
|
||||||
|
// Defaults used when a DownloadSpec leaves the field unset. Live-adjustable via
|
||||||
|
// the setters below (they take effect on the next segment (re)assignment, not by
|
||||||
|
// resizing an in-flight buffer).
|
||||||
|
std::uint32_t default_segments = 8; // connection.maxSegmentsPerDownload
|
||||||
|
std::uint64_t default_buffer_bytes = 1u << 20; // connection.bufferBytes (1 MiB)
|
||||||
|
std::uint64_t min_segment_bytes = 1u << 20; // never split below this
|
||||||
|
std::uint64_t max_total_buffer_bytes = 128ull << 20; // connection.maxTotalBufferBytes
|
||||||
|
std::uint32_t max_active_segments = 32; // connection.maxActiveSegments
|
||||||
|
std::uint32_t probe_pool_size = 4; // ADR 0011 §5, outside the budget
|
||||||
|
long default_max_retries = 10; // per segment
|
||||||
|
std::uint32_t http_workers = 0; // 0 => hardware-derived (<=4)
|
||||||
|
};
|
||||||
|
|
||||||
|
Engine(); // default Config
|
||||||
|
explicit Engine(Config cfg);
|
||||||
|
~Engine(); // cancels every running task and joins before returning
|
||||||
|
|
||||||
|
Engine(const Engine &) = delete;
|
||||||
|
Engine &operator=(const Engine &) = delete;
|
||||||
|
|
||||||
|
// Start a download. Returns immediately with a handle; the task begins in `probing`
|
||||||
|
// (or `connecting` when spec.probe_hint is supplied). Every failure — bad URL, DNS,
|
||||||
|
// an unwritable save_path — is delivered through callbacks.on_finished, never thrown.
|
||||||
|
[[nodiscard]] task::DownloadHandle start(task::DownloadSpec spec,
|
||||||
|
task::DownloadCallbacks callbacks);
|
||||||
|
|
||||||
|
// The global segment allocator. DAEMON's scheduler drives admission through this
|
||||||
|
// (set_max_active_segments / set_host_segment_cap / set_task_order) and reads
|
||||||
|
// occupancy from it (budget() / segments_active() / starved_tasks() /
|
||||||
|
// on_budget_changed). See ADR 0011.
|
||||||
|
[[nodiscard]] segment::SegmentBudget &segment_budget() noexcept;
|
||||||
|
|
||||||
|
// Live settings (connection.* changes from settings.set). Each affects future work.
|
||||||
|
void set_default_segments(std::uint32_t n);
|
||||||
|
void set_default_buffer_bytes(std::uint64_t bytes);
|
||||||
|
void set_max_total_buffer_bytes(std::uint64_t bytes);
|
||||||
|
void set_probe_pool_size(std::uint32_t n);
|
||||||
|
|
||||||
|
// A standalone probe for the File Info dialog, on the same pool as spec-less probes
|
||||||
|
// (never charged against the segment budget). capture.offer's 750 ms deadline is
|
||||||
|
// DAEMON's to enforce — it should answer `ignore` and probe after, never block on
|
||||||
|
// this.
|
||||||
|
void probe(net::ProbeRequest req, std::function<void(Result<net::ProbeResult>)> done);
|
||||||
|
|
||||||
|
private:
|
||||||
|
struct Impl;
|
||||||
|
std::unique_ptr<Impl> impl_;
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace vdm
|
||||||
|
|
||||||
|
#endif // VDM_ENGINE_HPP
|
||||||
@@ -0,0 +1,213 @@
|
|||||||
|
// vdm/task/download.hpp — the public download API: what DAEMON hands the engine and how
|
||||||
|
// the engine reports back. REVIEW SKETCH (stage 7 pre-work) — value types are final
|
||||||
|
// enough to build against; Engine/DownloadHandle bodies land in stage 8.
|
||||||
|
//
|
||||||
|
// See core/docs/engine-api-m1.md for the threading, lifetime, and pause/resume/cancel
|
||||||
|
// contract that goes with these signatures.
|
||||||
|
//
|
||||||
|
// This header compiles standalone.
|
||||||
|
|
||||||
|
#ifndef VDM_TASK_DOWNLOAD_HPP
|
||||||
|
#define VDM_TASK_DOWNLOAD_HPP
|
||||||
|
|
||||||
|
#include <chrono>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <functional>
|
||||||
|
#include <memory>
|
||||||
|
#include <optional>
|
||||||
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#include "vdm/ids.hpp"
|
||||||
|
#include "vdm/net/http_types.hpp"
|
||||||
|
#include "vdm/net/probe.hpp"
|
||||||
|
#include "vdm/segment/segmenter.hpp"
|
||||||
|
#include "vdm/util/error.hpp"
|
||||||
|
#include "vdm/util/result.hpp"
|
||||||
|
|
||||||
|
namespace vdm {
|
||||||
|
class Engine; // owns and fills DownloadHandle (see vdm/engine.hpp)
|
||||||
|
} // namespace vdm
|
||||||
|
|
||||||
|
namespace vdm::task {
|
||||||
|
|
||||||
|
// --- input ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
struct Checksum {
|
||||||
|
enum class Algo { md5, sha1, sha256 };
|
||||||
|
Algo algo = Algo::sha256;
|
||||||
|
std::string hex; // lower-case, no separators
|
||||||
|
};
|
||||||
|
|
||||||
|
// Everything the engine needs to run ONE download. DAEMON has already run the rules
|
||||||
|
// engine, canonicalised the path and checked it against the allowed roots, and resolved
|
||||||
|
// the filename — `save_path` is absolute and final. `<save_path>.veloxpart` and
|
||||||
|
// `<save_path>.veloxpart.meta` live beside it during the transfer.
|
||||||
|
struct DownloadSpec {
|
||||||
|
std::string url;
|
||||||
|
std::vector<std::string> mirrors; // alternative URLs for the same bytes
|
||||||
|
|
||||||
|
std::vector<net::HeaderField> headers; // the browser's, verbatim
|
||||||
|
std::vector<net::Cookie> cookies;
|
||||||
|
std::string referrer;
|
||||||
|
std::string user_agent;
|
||||||
|
|
||||||
|
std::string save_path; // absolute; the engine never canonicalises or root-checks
|
||||||
|
|
||||||
|
std::optional<std::uint32_t> segments; // requested 1..32; nullopt => engine default
|
||||||
|
std::optional<std::uint64_t> buffer_bytes; // requested per segment; nullopt => default
|
||||||
|
|
||||||
|
net::ProxyConfig proxy;
|
||||||
|
net::AuthConfig auth; // credentials known up front (e.g. from the Secret Service);
|
||||||
|
// leave scheme == none to be prompted on a 401/407
|
||||||
|
|
||||||
|
std::optional<Checksum> checksum; // verified during `verifying`; mismatch => failed
|
||||||
|
|
||||||
|
// DAEMON usually probed already for the File Info dialog. Pass it to skip a second
|
||||||
|
// probe; the engine still revalidates on resume. nullopt => the engine probes.
|
||||||
|
std::optional<net::ProbeResult> probe_hint;
|
||||||
|
|
||||||
|
bool allow_resume = true; // if a valid .veloxpart.meta sits beside save_path, resume
|
||||||
|
// from it; false starts fresh and overwrites
|
||||||
|
std::optional<long> max_retries; // per-segment; nullopt => engine default (10)
|
||||||
|
};
|
||||||
|
|
||||||
|
// --- lifecycle (the CORE-owned subset of the wire TaskState; ADR 0013 §1) -------------
|
||||||
|
|
||||||
|
enum class EngineState {
|
||||||
|
probing,
|
||||||
|
connecting,
|
||||||
|
downloading,
|
||||||
|
paused, // shared with DAEMON; entered by either side, idempotently
|
||||||
|
retry_wait, // the engine's own backoff timer
|
||||||
|
assembling, // no-op rename in M1; a real mux step for HLS/DASH (M4)
|
||||||
|
verifying, // checksum
|
||||||
|
complete, // terminal
|
||||||
|
failed, // terminal
|
||||||
|
cancelled, // terminal; always DAEMON- or user-initiated
|
||||||
|
};
|
||||||
|
|
||||||
|
[[nodiscard]] constexpr bool is_terminal(EngineState s) noexcept {
|
||||||
|
return s == EngineState::complete || s == EngineState::failed || s == EngineState::cancelled;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- progress ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
struct SegmentProgress {
|
||||||
|
std::uint32_t index = 0;
|
||||||
|
std::uint64_t start = 0;
|
||||||
|
std::uint64_t end = 0; // inclusive
|
||||||
|
std::uint64_t completed = 0;
|
||||||
|
std::uint64_t speed_bps = 0;
|
||||||
|
segment::SegState state = segment::SegState::idle;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct Progress {
|
||||||
|
std::uint64_t downloaded = 0;
|
||||||
|
std::optional<std::uint64_t> total; // absent for a chunked source until it ends
|
||||||
|
std::uint64_t speed_bps = 0; // aggregate over the last window
|
||||||
|
std::optional<std::uint32_t> eta_seconds;
|
||||||
|
|
||||||
|
std::uint32_t effective_segments = 0; // slots the budget granted (held)
|
||||||
|
std::uint64_t effective_buffer_bytes = 0; // per segment, after the maxTotal clamp
|
||||||
|
std::vector<SegmentProgress> segments;
|
||||||
|
};
|
||||||
|
|
||||||
|
// --- interaction callbacks ----------------------------------------------------------
|
||||||
|
|
||||||
|
// A 401/407. The task has already auto-paused (state -> paused, error == auth_required).
|
||||||
|
// DAEMON collects credentials and calls handle.provide_auth().
|
||||||
|
struct AuthChallenge {
|
||||||
|
std::string host;
|
||||||
|
std::string realm;
|
||||||
|
enum class Scheme { basic, digest, ntlm, negotiate, unknown };
|
||||||
|
Scheme scheme = Scheme::unknown;
|
||||||
|
};
|
||||||
|
|
||||||
|
// The server's copy changed under us (a 200 where a 206 was expected, or an If-Range /
|
||||||
|
// ETag mismatch on resume — docs/04 §5), or the range metadata went stale (416). The
|
||||||
|
// task has auto-paused. DAEMON asks the user and calls handle.decide().
|
||||||
|
struct DecisionRequest {
|
||||||
|
enum class Kind { server_file_changed, range_metadata_stale };
|
||||||
|
Kind kind = Kind::server_file_changed;
|
||||||
|
std::string detail; // human-readable, for the dialog body
|
||||||
|
};
|
||||||
|
|
||||||
|
enum class Decision {
|
||||||
|
restart, // discard the partial file, download again from scratch
|
||||||
|
keep_partial, // trust what is on disk and continue (the user's risk)
|
||||||
|
abort, // give up: the task goes to `failed`
|
||||||
|
};
|
||||||
|
|
||||||
|
struct DownloadOutcome {
|
||||||
|
std::string final_path;
|
||||||
|
std::uint64_t bytes = 0;
|
||||||
|
std::optional<std::string> sha256_hex; // present when a checksum was requested/derived
|
||||||
|
std::chrono::milliseconds elapsed{0};
|
||||||
|
};
|
||||||
|
|
||||||
|
// All callbacks are optional. See core/docs/engine-api-m1.md for the rules; in short:
|
||||||
|
// they arrive on an engine thread, are serialised per task, must not block, and must not
|
||||||
|
// re-enter THIS task's handle synchronously.
|
||||||
|
struct DownloadCallbacks {
|
||||||
|
// Coalesced to <= 4 Hz per task (matches the wire event.task.progress cadence).
|
||||||
|
std::function<void(const Progress &)> on_progress;
|
||||||
|
|
||||||
|
// Every lifecycle transition, including the auto-pauses above (to == paused with a
|
||||||
|
// populated ErrorInfo) and terminals.
|
||||||
|
std::function<void(EngineState from, EngineState to, const std::optional<ErrorInfo> &)>
|
||||||
|
on_state;
|
||||||
|
|
||||||
|
std::function<void(const AuthChallenge &)> on_auth_required;
|
||||||
|
std::function<void(const DecisionRequest &)> on_decision_needed;
|
||||||
|
|
||||||
|
// Fired exactly once, last. Success carries the outcome; failure carries the mapped
|
||||||
|
// ErrorInfo. After it returns the engine makes no further callbacks for this task and
|
||||||
|
// the handle's control methods become no-ops.
|
||||||
|
std::function<void(Result<DownloadOutcome>)> on_finished;
|
||||||
|
};
|
||||||
|
|
||||||
|
// --- the handle -------------------------------------------------------------------------
|
||||||
|
|
||||||
|
// Copyable (shared state). Every method is safe to call from any thread; each posts to
|
||||||
|
// the engine and returns immediately. Dropping the last handle does NOT cancel the task —
|
||||||
|
// call cancel() for that. Bodies land in stage 8.
|
||||||
|
class DownloadHandle {
|
||||||
|
public:
|
||||||
|
DownloadHandle() = default;
|
||||||
|
|
||||||
|
[[nodiscard]] TaskId id() const noexcept;
|
||||||
|
[[nodiscard]] bool valid() const noexcept { return static_cast<bool>(state_); }
|
||||||
|
|
||||||
|
// Idempotent. pause() on an already-paused or terminal task is a no-op (no error);
|
||||||
|
// likewise resume() on a task that is not paused. The resulting state is observed via
|
||||||
|
// on_state / this->state(), never a return value (ADR 0013 §2).
|
||||||
|
void pause();
|
||||||
|
void resume();
|
||||||
|
// Idempotent, terminal. discard_partial also removes the .veloxpart[.meta] files.
|
||||||
|
void cancel(bool discard_partial = false);
|
||||||
|
|
||||||
|
// Only act while the task is awaiting the matching input (auto-paused for auth /
|
||||||
|
// decision); otherwise a no-op. `remember` asks DAEMON to persist to the Secret
|
||||||
|
// Service — the engine never stores a credential.
|
||||||
|
void provide_auth(std::string username, std::string password, bool remember);
|
||||||
|
void decide(Decision d);
|
||||||
|
|
||||||
|
// IDM's "Refresh Download Address": swap the URL (e.g. a fresh signed URL) on a live
|
||||||
|
// or paused task without losing progress. Empty `headers` keeps the current ones.
|
||||||
|
void refresh_url(std::string url, std::vector<net::HeaderField> headers = {});
|
||||||
|
|
||||||
|
// Synchronous snapshots — cheap, lock-guarded, safe any time.
|
||||||
|
[[nodiscard]] EngineState state() const;
|
||||||
|
[[nodiscard]] Progress progress() const;
|
||||||
|
|
||||||
|
private:
|
||||||
|
friend class vdm::Engine;
|
||||||
|
struct State;
|
||||||
|
explicit DownloadHandle(std::shared_ptr<State> s) : state_(std::move(s)) {}
|
||||||
|
std::shared_ptr<State> state_;
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace vdm::task
|
||||||
|
|
||||||
|
#endif // VDM_TASK_DOWNLOAD_HPP
|
||||||
@@ -30,6 +30,7 @@ vdm_add_test(veloxcore_write_buffer_test io/write_buffer_test.cpp)
|
|||||||
vdm_add_test(veloxcore_veloxpart_test meta/veloxpart_test.cpp)
|
vdm_add_test(veloxcore_veloxpart_test meta/veloxpart_test.cpp)
|
||||||
vdm_add_test(veloxcore_segmenter_test segment/segmenter_test.cpp)
|
vdm_add_test(veloxcore_segmenter_test segment/segmenter_test.cpp)
|
||||||
vdm_add_test(veloxcore_budget_test segment/budget_test.cpp)
|
vdm_add_test(veloxcore_budget_test segment/budget_test.cpp)
|
||||||
|
vdm_add_test(veloxcore_engine_api_test task/api_compiles_test.cpp)
|
||||||
|
|
||||||
set(_testserver ${CMAKE_SOURCE_DIR}/tools/testserver/testserver.py)
|
set(_testserver ${CMAKE_SOURCE_DIR}/tools/testserver/testserver.py)
|
||||||
foreach(net_it http_client probe)
|
foreach(net_it http_client probe)
|
||||||
|
|||||||
@@ -0,0 +1,84 @@
|
|||||||
|
// The engine API sketch must compile and its value types must behave. Engine /
|
||||||
|
// DownloadHandle bodies land in stage 8; this only exercises the data shapes DAEMON
|
||||||
|
// builds against.
|
||||||
|
|
||||||
|
#include "vdm/engine.hpp"
|
||||||
|
#include "vdm/task/download.hpp"
|
||||||
|
|
||||||
|
#include <type_traits>
|
||||||
|
|
||||||
|
#include "vtest.hpp"
|
||||||
|
|
||||||
|
using namespace vdm;
|
||||||
|
using namespace vdm::task;
|
||||||
|
|
||||||
|
VT_TEST(api_download_spec_defaults) {
|
||||||
|
DownloadSpec s;
|
||||||
|
s.url = "https://example.com/big.iso";
|
||||||
|
s.save_path = "/home/u/Downloads/big.iso";
|
||||||
|
VT_CHECK(s.mirrors.empty());
|
||||||
|
VT_CHECK(!s.segments.has_value());
|
||||||
|
VT_CHECK(!s.buffer_bytes.has_value());
|
||||||
|
VT_CHECK(!s.checksum.has_value());
|
||||||
|
VT_CHECK(!s.probe_hint.has_value());
|
||||||
|
VT_CHECK(s.allow_resume);
|
||||||
|
VT_CHECK(s.proxy.kind == net::ProxyKind::none);
|
||||||
|
VT_CHECK(s.auth.scheme == net::AuthScheme::none);
|
||||||
|
}
|
||||||
|
|
||||||
|
VT_TEST(api_state_helpers) {
|
||||||
|
VT_CHECK(is_terminal(EngineState::complete));
|
||||||
|
VT_CHECK(is_terminal(EngineState::failed));
|
||||||
|
VT_CHECK(is_terminal(EngineState::cancelled));
|
||||||
|
VT_CHECK(!is_terminal(EngineState::paused));
|
||||||
|
VT_CHECK(!is_terminal(EngineState::downloading));
|
||||||
|
}
|
||||||
|
|
||||||
|
VT_TEST(api_callbacks_are_all_optional) {
|
||||||
|
DownloadCallbacks cb; // every std::function default-constructs empty
|
||||||
|
VT_CHECK(!cb.on_progress);
|
||||||
|
VT_CHECK(!cb.on_state);
|
||||||
|
VT_CHECK(!cb.on_auth_required);
|
||||||
|
VT_CHECK(!cb.on_decision_needed);
|
||||||
|
VT_CHECK(!cb.on_finished);
|
||||||
|
|
||||||
|
cb.on_state = [](EngineState, EngineState, const std::optional<vdm::ErrorInfo> &) {};
|
||||||
|
cb.on_finished = [](Result<DownloadOutcome>) {};
|
||||||
|
VT_CHECK(cb.on_state && cb.on_finished);
|
||||||
|
}
|
||||||
|
|
||||||
|
VT_TEST(api_value_types_roundtrip) {
|
||||||
|
Progress p;
|
||||||
|
p.downloaded = 1234;
|
||||||
|
p.total = 5000;
|
||||||
|
p.effective_segments = 4;
|
||||||
|
SegmentProgress sp;
|
||||||
|
sp.index = 0;
|
||||||
|
sp.end = 1249;
|
||||||
|
sp.completed = 1234;
|
||||||
|
p.segments.push_back(sp);
|
||||||
|
VT_CHECK_EQ(p.segments.size(), 1u);
|
||||||
|
VT_CHECK_EQ(p.segments[0].end, 1249u);
|
||||||
|
|
||||||
|
DownloadOutcome o;
|
||||||
|
o.final_path = "/x";
|
||||||
|
o.bytes = 5000;
|
||||||
|
VT_CHECK_EQ(o.bytes, 5000u);
|
||||||
|
|
||||||
|
AuthChallenge a;
|
||||||
|
a.host = "h";
|
||||||
|
a.scheme = AuthChallenge::Scheme::digest;
|
||||||
|
VT_CHECK(a.scheme == AuthChallenge::Scheme::digest);
|
||||||
|
|
||||||
|
DecisionRequest d;
|
||||||
|
d.kind = DecisionRequest::Kind::server_file_changed;
|
||||||
|
d.detail = "changed";
|
||||||
|
VT_CHECK(d.kind == DecisionRequest::Kind::server_file_changed);
|
||||||
|
}
|
||||||
|
|
||||||
|
VT_TEST(api_handle_and_engine_are_move_only_shaped) {
|
||||||
|
static_assert(!std::is_copy_constructible_v<Engine>, "Engine is non-copyable");
|
||||||
|
static_assert(std::is_copy_constructible_v<DownloadHandle>, "handle is a shared handle");
|
||||||
|
DownloadHandle h; // default handle is invalid until Engine::start() fills it
|
||||||
|
VT_CHECK(!h.valid());
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user