vdm/core/docs/engine-api-m1.md

# CORE → DAEMON — the engine API for review (M1)

**Status: for review.** This is the `libveloxcore` public surface `veloxd` links and calls
to actually run a download. It is the thing the AGENT-CORE brief asked for on day one and
that slipped: DAEMON has an RPC surface and a store and, until this is agreed, nothing to
call. Sketch headers: `core/include/vdm/engine.hpp`, `core/include/vdm/task/download.hpp`
(both compile now; `Engine` / `DownloadHandle` bodies land in CORE stage 8). Nothing here
touches `contracts/` — DAEMON projects these callbacks onto `TaskSummary` / `TaskDetail` /
`event.*`.

Please review the field semantics, the threading/lifetime rules, and the
pause/resume/cancel contract, and raise anything that would force a `daemon/src/sched/` or
RPC-dispatch rewrite later. Open questions are at the bottom.

---

## 1. `DownloadSpec` — what DAEMON hands the engine

DAEMON has already run the rules engine, picked the category folder, canonicalised the
path and checked it against the allowed roots (`-32011` is DAEMON's error, raised before
`start()`), and resolved the filename. The engine's spec is the concrete result.

| field | who fills it | notes |
|---|---|---|
| `url`, `mirrors` | DAEMON | `mirrors` are alternative URLs for the *same bytes*; the segmenter's requeue prefers a different host after 3 connection failures (docs/04 §3). |
| `headers`, `cookies`, `referrer`, `user_agent` | DAEMON, verbatim from the capture | replayed on every segment request and on the probe. |
| `save_path` | DAEMON | **absolute and final.** The engine never canonicalises or root-checks. `<save_path>.veloxpart` and `.veloxpart.meta` sit beside it; on success the part file is renamed in place. |
| `segments` | user / `connection.maxSegmentsPerDownload` | requested upper bound 1–32; the engine lowers it to the per-host cap and to 1 for a non-resumable source. Effective value comes back in `Progress.effective_segments`. |
| `buffer_bytes` | user / `connection.bufferBytes` | requested per-segment; silently reduced to fit `maxTotalBufferBytes` across all live segments. Effective value in `Progress.effective_buffer_bytes` → your `TaskDetail.effectiveBufferBytes`. |
| `proxy`, `auth` | DAEMON | `auth` carries credentials only if known up front (Secret Service). Leave `scheme == none` to get an `on_auth_required` on a 401/407 instead. |
| `checksum` | user (`download.add.checksum`) | `{algo, hex}`. Verified during `verifying`; a mismatch is a terminal `failed` with `Error::checksum_mismatch`. |
| `probe_hint` | DAEMON | the `ProbeResult` you already got for the File Info dialog. Supplying it skips the engine's own probe — the task starts in `connecting`, not `probing`. The engine still revalidates with `If-Range` on resume. |
| `allow_resume` | DAEMON | `true`: if a CRC-valid `.veloxpart.meta` sits beside `save_path`, resume from it. `false`: start fresh, overwrite. Your restart flow (ADR 0013 §5) sets this per task. |
| `max_retries` | user / default 10 | per segment, before `failed` with `Error::max_retries_exhausted`. |

`start()` returns immediately. It never throws and never blocks on the network; a bad URL,
DNS failure, or unwritable `save_path` is delivered through `on_finished`.

## 2. The state machine the engine drives

`EngineState` is the CORE-owned subset of the wire `TaskState` (ADR 0013 §1):

```
probing ─▶ connecting ⇄ downloading ─▶ assembling ─▶ verifying ─▶ complete
   │            │  ▲          │  │            (M4 mux; a no-op rename in M1)
   │            │  └─ retry_wait ┘
   ▼            ▼
 (any) ──────▶ paused ──(resume)──▶ connecting
 (any CORE state) ──────────────────▶ failed        (terminal, engine-initiated)
 (any state, on cancel()) ──────────▶ cancelled     (terminal, DAEMON-initiated)
```

`new` and `queued` are yours; the engine never emits them. `start()` corresponds to your
`queued → probing`. Every transition is reported through `on_state(from, to, error?)`,
including the auto-pauses (§4) and the terminals. `previousState` on your
`event.task.state` maps straight from the `from` argument.

## 3. Threading and lifetime

- **Callbacks run on an engine thread** — a transfer worker, the progress timer, or a
  dispatch thread — **never** the thread that called `start()` / `pause()` / etc.
- **Per task, callbacks are serialised.** You will never get two callbacks for the same
  handle at once. Across tasks they run concurrently.
- **A callback must not block.** It runs on a thread doing real transfer work; a slow
  callback stalls that work. Hand off to your own queue/loop.
- **A callback must not re-enter the same handle synchronously** — no `pause()` from
  inside `on_state`, etc. Post it. (Calling into a *different* handle, or into
  `segment_budget()`, is fine.)
- **`on_finished` is always the last callback** for a task. After it returns the engine
  makes no further callbacks for that handle and the handle's control methods are no-ops.
- **The handle is copyable and thread-safe.** Dropping the last copy does **not** cancel —
  the task runs on. Call `cancel()` to stop it. (DAEMON holds the handle for the task's
  life anyway.)
- **`Engine` must outlive every handle.** `~Engine()` cancels all running tasks and joins
  their workers before returning — expect it to block briefly.
- **Logging**: the engine writes through `vdm::set_log_sink()` (a `core/util` global).
  Install your sink once at startup; the engine never opens a file itself.

## 4. `paused` is shared, and idempotency is the contract (ADR 0013 §2)

Both sides put a task in `paused`, for disjoint reasons:

- **DAEMON-initiated**: `handle.pause()` — user pause, a schedule window closing, a queue
  stop, `Queue.onComplete`, the admission governor reconciling a lowered
  `maxActiveSegments`.
- **Engine auto-pause**: `on_auth_required` (401/407), `on_decision_needed`
  (`server_file_changed` / stale range), disk full. The engine transitions to `paused`
  on its own and fires `on_state(_, paused, ErrorInfo{...})` — the same path as any other
  transition. `ErrorInfo.code` present ⇒ engine-initiated; absent ⇒ you did it. That is
  the only discriminator, and it is what your `error`-on-`paused` widening (open item 3 of
  ADR 0013) carries on the wire.

**Idempotency, now a signature:**

| call | already in that state / terminal | otherwise |
|---|---|---|
| `pause()` | no-op, no error | stop new segment requests, flush + `fdatasync` in-flight buffers, write `.veloxpart.meta`, release the budget slots, `on_state(_, paused, nullopt)`. Bounded by the slowest in-flight flush. |
| `resume()` | no-op if not `paused`; no-op if terminal | revalidate with `If-Range`, re-acquire budget slots, `paused → connecting`, resume from the sidecar offsets. |
| `cancel(discard_partial)` | no-op if already terminal | stop everything, `on_state(_, cancelled, nullopt)`, `on_finished(Err{cancelled})`. `discard_partial` also unlinks `.veloxpart[.meta]` — wire this to `download.remove {deleteFile}`. |

`resume()` after an auto-pause for `auth_required` **without** a preceding
`provide_auth()` is a no-op — the task stays paused. This is ADR 0013 §3's "resume must
not cross reasons", enforced on CORE's side: the scheduler cannot accidentally un-pause a
task waiting on credentials.

- **`provide_auth(user, pass, remember)`** — acts only while the task is auto-paused for
  auth; supplies the credential for the retry and resumes. `remember` asks *you* to
  persist to the Secret Service; the engine never stores it. No-op otherwise.
- **`decide(Decision)`** — acts only while auto-paused for a decision. `restart` discards
  the partial and re-downloads; `keep_partial` continues against what is on disk (the
  user's stated risk); `abort` → `failed`. No-op otherwise.
- **`refresh_url(url, headers)`** — IDM's "Refresh Download Address": swap the URL on a
  live or paused task without losing progress (a fresh signed URL). Maps to
  `download.refreshUrl`.

## 5. Progress

`on_progress` is coalesced to **≤ 4 Hz per task** inside the engine — the same cadence as
`event.task.progress`, so your batcher can forward without re-throttling. It carries
aggregate `downloaded` / `speed_bps` / `eta_seconds`, the effective segment count and
buffer size, and a `SegmentProgress[]` (index, inclusive `[start,end]`, `completed`,
per-segment speed, state) for the GUI's segment bars. `total` is absent for a chunked
source until the stream ends.

`handle.state()` and `handle.progress()` are synchronous lock-guarded snapshots for
`download.get` / `download.list` — call them any time, including from your RPC thread.

## 6. What the engine does NOT do

- No filename resolution, no category matching, no path canonicalisation, no allowed-root
  check — all DAEMON, before `start()`.
- No queueing, scheduling, priority, or "when queue completes" — DAEMON, via
  `segment_budget().set_task_order()` and by choosing when to call `start()` / `pause()`.
- No persistence beyond `.veloxpart.meta`. On a daemon restart the engine knows nothing;
  you reload from SQLite, rewrite CORE-owned states to `queued`, and re-`start()` with
  `allow_resume = true` (ADR 0013 §5).
- No credential storage. Ever (`CLAUDE.md` §4).

---

## Open questions for DAEMON

1. **`DownloadSpec.probe_hint`** — do you want to pass the File-Info `ProbeResult` in, or
   would you rather the engine always probe (one code path, ~1 extra round trip)? The
   sketch supports both; picking one simplifies stage 8.
2. **`cancel(discard_partial)`** vs a separate `handle.remove()` — the wire has
   `download.cancel` and `download.remove {deleteFile}` as two methods. One call with a
   flag, or two?
3. **`on_decision_needed` granularity** — is `{restart, keep_partial, abort}` the right
   choice set for `server_file_changed`, or do you also need "retry the same range once
   more" as a distinct option for the stale-416 case?
4. **Progress cadence** — 4 Hz per task matches the wire. With 20 active tasks that is 80
   callbacks/s on the engine's timer thread. Acceptable, or do you want a single
   `on_progress_batch(span<Progress>)` so the engine coalesces across tasks too?
5. **`refresh_url` while `downloading`** — should an in-flight segment finish on the old
   URL and only new/retried segments use the new one (less disruption), or should the
   engine restart all segments on the new URL immediately (guaranteed consistency)? The
   signed-URL-expiry case wants the latter; a mirror swap wants the former.