2 Commits

Include renames

Author	SHA1	Message	Date
samiandClaude Sonnet 5	91636f8a4d	core: add the download engine — task machine, DownloadHandle, Engine ... Stage 8 of the CORE build order: the bodies behind the DownloadSpec / callback API reviewed in core/docs/engine-api-m1.md. Wires probe -> segment workers -> WriteBuffer -> SparseFile -> .veloxpart.meta -> retry/backoff -> SegmentBudget -> RateLimiter -> callbacks into one event-driven machine. - Engine (src/engine.cpp): owns HttpClient, Prober, SegmentBudget, RateLimiter and one timer jthread (min-heap of scheduled fns). start() builds a task and returns a DownloadHandle; ~Impl quiesces every task before joining the timer so no callback fires during teardown. - DownloadTaskState (src/task/download_task.cpp): one `mu` task lock; a shared_mutex over the worker map for the curl write path; callbacks collected under `mu` and fired after release via a separate deferred queue; weak_from_this() in every async hop. State machine over the CORE-owned EngineState subset, auto-pause on 401/407 and on a 200 where 206 was expected, validated resume via If-Range. - digest (src/task/digest.cpp): OpenSSL EVP hash_file() for the optional post-download checksum; links OpenSSL::Crypto PRIVATE. - Segmenter::release_segment(): hand a paused segment back to the pool unassigned so resume's assign_slot() picks it up instead of splitting a still-"assigned" range and orphaning its front half. - DownloadHandle now names the real control block (vdm::task:: DownloadTaskState, defined only in the engine TU) via a namespace-scope fwd decl and a public-but-effectively-engine-only ctor, replacing the nested State/friend pair. Every public signature is unchanged; DAEMON (vdm-79) confirmed sched/ names only the public API. Fixes found while building the end-to-end suite (tests/task/engine_test.cpp, 9 cases against tools/testserver, green under ASan/UBSan and TSan): - a dropped connection lost its unflushed WriteBuffer tail while advance() had already counted those bytes as done -> a retry resumed past an unwritten hole. Flush on the failure path. - when the byte counters hit total while other workers were still live, teardown dropped their buffered tails. Now: cancel them and let each worker's own seg_finished drain it (the `assembling` state), last one starts verification -- no cross-thread buffer access. - seg_head() let a 401 with credentials present abort before libcurl's resend; now it proceeds once and acts on the final status. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS	2026-09-10 20:19:31 +04:00
samiandClaude Sonnet 5	5d81b4cdae	core: segment/segmenter + segment/budget (stage 6) ... vdm/ids.hpp — TaskId, an opaque engine handle (DAEMON keeps the wire UUID <-> TaskId map; the engine never sees the UUID). segment/segmenter — per-download range management (docs/04 §3). Initial lazy split; assign_slot() splits the largest remaining range when the budget grants a slot; on_complete(may_steal) either *steals* the second half of the largest remaining range for the same worker (slot-neutral) or returns nullopt so the caller *yields* the slot (ADR 0011 A1); on_failed() returns requeue only on the 3rd consecutive connection error with a mirror present — the remaining range is orphaned and re-split. Non-resumable or unknown-size => exactly 1 segment; never split below min_segment_bytes (1 MiB). Resume ctor rebuilds from a persisted table (falls back to a fresh layout if it doesn't tile [0,total)). One mutex == "the task lock"; segment fields are std::atomic and the store is a std::deque so a steal's append never moves a worker's record. segment/budget — the global allocator (ADR 0011). Owns exactly one ceiling (maxActiveSegments) and min-1-before-seconds fairness: a two-pass allocation (guarantee pass gives every wanting task 1 slot in DAEMON's priority order, then a growth pass round-robins the rest up to each task's effective cap = min(per_task_cap, host cap, 1 if non-resumable)), recomputed from scratch on every edge so a live set_max_active_segments cut naturally yields the excess lowest-priority-first, never a mid-segment kill. DAEMON-facing surface exactly as promised in daemon/docs/core-requests-m1.md / ADR 0011: budget(), segments_active(), starved_tasks(), starved_since(), set_max_active_segments (drain), set_host_segment_cap, set_task_order, on_budget_changed (a jthread coalesces at <=4 Hz; the tasks_starved 0<->nonzero edge fires immediately). Callbacks are copied out and run after the lock is dropped. Tests: segmenter split/steal/requeue/resume math + a concurrent steal-and-advance run; budget min-1 under a tight budget, round-robin growth, host-cap and non-resumable clamps, live-lower shedding lowest-priority-first, starvation below the task count, starved-edge notification, and a concurrent set_want hammer. Green under ASan/UBSan; the steal path and the budget green under TSan. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS	2026-09-10 15:14:14 +04:00