79c29b47e8c2cb0995f522779007eeca52dca214
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  samiandClaude Sonnet 5
|
79c29b47e8 |
core: finish the hostile-mode matrix -- 8 remaining end-to-end cases
Was 7/16 of tools/testserver/README.md's mode table covered by engine_test.cpp. Adds the rest: - engine_expiring_signed_url_recovers_via_refresh_url: an expired signed URL 403s, the engine asks (paused, decision_calls >= 1) rather than failing terminally, and DownloadHandle::refresh_url() with a freshly signed URL completes it -- exercises both do_refresh_url() fixes and the probe-level referrer retry's second-403 path from the previous commit. - engine_403_without_referer_retries_with_origin: no spec.referrer set, the automatic single retry (previous commit) recovers with zero decisions asked. - engine_redirect_chain_follows_to_completion: 5 hops of a plain 302. No core-side change needed -- documents that CURLOPT_FOLLOWLOCATION/ MAXREDIRS (already on, RequestOptions::follow_redirects) cover both the probe's and every worker's own request, not just one of the two. - engine_slow_loris_stall_timeout_fires: proves curl's stall detector (CURLOPT_LOW_SPEED_LIMIT/_TIME, download_task.cpp's hardcoded 1024 B/s for 30s) actually fires rather than hanging. Needed a real fix, not just a test: every other test in this file relies on TestServer's short 1s loris dribble to keep runtime down, but 1s of trickle followed by full-speed streaming never accumulates curl's required 30 CONSECUTIVE seconds under the floor, so it would never actually abort -- a test built on the default dribble would pass by the download merely finishing a bit late, not by observing the stall timeout fire. testserver_fixture.hpp's TestServer gained an explicit-loris-seconds constructor (default ctor unchanged, still 1s) so this one test can ask for a dribble (40s) that genuinely outlasts the threshold. - engine_401_digest_then_provide_auth_completes: same shape as the existing 401-basic test: http_client.cpp already asks libcurl for CURLAUTH_ANY regardless of net::AuthScheme, so this needed no core change -- it passed on the first run and is here to prove that's true end-to-end, not just at the http_client unit level. - engine_chunked_no_length_completes_single_segment: Transfer-Encoding: chunked, no Content-Length anywhere (including HEAD). No core change needed -- takes the same size-agnostic "unknown size, one plain-GET segment" path as the existing no-range test. - utf8/legacy-content-disposition: already covered end-to-end by probe_reads_utf8_content_disposition and probe_reads_legacy_content_disposition in probe_test.cpp (probe-level, as these modes only affect the initial request) -- verified passing, no new test needed. All 20 engine_test.cpp cases and all 10 probe_test.cpp cases pass. Every testserver.py spawned while writing and running this was reaped by TestServer's destructor; verified no stragglers with `ps aux` after each run. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
samiandClaude Sonnet 5
|
ef58796d22 |
core: fix Progress.speed_bps reading 0 on the polled path
DAEMON reported Progress.speed_bps reading 0 for the whole life of a live throttled download while downloaded bytes visibly advanced. DAEMON reads progress by polling DownloadHandle::progress() (engine_port_core.hpp), not the on_progress push callback. DownloadTaskState::snapshot_progress() -- the body behind progress() -- never set speed_bps, per-segment speed_bps, or eta_seconds at all; only the event-driven emit_progress_if_due() (which drives the on_progress callback) computed them, from the same live SegWorker::speed_bps EMA seg_data() maintains. snapshot_progress() now reads that same per-worker speed while building its segment list, so a segment with no live worker (idle, paused, complete, failed) correctly reports 0 and a segment with an active transfer reports its real EMA, matching emit_progress_if_due()'s math including the eta_seconds derivation. engine_polled_progress_reports_nonzero_speed reproduces the bug (fails without the fix, confirmed) by polling .progress() -- the same path DAEMON uses -- during a throttled download and asserting speed_bps > 0 once real progress has accumulated. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
samiandClaude Sonnet 5
|
c99d1d9701 |
core: drain-aware hostile-mode handling (etag/416/lying-ranges/mismatch)
Resumes work left mid-session on the M1 hostile-mode matrix. download_task.cpp: - Replace the old cancel-then-clear teardown (cancel_all_transfers_locked / start_assembly_locked) with a single begin_drain_locked()/PendingAction mechanism: cancel every live worker, remember what to do (verify / fail / auto_pause / demote), and let whichever worker's seg_finished finds the worker map empty carry it out. Every sibling still flushes its buffer on the way out, so no buffered-but-unflushed tail is lost when a download finishes or fails while other segments are still mid-transfer. - A 200 where 206 was expected (wrong_status) now checks the response's ETag/Last-Modified against the probe's: a real mismatch asks the user (server_file_changed, "ask, never silently corrupt" -- docs/04 §5); a match means the server just stopped honouring Range for this connection, so demote to one segment and keep going without a round trip (docs/04 §7). - 416 mid-download (stale range metadata) now surfaces as a decision instead of retrying the same now-invalid range to exhaustion. - do_decide's abort path surfaces the actual reason a decision was asked for (last_error) instead of hardcoding server_file_changed, which was mislabeling a 416 abort. engine_test.cpp adds the four hostile modes where a bug means silent corruption rather than a visible failure: etag-changes, 416-always, lies-about-accept-ranges, content-length-mismatch. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
samiandClaude Sonnet 5
|
91636f8a4d |
core: add the download engine — task machine, DownloadHandle, Engine
Stage 8 of the CORE build order: the bodies behind the DownloadSpec / callback API reviewed in core/docs/engine-api-m1.md. Wires probe -> segment workers -> WriteBuffer -> SparseFile -> .veloxpart.meta -> retry/backoff -> SegmentBudget -> RateLimiter -> callbacks into one event-driven machine. - Engine (src/engine.cpp): owns HttpClient, Prober, SegmentBudget, RateLimiter and one timer jthread (min-heap of scheduled fns). start() builds a task and returns a DownloadHandle; ~Impl quiesces every task before joining the timer so no callback fires during teardown. - DownloadTaskState (src/task/download_task.cpp): one `mu` task lock; a shared_mutex over the worker map for the curl write path; callbacks collected under `mu` and fired after release via a separate deferred queue; weak_from_this() in every async hop. State machine over the CORE-owned EngineState subset, auto-pause on 401/407 and on a 200 where 206 was expected, validated resume via If-Range. - digest (src/task/digest.cpp): OpenSSL EVP hash_file() for the optional post-download checksum; links OpenSSL::Crypto PRIVATE. - Segmenter::release_segment(): hand a paused segment back to the pool unassigned so resume's assign_slot() picks it up instead of splitting a still-"assigned" range and orphaning its front half. - DownloadHandle now names the real control block (vdm::task:: DownloadTaskState, defined only in the engine TU) via a namespace-scope fwd decl and a public-but-effectively-engine-only ctor, replacing the nested State/friend pair. Every public signature is unchanged; DAEMON (vdm-79) confirmed sched/ names only the public API. Fixes found while building the end-to-end suite (tests/task/engine_test.cpp, 9 cases against tools/testserver, green under ASan/UBSan and TSan): - a dropped connection lost its unflushed WriteBuffer tail while advance() had already counted those bytes as done -> a retry resumed past an unwritten hole. Flush on the failure path. - when the byte counters hit total while other workers were still live, teardown dropped their buffered tails. Now: cancel them and let each worker's own seg_finished drain it (the `assembling` state), last one starts verification -- no cross-thread buffer access. - seg_head() let a 401 with credentials present abort before libcurl's resend; now it proceeds once and acts on the final status. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |