sami/vdm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

core: drain-aware hostile-mode handling (etag/416/lying-ranges/mismatch)

Browse Source 
Resumes work left mid-session on the M1 hostile-mode matrix. download_task.cpp:

- Replace the old cancel-then-clear teardown (cancel_all_transfers_locked /
  start_assembly_locked) with a single begin_drain_locked()/PendingAction
  mechanism: cancel every live worker, remember what to do (verify / fail /
  auto_pause / demote), and let whichever worker's seg_finished finds the
  worker map empty carry it out. Every sibling still flushes its buffer on
  the way out, so no buffered-but-unflushed tail is lost when a download
  finishes or fails while other segments are still mid-transfer.
- A 200 where 206 was expected (wrong_status) now checks the response's
  ETag/Last-Modified against the probe's: a real mismatch asks the user
  (server_file_changed, "ask, never silently corrupt" -- docs/04 §5); a match
  means the server just stopped honouring Range for this connection, so
  demote to one segment and keep going without a round trip (docs/04 §7).
- 416 mid-download (stale range metadata) now surfaces as a decision instead
  of retrying the same now-invalid range to exhaustion.
- do_decide's abort path surfaces the actual reason a decision was asked
  for (last_error) instead of hardcoding server_file_changed, which was
  mislabeling a 416 abort.

engine_test.cpp adds the four hostile modes where a bug means silent
corruption rather than a visible failure: etag-changes, 416-always,
lies-about-accept-ranges, content-length-mismatch.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
This commit is contained in:
sami
2026-09-11 13:12:57 +04:00
co-authored by Claude Sonnet 5
parent 0a38867579
commit c99d1d9701
2 changed files with 293 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/tests/task/engine_test.cpp
+131
View File
@@ -62,6 +62,7 @@ struct Recorder {
std::lock_guard lk(mu);
states.push_back(to);
};
c.on_decision_needed = [this](const DecisionRequest &) { decision_calls.fetch_add(1); };
c.on_finished = [this](Result<DownloadOutcome> r) {
if (!fired.exchange(true))
done.set_value(std::move(r));
@@ -326,3 +327,133 @@ VT_TEST(engine_401_then_provide_auth_completes) {
VT_CHECK(rec.auth_calls.load() >= 1);
VT_CHECK_EQ(file_size(td.file("au.bin")), 1u * 1024 * 1024);
}
// --- hostile-mode matrix: the four where a bug is silent corruption, not a visible
// failure (docs/04 §5 "ask, never silently corrupt" / §7's failure-policy table). ---
VT_TEST(engine_etag_changes_asks_instead_of_splicing) {
// A server that revalidates with a different ETag on every response fails an If-Range
// on any retry or resume. That must surface as "ask the user" (server_file_changed),
// never as a silent restart-from-offset-0 spliced onto bytes already on disk.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
auto h = eng.start(spec_for(srv, "/throttled+etag-changes/file/2M", td.file("ec.bin")),
rec.cbs());
// Get real progress on at least one segment before pausing, so resume's If-Range (only
// sent once a segment has completed > 0) actually fires.
for (int i = 0; i < 300 && h.progress().downloaded < 64u * 1024; ++i)
std::this_thread::sleep_for(10ms);
VT_REQUIRE(h.progress().downloaded >= 64u * 1024);
h.pause();
for (int i = 0; i < 200 && h.state() != EngineState::paused; ++i)
std::this_thread::sleep_for(20ms);
VT_REQUIRE(h.state() == EngineState::paused);
h.resume();
for (int i = 0; i < 300 && rec.decision_calls.load() == 0; ++i)
std::this_thread::sleep_for(20ms);
VT_REQUIRE(rec.decision_calls.load() >= 1);
VT_CHECK_EQ(h.state(), EngineState::paused);
h.decide(Decision::restart);
auto r = rec.wait(90s);
VT_REQUIRE(r.has_value());
VT_CHECK_EQ(file_size(td.file("ec.bin")), 2u * 1024 * 1024);
auto got = hash_file(td.file("ec.bin"), Checksum::Algo::sha256);
VT_CHECK_EQ(got.value(), server_sha(srv, "throttled+etag-changes", "2M"));
}
VT_TEST(engine_416_mid_download_asks_instead_of_exhausting_retries) {
// 416-always 416s every ranged request, including the probe's own -- a live probe
// correctly concludes "not resumable" and a plain-GET download never touches Range
// (that path is the same shape as engine_non_resumable_single_segment). The failure
// mode docs/04 means -- a server that *was* proven resumable dropping Range support
// mid-download -- needs a worker to actually send Range against it, so force the
// resumable, multi-segment assumption directly via probe_hint.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
net::ProbeResult hint;
hint.total_size = 256u * 1024;
hint.last_modified = "Wed, 01 Jan 2025 00:00:00 GMT";
hint.accept_ranges = true;
hint.resumable = true;
auto s = spec_for(srv, "/416-always/file/256K", td.file("rb.bin"));
s.probe_hint = hint;
s.segments = 2;
auto h = eng.start(std::move(s), rec.cbs());
for (int i = 0; i < 300 && rec.decision_calls.load() == 0; ++i)
std::this_thread::sleep_for(20ms);
VT_REQUIRE(rec.decision_calls.load() >= 1);
VT_CHECK_EQ(h.state(), EngineState::paused);
// 416-always never recovers -- re-probing would just 416 again -- so the only sound
// resolution is to stop, honestly, rather than retry the stale range until exhaustion.
h.decide(Decision::abort);
auto r = rec.wait();
VT_REQUIRE(!r.has_value());
VT_CHECK_EQ(r.error().code, Error::range_not_satisfiable);
VT_CHECK_EQ(::access(td.file("rb.bin").c_str(), F_OK), -1); // never declared complete
}
VT_TEST(engine_lies_about_accept_ranges_demotes_without_asking) {
// Ranges are always ignored (a plain 200, full body) but ETag/Last-Modified are stable
// and honest -- unlike etag-changes, this is provably the *same* file, just a Range-
// blind connection. docs/04 §7: demote to 1 segment and continue, automatically, no
// user round-trip. As with 416-always, a live probe already gets this right up front
// (proven non-resumable), so probe_hint forces the interesting mid-download case.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
std::string want = server_sha(srv, "lies-about-accept-ranges", "512K");
VT_REQUIRE(!want.empty());
net::ProbeResult hint;
hint.total_size = 512u * 1024;
hint.last_modified = "Wed, 01 Jan 2025 00:00:00 GMT"; // testserver sends this verbatim
hint.accept_ranges = true;
hint.resumable = true;
auto s = spec_for(srv, "/lies-about-accept-ranges/file/512K", td.file("lar.bin"));
s.probe_hint = hint;
s.segments = 4;
auto h = eng.start(std::move(s), rec.cbs());
auto r = rec.wait(60s);
VT_REQUIRE(r.has_value());
VT_CHECK_EQ(rec.decision_calls.load(), 0); // demoted automatically, not asked
VT_CHECK_EQ(file_size(td.file("lar.bin")), 512u * 1024);
auto got = hash_file(td.file("lar.bin"), Checksum::Algo::sha256);
VT_CHECK_EQ(got.value(), want);
}
VT_TEST(engine_content_length_mismatch_fails_honestly) {
// Content-Length promises the true size but the connection always closes short of it.
// There is no recovery (unlike flaky-reset, this never "heals" on a later attempt), so
// the segment's remaining range shrinks every retry until it stalls at zero progress.
// The only correct outcome is a real, visible failure -- never a rename to save_path
// built from a file that is quietly missing bytes.
TestServer srv;
VT_REQUIRE(srv.available());
TmpDir td;
Recorder rec;
Engine eng;
auto s = spec_for(srv, "/content-length-mismatch/file/16K", td.file("clm.bin"));
s.segments = 1;
s.max_retries = 4;
auto h = eng.start(std::move(s), rec.cbs());
auto r = rec.wait(60s);
VT_REQUIRE(!r.has_value());
VT_CHECK_EQ(r.error().code, Error::max_retries_exhausted);
VT_CHECK(rec.saw(EngineState::failed));
VT_CHECK_EQ(::access(td.file("clm.bin").c_str(), F_OK), -1); // never renamed into place
}