27865888bea76f5dc383d646604df3ed2bb9fc48
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  samiandClaude Sonnet 5
|
79c29b47e8 |
core: finish the hostile-mode matrix -- 8 remaining end-to-end cases
Was 7/16 of tools/testserver/README.md's mode table covered by engine_test.cpp. Adds the rest: - engine_expiring_signed_url_recovers_via_refresh_url: an expired signed URL 403s, the engine asks (paused, decision_calls >= 1) rather than failing terminally, and DownloadHandle::refresh_url() with a freshly signed URL completes it -- exercises both do_refresh_url() fixes and the probe-level referrer retry's second-403 path from the previous commit. - engine_403_without_referer_retries_with_origin: no spec.referrer set, the automatic single retry (previous commit) recovers with zero decisions asked. - engine_redirect_chain_follows_to_completion: 5 hops of a plain 302. No core-side change needed -- documents that CURLOPT_FOLLOWLOCATION/ MAXREDIRS (already on, RequestOptions::follow_redirects) cover both the probe's and every worker's own request, not just one of the two. - engine_slow_loris_stall_timeout_fires: proves curl's stall detector (CURLOPT_LOW_SPEED_LIMIT/_TIME, download_task.cpp's hardcoded 1024 B/s for 30s) actually fires rather than hanging. Needed a real fix, not just a test: every other test in this file relies on TestServer's short 1s loris dribble to keep runtime down, but 1s of trickle followed by full-speed streaming never accumulates curl's required 30 CONSECUTIVE seconds under the floor, so it would never actually abort -- a test built on the default dribble would pass by the download merely finishing a bit late, not by observing the stall timeout fire. testserver_fixture.hpp's TestServer gained an explicit-loris-seconds constructor (default ctor unchanged, still 1s) so this one test can ask for a dribble (40s) that genuinely outlasts the threshold. - engine_401_digest_then_provide_auth_completes: same shape as the existing 401-basic test: http_client.cpp already asks libcurl for CURLAUTH_ANY regardless of net::AuthScheme, so this needed no core change -- it passed on the first run and is here to prove that's true end-to-end, not just at the http_client unit level. - engine_chunked_no_length_completes_single_segment: Transfer-Encoding: chunked, no Content-Length anywhere (including HEAD). No core change needed -- takes the same size-agnostic "unknown size, one plain-GET segment" path as the existing no-range test. - utf8/legacy-content-disposition: already covered end-to-end by probe_reads_utf8_content_disposition and probe_reads_legacy_content_disposition in probe_test.cpp (probe-level, as these modes only affect the initial request) -- verified passing, no new test needed. All 20 engine_test.cpp cases and all 10 probe_test.cpp cases pass. Every testserver.py spawned while writing and running this was reaped by TestServer's destructor; verified no stragglers with `ps aux` after each run. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
samiandClaude Sonnet 5
|
322a20efa5 |
core: fix SegmentBudget over-admission and add a wait-list wakeup
Root cause of the M7 RSS gap (core/docs/m7-baseline.md: ~70 MB measured against a 60 MB target): SegmentBudget::confirm_slot() only checked a task's own held count against its own target -- never the engine-wide active_ sum. reallocate_locked()'s two-pass fairness allocation does bound sum(target) <= max_active_ at the moment it computes a plan, but that bound says nothing about sum(held): a task can be legitimately holding more than its own just-lowered target for a while (yield is deferred to a segment boundary, never mid-segment -- ADR 0011 A1), and another task's target can correctly rise to claim that capacity before the first task has physically released it. Both confirm_slot() calls could then succeed against their own, individually-correct targets while sum(held) exceeded max_active_ -- tools/bench heap-profile caught this directly: budget.active reading 56-86 against a total of 32. confirm_slot() now also checks active_ < max_active_, unconditionally, as a backstop that doesn't depend on any task's target bookkeeping being in sync with what every other task holds. That creates a liveness question the original design never answered: a task denied only by this new check has a target that's already correct, so it never changes again and reallocate_locked()'s plain "fire a callback when a task's target changes" mechanism never revisits it. Task gained a waiting_for_slot flag, set on exactly this denial; release_slot()/deregister_task() (the only two places that free real capacity) now hand a freed slot directly to the highest-priority waiting task via wake_one_waiter_locked(), if reallocate_locked()'s own plan didn't already produce a callback for anyone. download_task.cpp's fill_slots_locked() needed a matching fix: a woken task's stalled segments (SegState::stalled -- backed off mid-retry, its own release_slot() already called) have no live worker and never surface through Segmenter::assign_slot(), which only hands out unassigned or fresh ranges. fill_slots_locked() now restarts any stalled segment with no live worker directly (bounded by slot_target, same as its assign_slot() loop) before looking for new work; a segment it doesn't get to keeps its own scheduled retry_worker() timer as a second chance. Also fixes a real TSan-caught data race this work surfaced: SegWorker:: speed_bps was written only by its own segment's curl callback and, before Progress.speed_bps's polled-path fix, only ever read from that same thread -- safe without synchronization. snapshot_progress() reading it from whatever thread calls DownloadHandle::progress() broke that invariant (workers_mu's shared_lock protects the workers map's structure, not an individual SegWorker's fields). Now std::atomic<double> with relaxed ordering -- an informational EMA, nothing synchronizes real state on it -- rather than adding a lock to the write side. core/tests/segment/budget_test.cpp adds two tests reproducing the actual gap (budget_active_never_exceeds_max_active_segments_under_concurrent_load, budget_wait_list_wakes_a_task_whose_target_never_changed) plus a sanity baseline (budget_release_wakes_a_denied_waiter), and introduces AsyncFakeTask + TestTimer for the one existing test that drives the budget from multiple concurrent threads -- mirroring production's real dispatch (register_task()'s on_target lambda posts through host.schedule(), download_task.cpp, never a synchronous call) rather than adding reentrancy-guarding machinery to SegmentBudget itself to compensate for a synchronous test double being unlike production. See docs/adr/0017 for the full writeup, including what an earlier version of this fix got wrong chasing a same-thread reentrancy hazard that doesn't actually exist in production. core/docs/m7-baseline.md updated: the RSS number now clears the DoD line (45.41 MiB via heap-profile), root-caused rather than just re-measured. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
samiandClaude Sonnet 5
|
ef58796d22 |
core: fix Progress.speed_bps reading 0 on the polled path
DAEMON reported Progress.speed_bps reading 0 for the whole life of a live throttled download while downloaded bytes visibly advanced. DAEMON reads progress by polling DownloadHandle::progress() (engine_port_core.hpp), not the on_progress push callback. DownloadTaskState::snapshot_progress() -- the body behind progress() -- never set speed_bps, per-segment speed_bps, or eta_seconds at all; only the event-driven emit_progress_if_due() (which drives the on_progress callback) computed them, from the same live SegWorker::speed_bps EMA seg_data() maintains. snapshot_progress() now reads that same per-worker speed while building its segment list, so a segment with no live worker (idle, paused, complete, failed) correctly reports 0 and a segment with an active transfer reports its real EMA, matching emit_progress_if_due()'s math including the eta_seconds derivation. engine_polled_progress_reports_nonzero_speed reproduces the bug (fails without the fix, confirmed) by polling .progress() -- the same path DAEMON uses -- during a throttled download and asserting speed_bps > 0 once real progress has accumulated. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
samiandClaude Sonnet 5
|
d4ad48d494 |
core: stage 9 — rules/ (filename sanitization, collision policy, rule matching)
Pure functions only, per AGENT-CORE.md's build order: no I/O, no JSON, no SQL,
no notion of the wire Rule type — DAEMON decodes its own stored/wire
representation into these plain structs and calls in.
- rules/filename.hpp: sanitize_filename() turns a raw candidate (from
net::parse_content_disposition or net::url_filename — neither is
filesystem-safe by design; both headers say so and point here) into one
safe to create on ext4/APFS/NTFS: strips separators and control bytes,
folds NTFS-illegal characters, neutralizes reserved Windows device names,
clamps length on a UTF-8 boundary. Total on hostile input; never empty.
Not the path-traversal security boundary — that's daemon/fs/safepath,
downstream of this and the one that actually matters adversarially.
- rules/collision.hpp: resolve_collision() finds the next free name
Explorer/Finder-style ("name (1).ext", ...) given an existence predicate,
or returns the desired name unchanged under an overwrite policy. Never
fabricates a guaranteed-unique name past its attempt bound — hands back
the last candidate tried rather than hiding a persistent collision.
- rules/match.hpp: match_rules() is the evaluation half of
contracts/schema/types/Rule.schema.json — priority order, first rule
whose present match clauses (extensions/mimeTypes/host & url glob/size
bounds) all hold, wins; a size clause never matches speculatively before
the probe fills in size_bytes. glob_match() is the iterative (not
recursive — bounded work on an all-'*' pattern) matcher both host_pattern
and url_pattern use.
Every header compiles standalone; tests (39 cases) pass under ASan+UBSan and
TSan. core/include/vdm/README.md documents the new public surface.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
|
||
|
samiandClaude Sonnet 5
|
c99d1d9701 |
core: drain-aware hostile-mode handling (etag/416/lying-ranges/mismatch)
Resumes work left mid-session on the M1 hostile-mode matrix. download_task.cpp: - Replace the old cancel-then-clear teardown (cancel_all_transfers_locked / start_assembly_locked) with a single begin_drain_locked()/PendingAction mechanism: cancel every live worker, remember what to do (verify / fail / auto_pause / demote), and let whichever worker's seg_finished finds the worker map empty carry it out. Every sibling still flushes its buffer on the way out, so no buffered-but-unflushed tail is lost when a download finishes or fails while other segments are still mid-transfer. - A 200 where 206 was expected (wrong_status) now checks the response's ETag/Last-Modified against the probe's: a real mismatch asks the user (server_file_changed, "ask, never silently corrupt" -- docs/04 §5); a match means the server just stopped honouring Range for this connection, so demote to one segment and keep going without a round trip (docs/04 §7). - 416 mid-download (stale range metadata) now surfaces as a decision instead of retrying the same now-invalid range to exhaustion. - do_decide's abort path surfaces the actual reason a decision was asked for (last_error) instead of hardcoding server_file_changed, which was mislabeling a 416 abort. engine_test.cpp adds the four hostile modes where a bug means silent corruption rather than a visible failure: etag-changes, 416-always, lies-about-accept-ranges, content-length-mismatch. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ |
||
|
samiandClaude Sonnet 5
|
91636f8a4d |
core: add the download engine — task machine, DownloadHandle, Engine
Stage 8 of the CORE build order: the bodies behind the DownloadSpec / callback API reviewed in core/docs/engine-api-m1.md. Wires probe -> segment workers -> WriteBuffer -> SparseFile -> .veloxpart.meta -> retry/backoff -> SegmentBudget -> RateLimiter -> callbacks into one event-driven machine. - Engine (src/engine.cpp): owns HttpClient, Prober, SegmentBudget, RateLimiter and one timer jthread (min-heap of scheduled fns). start() builds a task and returns a DownloadHandle; ~Impl quiesces every task before joining the timer so no callback fires during teardown. - DownloadTaskState (src/task/download_task.cpp): one `mu` task lock; a shared_mutex over the worker map for the curl write path; callbacks collected under `mu` and fired after release via a separate deferred queue; weak_from_this() in every async hop. State machine over the CORE-owned EngineState subset, auto-pause on 401/407 and on a 200 where 206 was expected, validated resume via If-Range. - digest (src/task/digest.cpp): OpenSSL EVP hash_file() for the optional post-download checksum; links OpenSSL::Crypto PRIVATE. - Segmenter::release_segment(): hand a paused segment back to the pool unassigned so resume's assign_slot() picks it up instead of splitting a still-"assigned" range and orphaning its front half. - DownloadHandle now names the real control block (vdm::task:: DownloadTaskState, defined only in the engine TU) via a namespace-scope fwd decl and a public-but-effectively-engine-only ctor, replacing the nested State/friend pair. Every public signature is unchanged; DAEMON (vdm-79) confirmed sched/ names only the public API. Fixes found while building the end-to-end suite (tests/task/engine_test.cpp, 9 cases against tools/testserver, green under ASan/UBSan and TSan): - a dropped connection lost its unflushed WriteBuffer tail while advance() had already counted those bytes as done -> a retry resumed past an unwritten hole. Flush on the failure path. - when the byte counters hit total while other workers were still live, teardown dropped their buffered tails. Now: cancel them and let each worker's own seg_finished drain it (the `assembling` state), last one starts verification -- no cross-thread buffer access. - seg_head() let a 401 with credentials present abort before libcurl's resend; now it proceeds once and acts on the final status. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
efbf366c18 |
core: add test-name filters to the vtest harness
run_all() takes an optional substring list; vtest_main forwards argv and a comma-separated $VT_ONLY. No filter => run everything, as before. Makes iterating on one slow end-to-end case (the engine suite) practical without a framework swap. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
479f882324 |
core: O_NOFOLLOW the download target open
DAEMON's safepath-adversarial.md accepts a TOCTOU residual between its canonicalise-and-check and the download starting, on the stated grounds that CORE's O_NOFOLLOW open of the final file closes it. That flag was never actually set: SparseFile::open used O_WRONLY|O_CREAT|O_CLOEXEC, so a symlink swapped in as the final path component after DAEMON's check would be followed and redirect our pwrites outside the allowed roots. Add O_NOFOLLOW. A symlinked leaf now fails the open with ELOOP, which errno_to_error already maps to Error::path_rejected. Regular files and the O_CREAT of a fresh part file are unaffected; resume (existing regular part file) is unaffected. Test that a symlinked destination is rejected rather than silently followed, and that the link target is never touched. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
3da4cd6e91 |
core: rate/token_bucket + fold in DAEMON's engine-API review (stage 7)
rate/token_bucket.hpp — a lazily-refilled TokenBucket (starts full: burst
then throttle, IDM behaviour; rate 0 == unlimited; burst caps idle
accumulation) and RateLimiter, the global -> per-queue -> per-task
hierarchy (docs/04 §6). acquire(task, n) peeks every applicable level and
commits on all-or-none so a blocked attempt never leaks tokens at a level
that had them; held under one mutex so a concurrent detach can't dangle
the bucket it's using. vdm/ids.hpp gains QueueId.
Tests: burst/refill/cap/unlimited for the bucket; tightest-level-binds,
no-partial-consumption, detach-safety, and an 8-thread aggregate-rate
check for the hierarchy. Green under ASan/UBSan and TSan.
Engine-API review (DAEMON signed off, no sched/ or dispatch rewrite):
- Engine::rate_limiter() accessor added (limiter.set -> set_global_limit).
- Checksum::Algo gains sha512 to match the wire Checksum set.
- DownloadSpec: DAEMON creates save_path's parent dir before start();
missing dir -> Error::path_rejected (made explicit).
- cancel(): documented to fire on_state(_, cancelled, nullopt) then
on_finished(Err{canceled}), in that order; download.cancel ==
cancel(false), download.remove == cancel(true).
- engine-api-m1.md: the five open questions resolved with DAEMON's
answers (probe_hint optional, single cancel flag, {restart,
keep_partial, abort} is the whole set, per-task 4 Hz is fine,
refresh_url restarts all segments after a validating re-probe).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
|
||
|
samiandClaude Sonnet 5
|
d6cf1fe7dc |
core: post the engine API for DAEMON review (pre-stage-7)
The download entry point the AGENT-CORE brief asked for on day one and that slipped. DAEMON has an RPC surface and a store and, until this is agreed, nothing in velox::core to call. vdm/task/download.hpp — DownloadSpec (the resolved subset DAEMON hands in: absolute save_path, verbatim browser headers, requested segments/buffer, optional probe_hint / checksum / auth, allow_resume), EngineState (the CORE-owned subset of the wire TaskState), Progress / SegmentProgress, DownloadCallbacks (on_progress <=4 Hz, on_state for every transition incl. auto-pauses, on_auth_required, on_decision_needed, on_finished last), DownloadHandle (pause/resume/cancel — idempotent per the ADR 0013 signature — plus provide_auth / decide / refresh_url, and synchronous state()/progress() snapshots). vdm/engine.hpp — Engine: start(spec, callbacks) -> handle, segment_budget() (DAEMON's sched/ admission surface, ADR 0011), live connection.* setters, a standalone probe() on the pool outside the segment budget. core/docs/engine-api-m1.md — the review doc: field semantics, the state machine, threading/lifetime rules (which thread callbacks arrive on, what is legal from inside one, handle/engine lifetime), the shared-`paused` idempotency contract as a signature, and five open questions for DAEMON. Value types compile and are covered by api_compiles_test; Engine / DownloadHandle bodies land in stage 8, built against whatever DAEMON signs off here. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
5d81b4cdae |
core: segment/segmenter + segment/budget (stage 6)
vdm/ids.hpp — TaskId, an opaque engine handle (DAEMON keeps the wire UUID <-> TaskId map; the engine never sees the UUID). segment/segmenter — per-download range management (docs/04 §3). Initial lazy split; assign_slot() splits the largest remaining range when the budget grants a slot; on_complete(may_steal) either *steals* the second half of the largest remaining range for the same worker (slot-neutral) or returns nullopt so the caller *yields* the slot (ADR 0011 A1); on_failed() returns requeue only on the 3rd consecutive connection error with a mirror present — the remaining range is orphaned and re-split. Non-resumable or unknown-size => exactly 1 segment; never split below min_segment_bytes (1 MiB). Resume ctor rebuilds from a persisted table (falls back to a fresh layout if it doesn't tile [0,total)). One mutex == "the task lock"; segment fields are std::atomic and the store is a std::deque so a steal's append never moves a worker's record. segment/budget — the global allocator (ADR 0011). Owns exactly one ceiling (maxActiveSegments) and min-1-before-seconds fairness: a two-pass allocation (guarantee pass gives every wanting task 1 slot in DAEMON's priority order, then a growth pass round-robins the rest up to each task's effective cap = min(per_task_cap, host cap, 1 if non-resumable)), recomputed from scratch on every edge so a live set_max_active_segments cut naturally yields the excess lowest-priority-first, never a mid-segment kill. DAEMON-facing surface exactly as promised in daemon/docs/core-requests-m1.md / ADR 0011: budget(), segments_active(), starved_tasks(), starved_since(), set_max_active_segments (drain), set_host_segment_cap, set_task_order, on_budget_changed (a jthread coalesces at <=4 Hz; the tasks_starved 0<->nonzero edge fires immediately). Callbacks are copied out and run after the lock is dropped. Tests: segmenter split/steal/requeue/resume math + a concurrent steal-and-advance run; budget min-1 under a tight budget, round-robin growth, host-cap and non-resumable clamps, live-lower shedding lowest-priority-first, starvation below the task count, starved-edge notification, and a concurrent set_want hammer. Green under ASan/UBSan; the steal path and the budget green under TSan. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
092e99f7a0 |
core: meta/veloxpart — resume sidecar, reader first + fuzzed (stage 5)
util/crc32.hpp — header-only CRC-32 (zlib polynomial, reflected), used to integrity-check the sidecar. meta/veloxpart — the <name>.veloxpart.meta resume file (docs/04 §5). Little-endian, versioned, CRC-32 over the whole record. Layout: magic, version, flags, total_size, downloaded, url set (original/effective/ mirrors), etag/last-modified/content-type, segment records (start, end INCLUSIVE, completed), optional sha256 streaming-hash blob. parse_veloxpart() is the attacker-facing surface (the file sits in a world-writable-ish download dir) and is total on any byte string: CRC checked before any field is interpreted; magic, a version it understands, every count and length bounded by a hard cap AND checked against the remaining buffer; ByteReader latches on overrun; trailing bytes rejected. Every malformation is meta_corrupt / meta_version_unsupported, never a crash or an unbounded allocation. serialize_veloxpart() is deterministic (unchanged sidecar isn't rewritten). File helpers write atomically (temp + rename) and fdatasync the file and its directory. Tests: crc32 known vector; full + minimal round-trips; deterministic serialize; file round-trip; and a truncation/corruption table — bad magic, CRC mismatch (payload and CRC-field flips), future version, truncation at every stage, hostile url_count / segment_count / lp_string length (the case the brief singles out), trailing bytes, impossible segment.completed. tools/fuzz/fuzz_veloxpart — feeds raw bytes and bytes-with-valid-CRC (so the field parser and ByteReader bounds checks are actually reached), and round-trip-stability-checks anything accepted. Ran 1.1M execs clean under ASan+UBSan+libFuzzer (clang++-21); fuzz_content_disposition and fuzz_url likewise re-run to 1.1M. tools/fuzz gains a -runs=0 seed-replay CTest smoke per target (regression tripwire; the campaign stays manual). Fuzz-found and fixed: parse_content_disposition could emit a filename containing NUL / control bytes from a mangled filename* ext-value — strip_path only removed path separators. Now sanitize_leaf() also drops C0 controls and DEL (rules/ still owns the authoritative sanitize; `..` and printable-unsafe content pass through as before). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
bd0a24c87a |
core: io/sparse_file + io/write_buffer (stage 4)
io/sparse_file — the single O_WRONLY output file (docs/04 §4). open()
posix_fallocate's the full size (falls back to ftruncate on
EOPNOTSUPP/ENOSYS, reported via preallocated()); write_at() pwrites at an
absolute offset, looping short writes and retrying EINTR; sync() is
fdatasync (timer/pause only); advise_dontneed() is
posix_fadvise(DONTNEED); resize() trims a preallocated tail or sizes a
chunked download. errno -> vdm::Error (ENOSPC->disk_full,
EACCES->permission_denied, ENOENT/ENOTDIR/...->path_rejected). No lock on
write_at — POSIX makes each pwrite atomic for a regular file, so N
segment threads writing disjoint ranges is safe (tested, TSan-clean).
io/write_buffer — per-segment accumulate-and-flush buffer, preallocated
at construction; append() only memcpys (no allocation on the write-
callback hot path, docs/04 §8 — asserted by a global-new counter in the
test). Flushes on fill via a caller-supplied FlushFn; a chunk >= capacity
arriving on an empty buffer writes straight through. On a flush error
next_offset() stays at the last durable position. Single-threaded; the
disk-writer-thread handoff is stage 8.
Also: vtest.hpp VT_CHECK_EQ/NE now copy operands (auto, not auto&&) — an
assertion must not outlive a temporary the expression returned a
reference into (ASan caught this on Result<void>{}.error().code).
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
|
||
|
samiandClaude Sonnet 5
|
201ebc55d4 |
core: net/probe + Content-Disposition parser + URL splitter (stage 3)
net/content_disposition — total parser for the mojibake-prone header: RFC 6266 filename (quoted/token), RFC 5987 filename* ext-values (charset'lang'pct-encoded, incl. RFC 2231 continuations), legacy RFC 2047 encoded-words (=?UTF-8?B?..?= / ?Q?), and raw Latin-1 bytes; prefers filename* over filename; strips path components AFTER decoding (a base64 payload can hold '/'). 22-case test table. net/text_codec (internal) — percent-decode, UTF-8 validation, Latin-1-> UTF-8, base64, RFC 2047 — shared by the CD parser and the URL splitter. net/url — a small total URL splitter (scheme/userinfo/host/port/path/ query/fragment, http(s) validity) and url_filename() for the last path segment; used for the filename fallback. net/probe — HEAD then a ranged GET bytes=0-0 that PROVES resumability (206 + matching Content-Range + a validator), rather than trusting Accept-Ranges which servers lie about; the ranged GET is also the HEAD- refused (403/405/501) fallback. 401/407 -> success result with requires_auth, not an error. Runs on its own pool (max_concurrent, default 4) outside the segment budget per ADR 0011 §5. suggest_filename() does the resolution order (explicit -> disposition -> URL -> download.bin) with a light strip; rules/ (stage 9) owns the authoritative sanitize. tools/fuzz — libFuzzer targets for the CD parser and the URL splitter, compiling the parser sources directly so they're fully instrumented; self-guards on VELOX_BUILD_FUZZ + Clang (the top-level CMake adds every tools/* unconditionally). Seed corpora included. Fixed on the way: a p -> Transfer -> State -> cbs -> p reference cycle in Prober that leaked every probe (drop the stored Transfer; the worker keeps State alive). Tests green under ASan/UBSan and TSan. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
bd1bc029f3 |
core: net/http_client — libcurl multi wrapper (stage 2)
One HttpClient owns a small pool of workers, each with its own CURLM; an easy handle lives on one worker for its life. Public start/pause/resume/ cancel enqueue a command + curl_multi_wakeup(); callbacks (on_head / on_data / on_finished) run on the worker thread and return a DataAction (proceed / pause / abort). Covers redirects (final-response head only), ranges (inclusive ByteRange -> CURLOPT_RANGE), proxy/SOCKS5, basic/digest auth, cookies, verbatim headers, stall detection, a coarse recv-rate cap, and a curl_share DNS/TLS cache across workers. CURLcode + HTTP status -> vdm::Error in net/curl_error. A probe is on_head returning abort: it finishes successfully (head_complete), not canceled. Tests drive tools/testserver: full GET, ranged 206, redirect chain, 404 -> not_found, connection refused -> connect_failed, HEAD probe + ranged 0-0 probe (no body), cancel mid-transfer, pause/resume completes. Skip cleanly if testserver isn't in the tree. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
a40585f419 |
core: clang-format pass against the landed root .clang-format
Pure formatting, no behaviour change. PKG landed .clang-format (Google base, 4-space indent, 100 cols); this brings util/ and the test harness into conformance so `clang-format --dry-run -Werror` is clean. Build and all six test binaries unchanged and green. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
ddf36e848a |
core: add util layer — Result, Error taxonomy, bytes, event bus, pool, log
util/ carries no wire surface, so it lands before the contract freeze. - error: enum class Error, the engine-wide failure taxonomy; is_retryable enumerates every value (no default:) so -Wswitch forces the retry decision on each future addition. ErrorInfo carries context/http_status. - result: Result<T> over std::expected<T, ErrorInfo>, Result<void>, VDM_TRY / VDM_TRY_ASSIGN. Errors returned, never thrown, on the transfer path. - bytes: span aliases, LE load_le/store_le (debug-asserted precondition, not input validation), and a bounds-checked latching ByteReader for the .veloxpart.meta reader. - event_bus: typed thread-safe pub/sub; header states plainly that unsubscribe is not a quiesce point and download_task will need its own drain. - thread_pool: std::jthread pool; dtor joins in the body before members die (fixed a use-after-destruction on cv_/mu_). Header notes shutdown is drain-only and DAEMON will need a cancel mode. - log: sink interface (core does no I/O); DAEMON installs one. Tested: -Werror clean, 6 binaries green under plain / ASan+UBSan / TSan. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Sonnet 5
|
5ac74ecbd9 |
core: add build skeleton and provisional test harness
Self-contained core/CMakeLists.txt (veloxcore STATIC + velox::core alias), warnings at target scope so the sanitizer presets' CMAKE_CXX_FLAGS override doesn't drop -Werror. vtest: ~150-line header-only harness (VT_TEST / VT_CHECK / VT_REQUIRE / VT_CHECK_EQ) behind a one-function vdm_add_test(), so the swap to a real framework once PKG picks one is mechanical. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS |
||
|
samiandClaude Opus 5
|
8bb683b09d |
scaffold: project structure, wire contract, roadmap and agent briefs
Lays out Velox Download Manager (IDM-class download manager for Ubuntu 26.04) as a monorepo ready for parallel lane development. No implementation code by design. - docs/: architecture, roadmap M0-M7, IDM-parity GUI spec, engine design, Firefox extension spec, risks/spikes, packaging - contracts/: wire-contract skeleton (JSON Schema + fixture templates) — the single synchronization point between lanes - docs/agents/: one brief per lane (PROTO, CORE, DAEMON, GUI, EXT, PKG/QA) with owned directories, build order and definition of done - CLAUDE.md: rules of engagement — lane ownership, layering, non-negotiables - CMake scaffolding with dev/tsan/release/ci presets Two environment findings shape the design: Firefox here is the Mozilla snap (native-messaging risk, so the extension carries a loopback-WebSocket fallback), and Wayland forbids passive clipboard monitoring (so clipboard capture is explicit-action-first). Co-Authored-By: Claude Opus 5 <[email protected]> |