core: answer PROTO's bufferBytes range question; accept inclusive endByte
buffer-sizing.md: the frozen 4 KiB–8 MiB and docs/04's 64 KiB–64 MiB both miss. Recommend 64 KiB – 16 MiB, default 2 MiB, max_total_buffer_bytes unchanged at 256 MiB: - 4 KiB floor is smaller than one libcurl write callback -> a syscall per chunk; 64 KiB is the smallest floor that coalesces. - throughput vs write size is flat past ~8 MiB on NVMe; 8–16 MiB is disk-stall absorption headroom for the fast-pipe/slow-disk case; 64 MiB is cache pressure for zero gain. - 32 segments x 64 MiB = 2 GiB vs the 256 MiB cap means the docs/04 max is unreachable past 4 total active segments — a misleading Options value. 16 MiB is reachable for single-/light-multitask and clamps to 8 MiB under heavy parallelism, which is correct. - default 4 MiB x 20 downloads = 80 MiB, busting the "<=60 MB RSS / 20 downloads" DoD; 2 MiB fits. Filed as request B4. proto-requests-m1.md: B3 endByte accepted as inclusive (HTTP Range semantics, no curl-boundary off-by-one); [start,end) ask withdrawn; stage 6 designed against inclusive. New B3a: the Content-Length: 0 whole-file case needs a representable zero-length segment — min_segment_bytes means CORE never makes empty segments mid-download, so it's only the degenerate case; mild preference for startByte+length over an endByte=startByte-1 sentinel. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
This commit is contained in:
@@ -93,16 +93,47 @@ CORE will emit per segment:
|
||||
| CORE field | Type | Note |
|
||||
|---|---|---|
|
||||
| `index` | int ≥ 0 | **`event.task.progress` currently says `i`.** Pick one name for both. |
|
||||
| `start` | int ≥ 0 | absolute byte offset, inclusive |
|
||||
| `end` | int ≥ 0 | absolute byte offset, **exclusive** — range is `[start, end)` |
|
||||
| `startByte` | int ≥ 0 | absolute byte offset, inclusive |
|
||||
| `endByte` | int | absolute byte offset, **inclusive** — range is `[startByte, endByte]` (see below) |
|
||||
| `completed` | int ≥ 0 | bytes written in this range so far |
|
||||
| `speedBps` | int ≥ 0 | current per-segment rate |
|
||||
| `state` | enum | `connecting` \| `downloading` \| `stalled` \| `complete` \| `failed` |
|
||||
|
||||
Asks: (a) reconcile `i` vs `index` — one spelling in both the `Segment` type and the
|
||||
`event.task.progress` payload; (b) confirm the half-open `[start, end)` convention in the
|
||||
schema `description` so DAEMON and GUI don't off-by-one the last byte; (c) confirm the
|
||||
segment `state` enum values.
|
||||
**Resolved by PROTO at freeze:** `endByte` is **inclusive** (matches HTTP `Range`
|
||||
semantics — `Range: bytes=start-end` is inclusive — and removes an off-by-one at the curl
|
||||
boundary). CORE designs stage 6 (segmenter/stealer) against inclusive. The earlier
|
||||
`[start, end)` ask is withdrawn.
|
||||
|
||||
Still open: (a) reconcile `i` vs `index`; (b) confirm the segment `state` enum values;
|
||||
(c) **empty-segment representation** — see B3a.
|
||||
|
||||
### B3a. Zero-length segment must be representable — *PROTO is fixing; CORE's requirement*
|
||||
|
||||
With `endByte` inclusive and `minimum: 0`, a zero-length segment (`endByte = startByte -
|
||||
1`) at offset 0 is `endByte = -1`, which the schema forbids. The one case CORE actually
|
||||
needs: a **whole-file zero-length download** (`Content-Length: 0`) — one segment, length
|
||||
0. It is a valid HTTP response and the daemon/GUI must be able to hold it.
|
||||
|
||||
CORE will **not** produce empty segments mid-download: the `min_segment_bytes` floor
|
||||
(1 MiB, `docs/04` §3) means the segmenter never splits below 1 MiB and the stealer only
|
||||
takes a half-range if it is ≥ that floor. So B3a is purely about the degenerate
|
||||
whole-file case.
|
||||
|
||||
Preference: encode segments as `startByte` + `length` (+ `completed`) rather than an
|
||||
inclusive `endByte` with a `startByte - 1` sentinel — `length: 0` is then the natural
|
||||
representation and there is no negative value to allow. If `endByte` inclusive stays,
|
||||
then a 0-byte task needs an explicit encoding (an `empty`/`length` field, or permitting
|
||||
`endByte = startByte - 1` with `minimum: -1`) — any of those work for CORE as long as
|
||||
total length 0 round-trips. Flag back if the chosen fix needs anything else from CORE.
|
||||
|
||||
### B4. `bufferBytes` range is wrong in the frozen schema — *see `buffer-sizing.md`*
|
||||
|
||||
`contracts/` froze `bufferBytes` at 4 KiB – 8 MiB; `docs/04` §4 says 64 KiB – 64 MiB
|
||||
default 4 MiB; the RSS DoD (`docs/04` §8) can't hold either default. CORE's analysis and
|
||||
the recommended range (**64 KiB – 16 MiB, default 2 MiB**, `max_total_buffer_bytes`
|
||||
unchanged at 256 MiB) with the global-cap arithmetic is in
|
||||
[`core/docs/buffer-sizing.md`](buffer-sizing.md). PROTO to land schema + `docs/04` §4 +
|
||||
ADR together.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user