vdm/docs/adr/0012-buffer-and-segment-budget.md

ADR 0012 — Buffer bounds, the total-buffer cap, and maxActiveSegments

Status: accepted · Date: 2026-09-09 · Lane: PROTO Answers: core/docs/buffer-sizing.md (CORE's B4)

Context

docs/04 §4 (line 70) said 64 KiB – 64 MiB, default 4 MiB, global cap 256 MiB. The frozen contract said 4 KiB – 8 MiB in four places. Neither matched the other, and CORE's reconciliation found the deeper problem: docs/04's own performance target — "≤ 60 MB RSS with 20 active downloads at default buffers" (§8, line 125) — assumed one buffer per download. The design is one buffer per segment ("Per-segment ring buffer", §4 line 68). Twenty downloads at the 8-segment default is 160 buffers, not 20. At any of the candidate defaults (4 MiB, 2 MiB, even 1 MiB) that arithmetic busts the 60 MB target by 3–4×, and the 256 MiB global cap does too on its own.

CORE also found a second wire gap: the clamp the engine has to implement — reduce every live segment's buffer to fit the global cap — has no field to report the reduced value on, so the GUI cannot show what a download is actually using, only what was requested.

Decision

Bounds, in all four schema locations (DownloadSpec.bufferBytes, TaskDetail.bufferBytes, download.update's patch, Settings.connection.bufferBytes):

	Old (frozen 1.0.0)	New (1.1.0)
minimum	4096 (4 KiB)	65536 (64 KiB)
maximum	8388608 (8 MiB)	16777216 (16 MiB)
default	(unstated on the wire)	1048576 (1 MiB)

64 KiB because 4 KiB is smaller than one libcurl HTTP/2 write-callback delivery — a buffer that small does not coalesce anything, it just adds a layer between curl and pwrite that does nothing. 16 MiB because throughput from write size is flat past ~1–4 MiB on NVMe; the only thing 8–16 MiB buys past that is absorbing a disk stall without stalling the socket, and past 16 MiB there is stall-cover left to buy but not memory left to spend it on. 1 MiB default because it is the only default of the three considered (4 MiB, 2 MiB, 1 MiB) for which the RSS target below actually holds.

Two new settings keys, connection.maxTotalBufferBytes (default 134217728, 128 MiB) and connection.maxActiveSegments (default 32). Both are new, both minor. Without the first, CORE's clamp has no configuration surface. Without the second, "20 active downloads" has no wire meaning distinct from "160 live TLS connections", which is the actual RSS driver — docs/01 §2 already assumes a cap ("~8 active segments" per transfer thread) that docs/04 never stated as a number.

TaskDetail.effectiveBufferBytes (new, per B2a, already accepted in contracts/proto-answers-m1.md): what a live segment is actually using, after the daemon divides maxTotalBufferBytes across live segments (capped at maxActiveSegments) and clamps down to fit. bufferBytes stays the requested value on every type; effective and requested sit side by side wherever both apply, the same pattern TaskSummary.segments / DownloadSpec.segments already established in ADR 0010.

The RSS target: 60 MB stands, and is now conditional in writing. CORE offered a choice — keep 60 MB and rely on maxActiveSegments = 32, or raise it to 120 MB and use a looser cap. This ADR keeps 60 MB, because "lean daemon" is already a stated goal (docs/01) and CORE's own arithmetic gets to 45–50 MB at the chosen defaults — comfortable margin, not a number that only barely holds. docs/04 §8 now states the RSS target is conditional on maxActiveSegments = 32 and default buffers, so it stops being a claim nobody can check.

Why this is a minor bump, not major

Every change either widens a range (4 KiB–8 MiB → 64 KiB–16 MiB, which is not a superset in both directions — see below), adds a field, or adds a settings key. Rule 4 in contracts/README.md: optional field or new method is minor; only rename/remove/retype is major.

The one place this needs a caller's attention despite being "minor": the new minimum (64 KiB) is higher than the old one (4 KiB), and the new maximum (16 MiB) is lower than old docs/04's stated 64 MiB (though higher than the old frozen 8 MiB). A value that validated against the pre-1.1.0 schema — say, bufferBytes: 2048 — no longer validates. Nothing shipped against 1.0.0 yet (this repository is the only consumer), so there is no live client to break, but the general rule for a future minor bump that narrows one bound while widening another is: check it against every existing fixture before shipping, which tests/conformance/check_contract.py now does mechanically — this bump passed because both prior fixture values (4 MiB) happened to fall inside the new range too.

Consequences

• TaskDetail gains a field; Settings and SettingKey gain two keys; four bufferBytes constraint blocks change bounds and gain a stated default.
• DAEMON's scheduler needs maxActiveSegments to decide what to admit — this is the concrete data DAEMON asked for in its own admission-control proposal (currently under discussion as a separate ADR against a different number; PROTO takes no position on that split here beyond supplying the wire field DAEMON needs to read).
• docs/04 §4 and §8 are updated in the same change, per CLAUDE.md rule 5 (docs move with the behaviour they describe) and per CORE's explicit request to land schema, docs and ADR together.
• The download.get fixture now shows a real clamp — 16 MiB requested, 4 MiB effective — rather than a case where the cap happens not to bind, so the requested/effective split actually gets exercised by conformance.

Alternatives rejected

120 MB RSS target, looser cap. CORE offered this and it is defensible — 32 live TLS connections have an irreducible cost, and IDM budgets more. Rejected because the tighter number is achievable with margin per CORE's own arithmetic and "lean daemon" is already a stated design goal; raising a target because it is easier is the wrong direction to move without a measurement forcing it. If M7's tools/bench run shows 60 MB is not actually reachable in practice, that is new information and reopens this ADR — it is not a reason to soften the number pre-emptively.

Keeping docs/04's 64 MiB ceiling. Rejected on CORE's own reachability argument: against a 128 MiB total cap, 64 MiB per segment is unreachable past 2 live segments, which under correct per-segment accounting is a quarter of one default 8-segment download. A ceiling nothing can reach is not a ceiling, it is dead text.