core: io/sparse_file + io/write_buffer (stage 4)

io/sparse_file — the single O_WRONLY output file (docs/04 §4). open() posix_fallocate's the full size (falls back to ftruncate on EOPNOTSUPP/ENOSYS, reported via preallocated()); write_at() pwrites at an absolute offset, looping short writes and retrying EINTR; sync() is fdatasync (timer/pause only); advise_dontneed() is posix_fadvise(DONTNEED); resize() trims a preallocated tail or sizes a chunked download. errno -> vdm::Error (ENOSPC->disk_full, EACCES->permission_denied, ENOENT/ENOTDIR/...->path_rejected). No lock on write_at — POSIX makes each pwrite atomic for a regular file, so N segment threads writing disjoint ranges is safe (tested, TSan-clean). io/write_buffer — per-segment accumulate-and-flush buffer, preallocated at construction; append() only memcpys (no allocation on the write- callback hot path, docs/04 §8 — asserted by a global-new counter in the test). Flushes on fill via a caller-supplied FlushFn; a chunk >= capacity arriving on an empty buffer writes straight through. On a flush error next_offset() stays at the last durable position. Single-threaded; the disk-writer-thread handoff is stage 8. Also: vtest.hpp VT_CHECK_EQ/NE now copy operands (auto, not auto&&) — an assertion must not outlive a temporary the expression returned a reference into (ASan caught this on Result<void>{}.error().code). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
2026-09-10 00:41:31 +04:00
parent e8a6b64c2f
commit bd0a24c87a
9 changed files with 781 additions and 4 deletions
@@ -0,0 +1,80 @@
+// vdm/io/sparse_file.hpp — the one output file, written at absolute offsets.
+//
+// docs/04 §4: one file, opened once, O_WRONLY; posix_fallocate the full size up front;
+// each segment pwrite()s at its own offset so there is no reassembly pass; fadvise
+// DONTNEED on written ranges; fdatasync on a timer, never per write.
+//
+// This header compiles standalone.
+
+#ifndef VDM_IO_SPARSE_FILE_HPP
+#define VDM_IO_SPARSE_FILE_HPP
+
+#include <cstdint>
+#include <string>
+#include <string_view>
+
+#include "vdm/util/bytes.hpp"
+#include "vdm/util/result.hpp"
+
+namespace vdm::io {
+
+class SparseFile {
+  public:
+    struct OpenOptions {
+        std::uint64_t total_size = 0;    // full final size; 0 = unknown (chunked transfer)
+        bool preallocate = true;         // posix_fallocate; falls back to ftruncate
+        bool truncate_existing = false;  // start fresh (true) vs. resume into an existing
+                                         // part file (false)
+    };
+
+    SparseFile() = default;
+    ~SparseFile();
+
+    SparseFile(SparseFile &&) noexcept;
+    SparseFile &operator=(SparseFile &&) noexcept;
+    SparseFile(const SparseFile &) = delete;
+    SparseFile &operator=(const SparseFile &) = delete;
+
+    // Open `path` for writing at absolute offsets, creating it if needed. With
+    // `preallocate` and a known `total_size`, posix_fallocate the whole file (contiguous
+    // extents, no ENOSPC surprise at 99%). EOPNOTSUPP/ENOSYS (tmpfs, some network FS)
+    // falls back to ftruncate — sparse, no extent reservation — and is reported by
+    // preallocated().
+    [[nodiscard]] Result<void> open(std::string_view path, const OpenOptions &opts);
+    [[nodiscard]] Result<void> open(std::string_view path);  // default OpenOptions
+
+    [[nodiscard]] bool is_open() const noexcept { return fd_ >= 0; }
+    [[nodiscard]] bool preallocated() const noexcept { return preallocated_; }
+    [[nodiscard]] std::string_view path() const noexcept { return path_; }
+
+    // pwrite the whole span at `offset`, looping over short writes and retrying EINTR.
+    // Safe to call concurrently with other write_at()/sync() on the same object as long
+    // as the byte ranges do not overlap — POSIX guarantees each pwrite is atomic for a
+    // regular file, so no lock is taken on the hot path.
+    [[nodiscard]] Result<void> write_at(std::uint64_t offset, ConstByteSpan data);
+
+    // fdatasync. Call on a timer (default 5 s) and on pause — never per write (docs/04 §4).
+    [[nodiscard]] Result<void> sync();
+
+    // POSIX_FADV_DONTNEED on [offset, offset+len): drop already-written pages from the
+    // page cache so a 40 GB ISO does not evict the user's working set. Most effective
+    // after sync(). Best-effort — failures are ignored.
+    void advise_dontneed(std::uint64_t offset, std::uint64_t len) noexcept;
+
+    // ftruncate to `size`: give a chunked download its real size once known, or trim a
+    // preallocated tail that a steal/mirror never filled.
+    [[nodiscard]] Result<void> resize(std::uint64_t size);
+
+    [[nodiscard]] Result<void> close();
+
+  private:
+    void reset() noexcept;
+
+    int fd_ = -1;
+    bool preallocated_ = false;
+    std::string path_;
+};
+
+}  // namespace vdm::io
+
+#endif  // VDM_IO_SPARSE_FILE_HPP
@@ -0,0 +1,69 @@
+// vdm/io/write_buffer.hpp — per-segment accumulate-and-flush buffer.
+//
+// docs/04 §4: curl's write callback appends; the buffer is flushed with a single pwrite
+// when full or when the segment ends. §8: NO ALLOCATION in the write-callback hot path —
+// the buffer is preallocated at construction and append() only memcpys.
+//
+// Single-threaded. The owning download_task (stage 8) adds the disk-writer-thread handoff
+// (double buffering) on top; this primitive takes no lock.
+//
+// This header compiles standalone.
+
+#ifndef VDM_IO_WRITE_BUFFER_HPP
+#define VDM_IO_WRITE_BUFFER_HPP
+
+#include <cstddef>
+#include <cstdint>
+#include <functional>
+#include <vector>
+
+#include "vdm/util/bytes.hpp"
+#include "vdm/util/result.hpp"
+
+namespace vdm::io {
+
+class WriteBuffer {
+  public:
+    // Writes `span` durably at absolute file offset `offset`. Must not allocate
+    // (SparseFile::write_at doesn't). Return an error to abort the segment; WriteBuffer
+    // propagates it and keeps the un-flushed bytes so the caller can decide.
+    using FlushFn = std::function<Result<void>(std::uint64_t offset, ConstByteSpan span)>;
+
+    // `capacity` is the effective per-segment buffer_bytes (already clamped by the
+    // segmenter against max_total_buffer_bytes). Must be > 0.
+    WriteBuffer(std::uint64_t start_offset, std::size_t capacity, FlushFn flush);
+
+    WriteBuffer(WriteBuffer &&) noexcept = default;
+    WriteBuffer &operator=(WriteBuffer &&) noexcept = default;
+    WriteBuffer(const WriteBuffer &) = delete;
+    WriteBuffer &operator=(const WriteBuffer &) = delete;
+
+    // Append body bytes. Flushes automatically each time the buffer fills. A chunk at
+    // least `capacity` bytes long, arriving when the buffer is empty, is written straight
+    // through (one extra flush call, still no allocation and no memcpy).
+    [[nodiscard]] Result<void> append(ConstByteSpan span);
+
+    // Flush whatever is buffered right now. Call at segment end and on pause. A no-op
+    // (success) when nothing is pending.
+    [[nodiscard]] Result<void> flush();
+
+    [[nodiscard]] std::size_t capacity() const noexcept { return buf_.size(); }
+    [[nodiscard]] std::size_t pending() const noexcept { return len_; }
+    // File offset the next flush will write at (start + all bytes already flushed).
+    [[nodiscard]] std::uint64_t next_offset() const noexcept { return base_; }
+    // Total bytes handed to append() over this buffer's life.
+    [[nodiscard]] std::uint64_t total_appended() const noexcept { return appended_; }
+
+  private:
+    Result<void> flush_pending();
+
+    std::vector<std::byte> buf_;
+    std::size_t len_ = 0;     // bytes currently in buf_
+    std::uint64_t base_ = 0;  // file offset of buf_[0]
+    std::uint64_t appended_ = 0;
+    FlushFn flush_;
+};
+
+}  // namespace vdm::io
+
+#endif  // VDM_IO_WRITE_BUFFER_HPP