core: meta/veloxpart — resume sidecar, reader first + fuzzed (stage 5)
util/crc32.hpp — header-only CRC-32 (zlib polynomial, reflected), used to integrity-check the sidecar. meta/veloxpart — the <name>.veloxpart.meta resume file (docs/04 §5). Little-endian, versioned, CRC-32 over the whole record. Layout: magic, version, flags, total_size, downloaded, url set (original/effective/ mirrors), etag/last-modified/content-type, segment records (start, end INCLUSIVE, completed), optional sha256 streaming-hash blob. parse_veloxpart() is the attacker-facing surface (the file sits in a world-writable-ish download dir) and is total on any byte string: CRC checked before any field is interpreted; magic, a version it understands, every count and length bounded by a hard cap AND checked against the remaining buffer; ByteReader latches on overrun; trailing bytes rejected. Every malformation is meta_corrupt / meta_version_unsupported, never a crash or an unbounded allocation. serialize_veloxpart() is deterministic (unchanged sidecar isn't rewritten). File helpers write atomically (temp + rename) and fdatasync the file and its directory. Tests: crc32 known vector; full + minimal round-trips; deterministic serialize; file round-trip; and a truncation/corruption table — bad magic, CRC mismatch (payload and CRC-field flips), future version, truncation at every stage, hostile url_count / segment_count / lp_string length (the case the brief singles out), trailing bytes, impossible segment.completed. tools/fuzz/fuzz_veloxpart — feeds raw bytes and bytes-with-valid-CRC (so the field parser and ByteReader bounds checks are actually reached), and round-trip-stability-checks anything accepted. Ran 1.1M execs clean under ASan+UBSan+libFuzzer (clang++-21); fuzz_content_disposition and fuzz_url likewise re-run to 1.1M. tools/fuzz gains a -runs=0 seed-replay CTest smoke per target (regression tripwire; the campaign stays manual). Fuzz-found and fixed: parse_content_disposition could emit a filename containing NUL / control bytes from a mangled filename* ext-value — strip_path only removed path separators. Now sanitize_leaf() also drops C0 controls and DEL (rules/ still owns the authoritative sanitize; `..` and printable-unsafe content pass through as before). Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01HPPSGhiArbvQgwC2DNiURS
This commit is contained in:
Claude Sonnet 5
@@ -0,0 +1,109 @@
|
||||
// vdm/meta/veloxpart.hpp — the `<name>.veloxpart.meta` resume sidecar (docs/04 §5).
|
||||
//
|
||||
// Written next to the part file so a download survives a daemon crash, a reboot, and a
|
||||
// database loss. Little-endian, versioned, CRC-32 over the whole record, fdatasync'd at
|
||||
// segment boundaries.
|
||||
//
|
||||
// The reader is written first and fuzzed (AGENT-CORE §5): this file lives in a
|
||||
// world-writable-ish download directory, so parse_veloxpart() is total on hostile input —
|
||||
// every malformation is a Result error (meta_corrupt / meta_version_unsupported), never a
|
||||
// crash, an over-read, or an unbounded allocation.
|
||||
//
|
||||
// On-disk layout (all integers little-endian):
|
||||
//
|
||||
// magic "VDMP" 4 bytes
|
||||
// version u16 (this build writes/reads kVersion)
|
||||
// flags u16 (bit0: sha256_state present)
|
||||
// total_size u64 (0 = unknown / chunked)
|
||||
// downloaded u64 (sum of segment.completed; a fast read)
|
||||
// url_count u32
|
||||
// url[0] = original, url[1] = effective, url[2..] = mirrors, each length-prefixed
|
||||
// etag length-prefixed UTF-8
|
||||
// last_modified length-prefixed UTF-8
|
||||
// content_type length-prefixed UTF-8
|
||||
// segment_count u32
|
||||
// per segment: start u64, end u64 (INCLUSIVE), completed u64
|
||||
// [flags bit0] sha256_state_len u32, then that many opaque bytes
|
||||
// crc32 u32 (over every byte before this field)
|
||||
//
|
||||
// length-prefixed = u32 length, then that many bytes.
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_META_VELOXPART_HPP
|
||||
#define VDM_META_VELOXPART_HPP
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
#include <vector>
|
||||
|
||||
#include "vdm/util/bytes.hpp"
|
||||
#include "vdm/util/result.hpp"
|
||||
|
||||
namespace vdm::meta {
|
||||
|
||||
inline constexpr std::uint16_t kVersion = 1;
|
||||
inline constexpr std::uint16_t kFlagHasShaState = 0x0001;
|
||||
|
||||
// Hard caps the reader enforces so a hostile count/length can't drive allocation or work.
|
||||
inline constexpr std::uint32_t kMaxUrls = 64;
|
||||
inline constexpr std::uint32_t kMaxSegments = 1024; // contract ceiling is 32; headroom
|
||||
inline constexpr std::uint32_t kMaxStringLen = 16 * 1024;
|
||||
inline constexpr std::uint32_t kMaxShaStateLen = 4 * 1024;
|
||||
inline constexpr std::size_t kMaxImageBytes = 256 * 1024; // a real sidecar is < 4 KiB
|
||||
|
||||
struct SegmentRecord {
|
||||
std::uint64_t start = 0;
|
||||
std::uint64_t end = 0; // INCLUSIVE, per contract Segment.endByte / ADR 0010
|
||||
std::uint64_t completed = 0;
|
||||
|
||||
[[nodiscard]] std::uint64_t length() const noexcept {
|
||||
return end >= start ? end - start + 1 : 0;
|
||||
}
|
||||
bool operator==(const SegmentRecord &) const = default;
|
||||
};
|
||||
|
||||
struct VeloxPart {
|
||||
std::uint16_t version = kVersion;
|
||||
std::uint16_t flags = 0;
|
||||
std::uint64_t total_size = 0;
|
||||
std::uint64_t downloaded = 0;
|
||||
std::vector<std::string> urls; // [0]=original, [1]=effective, [2..]=mirrors
|
||||
std::string etag;
|
||||
std::string last_modified;
|
||||
std::string content_type;
|
||||
std::vector<SegmentRecord> segments;
|
||||
std::vector<std::byte> sha256_state;
|
||||
|
||||
[[nodiscard]] std::string_view original_url() const {
|
||||
return urls.empty() ? std::string_view{} : std::string_view(urls[0]);
|
||||
}
|
||||
[[nodiscard]] std::string_view effective_url() const {
|
||||
return urls.size() < 2 ? original_url() : std::string_view(urls[1]);
|
||||
}
|
||||
|
||||
bool operator==(const VeloxPart &) const = default;
|
||||
};
|
||||
|
||||
// Parse a sidecar image. Every failure is a Result error, never a throw or a crash:
|
||||
// meta_corrupt — bad magic, truncation, a count/length past a cap or past
|
||||
// the buffer, trailing bytes, or a CRC mismatch
|
||||
// meta_version_unsupported — magic OK, CRC OK, but version > kVersion
|
||||
[[nodiscard]] Result<VeloxPart> parse_veloxpart(ConstByteSpan image);
|
||||
|
||||
// Serialize. Deterministic: the same VeloxPart always produces the same bytes, so an
|
||||
// unchanged sidecar is not rewritten. The CRC-32 is appended.
|
||||
[[nodiscard]] std::vector<std::byte> serialize_veloxpart(const VeloxPart &vp);
|
||||
|
||||
// File helpers — the sidecar path is `<part file>.veloxpart.meta`.
|
||||
[[nodiscard]] Result<VeloxPart> read_veloxpart_file(std::string_view path);
|
||||
|
||||
// Writes atomically (temp + rename) and, when `fsync`, fdatasync's the file and its
|
||||
// directory before returning — call at every segment-boundary update (docs/04 §5).
|
||||
[[nodiscard]] Result<void> write_veloxpart_file(std::string_view path, const VeloxPart &vp,
|
||||
bool fsync = true);
|
||||
|
||||
} // namespace vdm::meta
|
||||
|
||||
#endif // VDM_META_VELOXPART_HPP
|
||||
@@ -21,9 +21,10 @@ struct ContentDisposition {
|
||||
|
||||
Type type = Type::none;
|
||||
|
||||
// Best-effort UTF-8 filename, path components stripped, or empty when the header
|
||||
// carries none. NOT sanitized for the filesystem — that is rules/ (stage 9); this
|
||||
// only decodes and de-mojibakes. `..` and control characters may still be present.
|
||||
// Best-effort UTF-8 filename: path components stripped, control bytes (incl. NUL) and
|
||||
// edge whitespace removed, or empty when the header carries none. NOT fully sanitized
|
||||
// for the filesystem — that is rules/ (stage 9). `..`, reserved names, and other
|
||||
// printable-but-unsafe content may still be present.
|
||||
std::string filename;
|
||||
|
||||
// The filename came from an RFC 5987 `filename*` ext-value (preferred over a plain
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
// vdm/util/crc32.hpp — CRC-32 (IEEE 802.3 / zlib polynomial), header-only.
|
||||
//
|
||||
// Used to integrity-check the .veloxpart.meta resume sidecar (docs/04 §5). Standard
|
||||
// reflected CRC-32 with 0xEDB88320, init/xorout 0xFFFFFFFF — byte-compatible with
|
||||
// zlib's crc32() and `cksum -o3` — so the value is reproducible outside this codebase.
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_UTIL_CRC32_HPP
|
||||
#define VDM_UTIL_CRC32_HPP
|
||||
|
||||
#include <array>
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
|
||||
#include "vdm/util/bytes.hpp"
|
||||
|
||||
namespace vdm {
|
||||
|
||||
namespace detail {
|
||||
inline constexpr std::array<std::uint32_t, 256> make_crc32_table() {
|
||||
std::array<std::uint32_t, 256> t{};
|
||||
for (std::uint32_t i = 0; i < 256; ++i) {
|
||||
std::uint32_t c = i;
|
||||
for (int k = 0; k < 8; ++k)
|
||||
c = (c & 1u) ? (0xEDB88320u ^ (c >> 1)) : (c >> 1);
|
||||
t[i] = c;
|
||||
}
|
||||
return t;
|
||||
}
|
||||
inline constexpr std::array<std::uint32_t, 256> kCrc32Table = make_crc32_table();
|
||||
} // namespace detail
|
||||
|
||||
// Incremental: pass the previous result back as `seed` to continue over split buffers.
|
||||
[[nodiscard]] inline std::uint32_t crc32_update(std::uint32_t seed, ConstByteSpan data) noexcept {
|
||||
std::uint32_t c = seed ^ 0xFFFFFFFFu;
|
||||
for (std::byte b : data)
|
||||
c = detail::kCrc32Table[(c ^ std::to_integer<std::uint8_t>(b)) & 0xFFu] ^ (c >> 8);
|
||||
return c ^ 0xFFFFFFFFu;
|
||||
}
|
||||
|
||||
[[nodiscard]] inline std::uint32_t crc32(ConstByteSpan data) noexcept {
|
||||
return crc32_update(0u, data);
|
||||
}
|
||||
|
||||
} // namespace vdm
|
||||
|
||||
#endif // VDM_UTIL_CRC32_HPP
|
||||
Reference in New Issue
Block a user