core: stage 9 — rules/ (filename sanitization, collision policy, rule matching)
Pure functions only, per AGENT-CORE.md's build order: no I/O, no JSON, no SQL,
no notion of the wire Rule type — DAEMON decodes its own stored/wire
representation into these plain structs and calls in.
- rules/filename.hpp: sanitize_filename() turns a raw candidate (from
net::parse_content_disposition or net::url_filename — neither is
filesystem-safe by design; both headers say so and point here) into one
safe to create on ext4/APFS/NTFS: strips separators and control bytes,
folds NTFS-illegal characters, neutralizes reserved Windows device names,
clamps length on a UTF-8 boundary. Total on hostile input; never empty.
Not the path-traversal security boundary — that's daemon/fs/safepath,
downstream of this and the one that actually matters adversarially.
- rules/collision.hpp: resolve_collision() finds the next free name
Explorer/Finder-style ("name (1).ext", ...) given an existence predicate,
or returns the desired name unchanged under an overwrite policy. Never
fabricates a guaranteed-unique name past its attempt bound — hands back
the last candidate tried rather than hiding a persistent collision.
- rules/match.hpp: match_rules() is the evaluation half of
contracts/schema/types/Rule.schema.json — priority order, first rule
whose present match clauses (extensions/mimeTypes/host & url glob/size
bounds) all hold, wins; a size clause never matches speculatively before
the probe fills in size_bytes. glob_match() is the iterative (not
recursive — bounded work on an all-'*' pattern) matcher both host_pattern
and url_pattern use.
Every header compiles standalone; tests (39 cases) pass under ASan+UBSan and
TSan. core/include/vdm/README.md documents the new public surface.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
This commit is contained in:
@@ -1,12 +1,12 @@
|
||||
# `libveloxcore` — public API
|
||||
|
||||
**Status: M1 in progress.** `util/`, `net/` (http_client, probe, url, content_disposition),
|
||||
`io/` (sparse_file, write_buffer), `meta/veloxpart`, and `segment/` (segmenter, budget)
|
||||
are landed. The **download entry point** — `vdm::Engine`, `vdm::task::DownloadSpec` /
|
||||
`DownloadHandle` / `DownloadCallbacks` — is sketched in `vdm/engine.hpp` and
|
||||
`vdm/task/download.hpp` and **out for DAEMON review**: see
|
||||
[`core/docs/engine-api-m1.md`](../../docs/engine-api-m1.md). Bodies land in CORE stage 8;
|
||||
build against the value types now.
|
||||
`io/` (sparse_file, write_buffer), `meta/veloxpart`, `segment/` (segmenter, budget),
|
||||
`rate/` (token_bucket), `task/`+`engine.hpp` (the download engine itself — `vdm::Engine`,
|
||||
`DownloadSpec`/`DownloadHandle`/`DownloadCallbacks`), and `rules/` (filename sanitization,
|
||||
collision policy, rule-table matching) are landed. `media/` is M4, not started — see
|
||||
[`core/docs/engine-api-m1.md`](../../docs/engine-api-m1.md) for the engine API's own
|
||||
DAEMON-review history.
|
||||
|
||||
Layering (CLAUDE.md §3): this library knows nothing about JSON, SQL, Qt, or RPC. Input is
|
||||
a spec value; output is bytes on disk plus typed callbacks. DAEMON projects engine state
|
||||
@@ -75,3 +75,44 @@ Sink interface — core does no I/O itself. `LogSink` abstract base; DAEMON inst
|
||||
via `set_log_sink()`, default discards. `CallbackSink` adapter (with a min-level filter).
|
||||
`VDM_LOG_{TRACE,DEBUG,INFO,WARN,ERROR}(category, fmt, args...)` — `std::format` syntax,
|
||||
only formatted when a sink is installed and wants the level.
|
||||
|
||||
---
|
||||
|
||||
## `rules/` — filename sanitization, collision policy, rule-table matching
|
||||
|
||||
Pure functions only: no I/O, no filesystem access, no notion of the wire `Rule` type or
|
||||
its JSON/SQL representation. DAEMON owns the rule table (storage, `rules.upsert`, the
|
||||
generated `Rule` type) and decodes it into the plain structs below before calling in.
|
||||
|
||||
### `vdm/rules/filename.hpp`
|
||||
|
||||
`sanitize_filename(raw, max_bytes = 255)` — turns a raw candidate (from
|
||||
`net::parse_content_disposition` or `net::url_filename`, neither of which is
|
||||
filesystem-safe by design — see their own headers) into one safe to create on ext4, APFS,
|
||||
and NTFS alike: strips separators/control bytes, folds NTFS-illegal characters to `_`,
|
||||
neutralizes reserved Windows device names (`CON`, `COM1`, ...), and clamps length on a
|
||||
UTF-8 boundary. Total: never empty, never throws. **Not** the path-traversal security
|
||||
boundary — that's DAEMON's `fs/safepath`, which runs after this and is the one that
|
||||
matters adversarially.
|
||||
|
||||
### `vdm/rules/collision.hpp`
|
||||
|
||||
`resolve_collision(desired, exists, policy, max_attempts = 1000)` — given an existence
|
||||
predicate (DAEMON supplies a real one; tests supply an in-memory set), finds the next free
|
||||
name Explorer/Finder-style (`"name (1).ext"`, `"name (2).ext"`, ...) under
|
||||
`CollisionPolicy::rename`, or returns `desired` unchanged under `::overwrite`. Never
|
||||
fabricates a guaranteed-unique name past `max_attempts` — returns the last candidate tried
|
||||
and leaves "still colliding" for the caller to treat as a real error.
|
||||
|
||||
### `vdm/rules/match.hpp`
|
||||
|
||||
`match_rules(rules, input) -> optional<RuleAction>` — the evaluation half of
|
||||
`contracts/schema/types/Rule.schema.json`: tries rules in ascending `priority` order
|
||||
(ties keep table order), skips disabled rows, returns the first whose every *present*
|
||||
match clause (`extensions`, `mime_types`, `host_pattern`, `url_pattern`,
|
||||
`min_size_bytes`/`max_size_bytes`) is satisfied — an absent clause is not a constraint,
|
||||
and a size clause never matches speculatively when `MatchInput::size_bytes` is still
|
||||
unknown (pre-probe). `std::nullopt` means no rule matched; the caller's own default
|
||||
category applies. `glob_match(pattern, text)` — the `*`/`?` matcher `host_pattern` and
|
||||
`url_pattern` both use, case-insensitive, bounded work even on a pathological
|
||||
all-`*` pattern (iterative, not recursive).
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
// vdm/rules/collision.hpp — when a chosen filename is already taken in the destination
|
||||
// directory, decide what to try next.
|
||||
//
|
||||
// Pure: takes an existence predicate rather than touching a filesystem itself, so it never
|
||||
// races what it's deciding about and stays testable without one. DAEMON (which owns the
|
||||
// actual directory listing / stat calls, downstream of its own fs/safepath gate) supplies
|
||||
// that predicate; a test supplies an in-memory set.
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_RULES_COLLISION_HPP
|
||||
#define VDM_RULES_COLLISION_HPP
|
||||
|
||||
#include <functional>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
namespace vdm::rules {
|
||||
|
||||
enum class CollisionPolicy {
|
||||
rename, // try "name (1).ext", "name (2).ext", ... until one is free
|
||||
overwrite, // return `desired` unchanged — caller intends to replace what's there
|
||||
};
|
||||
|
||||
// Under `CollisionPolicy::rename`: calls `exists(candidate)` first with `desired` itself,
|
||||
// then with "<stem> (1)<ext>", "<stem> (2)<ext>", ... (Explorer/Finder-style, splitting
|
||||
// `desired` on its last '.' the same way `sanitize_filename`'s truncation does), returning
|
||||
// the first candidate for which it returns false. `exists` is never called with anything
|
||||
// but a single leaf name, never a path.
|
||||
//
|
||||
// `max_attempts` bounds a pathological `exists` that always returns true (this function
|
||||
// always returns — it is not fallible): once reached, the last candidate tried is returned
|
||||
// as-is, still possibly colliding. That is deliberately not papered over with a
|
||||
// fabricated-unique name (a timestamp suffix, say) — silently handing back a name nobody
|
||||
// asked for is exactly the kind of thing that turns into a mystery file days later; a
|
||||
// caller that hits the bound should treat it as a real error, not swallow it here.
|
||||
//
|
||||
// Under `CollisionPolicy::overwrite`, `exists` and `max_attempts` are unused — `desired`
|
||||
// is returned unchanged.
|
||||
[[nodiscard]] std::string resolve_collision(std::string_view desired,
|
||||
const std::function<bool(std::string_view)> &exists,
|
||||
CollisionPolicy policy = CollisionPolicy::rename,
|
||||
int max_attempts = 1000);
|
||||
|
||||
} // namespace vdm::rules
|
||||
|
||||
#endif // VDM_RULES_COLLISION_HPP
|
||||
@@ -0,0 +1,49 @@
|
||||
// vdm/rules/filename.hpp — turn a raw, untrusted filename candidate into one safe to
|
||||
// create on a real filesystem, cross-platform.
|
||||
//
|
||||
// This is NOT the path-traversal security boundary — that's DAEMON's fs/safepath (the
|
||||
// process's one canonicalize-and-verify-against-allowed-roots gate; see its own header
|
||||
// comment). This runs earlier and is cooperative, not adversarial-proof on its own: turn
|
||||
// whatever `net::parse_content_disposition` or `net::url_filename` handed back (see their
|
||||
// headers — neither fully sanitizes for the filesystem, by design; this is where that
|
||||
// finishes) into a *reasonable* candidate so an ordinary download doesn't needlessly
|
||||
// collide with a reserved device name, get silently mangled by NTFS, or get truncated
|
||||
// mid-extension by safepath's own leaf check.
|
||||
//
|
||||
// Total on hostile input: never throws, never asserts, never returns empty (falls back to
|
||||
// a generic name). No JSON, no SQL, no Qt, no RPC, no filesystem access (CLAUDE.md §3) —
|
||||
// pure string transformation.
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_RULES_FILENAME_HPP
|
||||
#define VDM_RULES_FILENAME_HPP
|
||||
|
||||
#include <cstddef>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
namespace vdm::rules {
|
||||
|
||||
// Sanitizes `raw` into a single path component safe to create on ext4, APFS, and NTFS
|
||||
// alike:
|
||||
// - strips path separators ('/' always; '\' too — Windows treats it as one) and collapses
|
||||
// any run of '.' that would otherwise still read as a traversal attempt ("..", "...")
|
||||
// down to a single '.', so a stripped-separator name can't reconstitute one
|
||||
// - strips C0 control bytes (incl. NUL) and DEL (0x7F)
|
||||
// - replaces the other NTFS-illegal characters (`< > : " | ? *`) with '_', so a name
|
||||
// that's fine on Linux doesn't silently fail to sync/export to a Windows-formatted
|
||||
// drive or SMB share
|
||||
// - strips trailing '.' and ' ' (both are NTFS traps: silently dropped by the Win32 API,
|
||||
// so "name." and "name" would otherwise collide invisibly on export)
|
||||
// - a reserved Windows device name (CON, PRN, AUX, NUL, COM1–9, LPT1–9), matched
|
||||
// case-insensitively against the part before the first '.' (or the whole name if
|
||||
// there's no '.'), gets a trailing '_' so it stops shadowing a device
|
||||
// - clamps to `max_bytes` (default 255, the common ext4/APFS/NTFS component limit),
|
||||
// cutting on a UTF-8 boundary and preferring to keep a short trailing extension intact
|
||||
// - empty, or entirely stripped down to nothing, falls back to "download"
|
||||
[[nodiscard]] std::string sanitize_filename(std::string_view raw, std::size_t max_bytes = 255);
|
||||
|
||||
} // namespace vdm::rules
|
||||
|
||||
#endif // VDM_RULES_FILENAME_HPP
|
||||
@@ -0,0 +1,91 @@
|
||||
// vdm/rules/match.hpp — the pure evaluation half of the rules engine
|
||||
// (contracts/schema/types/Rule.schema.json): given a rule table and what's known about one
|
||||
// candidate download, decide which rule fires and what it says to do.
|
||||
//
|
||||
// DAEMON owns the rule table itself — storage, `rules.upsert`/`rules.list`, the wire
|
||||
// `Rule` type generated from the contract. It decodes its own JSON/SQL representation into
|
||||
// the plain structs below and calls in; core never sees JSON, SQL, or the generated
|
||||
// protocol types (CLAUDE.md §3) — these structs mirror the contract's shape in CORE's own
|
||||
// vocabulary, the same relationship `util/error.hpp`'s `Error` has to the wire error codes.
|
||||
//
|
||||
// Pure and total: no I/O, no throw, no crash on any input (an empty table, an empty
|
||||
// pattern, a rule with every match clause absent).
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_RULES_MATCH_HPP
|
||||
#define VDM_RULES_MATCH_HPP
|
||||
|
||||
#include <cstdint>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
#include <vector>
|
||||
|
||||
namespace vdm::rules {
|
||||
|
||||
enum class StartMode { now, later, queue }; // mirrors contracts' StartMode
|
||||
enum class CaptureVerdict { take, ignore }; // mirrors RuleAction.capture
|
||||
|
||||
// All present clauses must match; an absent clause is not a constraint
|
||||
// (contracts/schema/types/Rule.schema.json's own wording for RuleMatch).
|
||||
struct RuleMatch {
|
||||
std::optional<std::vector<std::string>> extensions; // no leading '.'; matched
|
||||
// case-insensitively
|
||||
std::optional<std::vector<std::string>> mime_types; // matched case-insensitively
|
||||
std::optional<std::string> host_pattern; // glob_match against the host
|
||||
std::optional<std::string> url_pattern; // glob_match against the whole URL
|
||||
std::optional<std::uint64_t> min_size_bytes; // inclusive
|
||||
std::optional<std::uint64_t> max_size_bytes; // inclusive
|
||||
};
|
||||
|
||||
// What to do with a matching download.
|
||||
struct RuleAction {
|
||||
std::optional<std::string> category_id;
|
||||
std::optional<std::string> save_dir;
|
||||
std::optional<std::string> queue_id;
|
||||
std::optional<std::uint32_t> segments;
|
||||
std::optional<StartMode> start_mode;
|
||||
std::optional<CaptureVerdict> capture;
|
||||
};
|
||||
|
||||
// One row of the rules engine.
|
||||
struct Rule {
|
||||
std::string rule_id;
|
||||
bool enabled = true;
|
||||
std::int64_t priority = 0; // lower runs first
|
||||
RuleMatch match;
|
||||
RuleAction action;
|
||||
};
|
||||
|
||||
// What's known about one candidate download, to match rules against. A field being empty
|
||||
// (not `std::nullopt` — these are plain strings, not optionals) means "unknown, matches no
|
||||
// non-empty clause that needs it" — e.g. `size_bytes` is absent pre-probe, so any rule with
|
||||
// a size clause simply doesn't match yet; the caller is expected to re-run matching once
|
||||
// the probe fills it in, same as DownloadSpec itself gets refined post-probe.
|
||||
struct MatchInput {
|
||||
std::string extension; // lowercased, no leading '.'; empty if none
|
||||
std::string mime_type; // lowercased; empty if unknown
|
||||
std::string host; // lowercased effective-URL host
|
||||
std::string url; // the whole effective URL
|
||||
std::optional<std::uint64_t> size_bytes;
|
||||
};
|
||||
|
||||
// Rules are tried in ascending `priority` order (ties broken by table order), skipping
|
||||
// disabled rows; the first whose every present match clause is satisfied wins.
|
||||
// `std::nullopt` means no rule matched — the caller falls back to its own default category
|
||||
// (this function has no notion of a default; that policy lives with the caller).
|
||||
[[nodiscard]] std::optional<RuleAction> match_rules(const std::vector<Rule> &rules,
|
||||
const MatchInput &input);
|
||||
|
||||
// Case-insensitive glob: '*' matches any run of characters including none, '?' matches
|
||||
// exactly one character. No character classes, no escaping — rule patterns are meant to
|
||||
// stay simple (contracts/schema/types/Rule.schema.json's own hostPattern/urlPattern
|
||||
// description gives only "*.example.com" as the example). Exposed on its own because
|
||||
// hostPattern and urlPattern are both just this against different text, and because it has
|
||||
// its own test table worth keeping separate from match_rules's.
|
||||
[[nodiscard]] bool glob_match(std::string_view pattern, std::string_view text) noexcept;
|
||||
|
||||
} // namespace vdm::rules
|
||||
|
||||
#endif // VDM_RULES_MATCH_HPP
|
||||
Reference in New Issue
Block a user