core: stage 9 — rules/ (filename sanitization, collision policy, rule matching)
Pure functions only, per AGENT-CORE.md's build order: no I/O, no JSON, no SQL,
no notion of the wire Rule type — DAEMON decodes its own stored/wire
representation into these plain structs and calls in.
- rules/filename.hpp: sanitize_filename() turns a raw candidate (from
net::parse_content_disposition or net::url_filename — neither is
filesystem-safe by design; both headers say so and point here) into one
safe to create on ext4/APFS/NTFS: strips separators and control bytes,
folds NTFS-illegal characters, neutralizes reserved Windows device names,
clamps length on a UTF-8 boundary. Total on hostile input; never empty.
Not the path-traversal security boundary — that's daemon/fs/safepath,
downstream of this and the one that actually matters adversarially.
- rules/collision.hpp: resolve_collision() finds the next free name
Explorer/Finder-style ("name (1).ext", ...) given an existence predicate,
or returns the desired name unchanged under an overwrite policy. Never
fabricates a guaranteed-unique name past its attempt bound — hands back
the last candidate tried rather than hiding a persistent collision.
- rules/match.hpp: match_rules() is the evaluation half of
contracts/schema/types/Rule.schema.json — priority order, first rule
whose present match clauses (extensions/mimeTypes/host & url glob/size
bounds) all hold, wins; a size clause never matches speculatively before
the probe fills in size_bytes. glob_match() is the iterative (not
recursive — bounded work on an all-'*' pattern) matcher both host_pattern
and url_pattern use.
Every header compiles standalone; tests (39 cases) pass under ASan+UBSan and
TSan. core/include/vdm/README.md documents the new public surface.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
This commit is contained in:
@@ -26,6 +26,9 @@ add_library(veloxcore STATIC
|
||||
src/task/digest.cpp
|
||||
src/task/download_task.cpp
|
||||
src/engine.cpp
|
||||
src/rules/filename.cpp
|
||||
src/rules/collision.cpp
|
||||
src/rules/match.cpp
|
||||
)
|
||||
add_library(velox::core ALIAS veloxcore)
|
||||
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# `libveloxcore` — public API
|
||||
|
||||
**Status: M1 in progress.** `util/`, `net/` (http_client, probe, url, content_disposition),
|
||||
`io/` (sparse_file, write_buffer), `meta/veloxpart`, and `segment/` (segmenter, budget)
|
||||
are landed. The **download entry point** — `vdm::Engine`, `vdm::task::DownloadSpec` /
|
||||
`DownloadHandle` / `DownloadCallbacks` — is sketched in `vdm/engine.hpp` and
|
||||
`vdm/task/download.hpp` and **out for DAEMON review**: see
|
||||
[`core/docs/engine-api-m1.md`](../../docs/engine-api-m1.md). Bodies land in CORE stage 8;
|
||||
build against the value types now.
|
||||
`io/` (sparse_file, write_buffer), `meta/veloxpart`, `segment/` (segmenter, budget),
|
||||
`rate/` (token_bucket), `task/`+`engine.hpp` (the download engine itself — `vdm::Engine`,
|
||||
`DownloadSpec`/`DownloadHandle`/`DownloadCallbacks`), and `rules/` (filename sanitization,
|
||||
collision policy, rule-table matching) are landed. `media/` is M4, not started — see
|
||||
[`core/docs/engine-api-m1.md`](../../docs/engine-api-m1.md) for the engine API's own
|
||||
DAEMON-review history.
|
||||
|
||||
Layering (CLAUDE.md §3): this library knows nothing about JSON, SQL, Qt, or RPC. Input is
|
||||
a spec value; output is bytes on disk plus typed callbacks. DAEMON projects engine state
|
||||
@@ -75,3 +75,44 @@ Sink interface — core does no I/O itself. `LogSink` abstract base; DAEMON inst
|
||||
via `set_log_sink()`, default discards. `CallbackSink` adapter (with a min-level filter).
|
||||
`VDM_LOG_{TRACE,DEBUG,INFO,WARN,ERROR}(category, fmt, args...)` — `std::format` syntax,
|
||||
only formatted when a sink is installed and wants the level.
|
||||
|
||||
---
|
||||
|
||||
## `rules/` — filename sanitization, collision policy, rule-table matching
|
||||
|
||||
Pure functions only: no I/O, no filesystem access, no notion of the wire `Rule` type or
|
||||
its JSON/SQL representation. DAEMON owns the rule table (storage, `rules.upsert`, the
|
||||
generated `Rule` type) and decodes it into the plain structs below before calling in.
|
||||
|
||||
### `vdm/rules/filename.hpp`
|
||||
|
||||
`sanitize_filename(raw, max_bytes = 255)` — turns a raw candidate (from
|
||||
`net::parse_content_disposition` or `net::url_filename`, neither of which is
|
||||
filesystem-safe by design — see their own headers) into one safe to create on ext4, APFS,
|
||||
and NTFS alike: strips separators/control bytes, folds NTFS-illegal characters to `_`,
|
||||
neutralizes reserved Windows device names (`CON`, `COM1`, ...), and clamps length on a
|
||||
UTF-8 boundary. Total: never empty, never throws. **Not** the path-traversal security
|
||||
boundary — that's DAEMON's `fs/safepath`, which runs after this and is the one that
|
||||
matters adversarially.
|
||||
|
||||
### `vdm/rules/collision.hpp`
|
||||
|
||||
`resolve_collision(desired, exists, policy, max_attempts = 1000)` — given an existence
|
||||
predicate (DAEMON supplies a real one; tests supply an in-memory set), finds the next free
|
||||
name Explorer/Finder-style (`"name (1).ext"`, `"name (2).ext"`, ...) under
|
||||
`CollisionPolicy::rename`, or returns `desired` unchanged under `::overwrite`. Never
|
||||
fabricates a guaranteed-unique name past `max_attempts` — returns the last candidate tried
|
||||
and leaves "still colliding" for the caller to treat as a real error.
|
||||
|
||||
### `vdm/rules/match.hpp`
|
||||
|
||||
`match_rules(rules, input) -> optional<RuleAction>` — the evaluation half of
|
||||
`contracts/schema/types/Rule.schema.json`: tries rules in ascending `priority` order
|
||||
(ties keep table order), skips disabled rows, returns the first whose every *present*
|
||||
match clause (`extensions`, `mime_types`, `host_pattern`, `url_pattern`,
|
||||
`min_size_bytes`/`max_size_bytes`) is satisfied — an absent clause is not a constraint,
|
||||
and a size clause never matches speculatively when `MatchInput::size_bytes` is still
|
||||
unknown (pre-probe). `std::nullopt` means no rule matched; the caller's own default
|
||||
category applies. `glob_match(pattern, text)` — the `*`/`?` matcher `host_pattern` and
|
||||
`url_pattern` both use, case-insensitive, bounded work even on a pathological
|
||||
all-`*` pattern (iterative, not recursive).
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
// vdm/rules/collision.hpp — when a chosen filename is already taken in the destination
|
||||
// directory, decide what to try next.
|
||||
//
|
||||
// Pure: takes an existence predicate rather than touching a filesystem itself, so it never
|
||||
// races what it's deciding about and stays testable without one. DAEMON (which owns the
|
||||
// actual directory listing / stat calls, downstream of its own fs/safepath gate) supplies
|
||||
// that predicate; a test supplies an in-memory set.
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_RULES_COLLISION_HPP
|
||||
#define VDM_RULES_COLLISION_HPP
|
||||
|
||||
#include <functional>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
namespace vdm::rules {
|
||||
|
||||
enum class CollisionPolicy {
|
||||
rename, // try "name (1).ext", "name (2).ext", ... until one is free
|
||||
overwrite, // return `desired` unchanged — caller intends to replace what's there
|
||||
};
|
||||
|
||||
// Under `CollisionPolicy::rename`: calls `exists(candidate)` first with `desired` itself,
|
||||
// then with "<stem> (1)<ext>", "<stem> (2)<ext>", ... (Explorer/Finder-style, splitting
|
||||
// `desired` on its last '.' the same way `sanitize_filename`'s truncation does), returning
|
||||
// the first candidate for which it returns false. `exists` is never called with anything
|
||||
// but a single leaf name, never a path.
|
||||
//
|
||||
// `max_attempts` bounds a pathological `exists` that always returns true (this function
|
||||
// always returns — it is not fallible): once reached, the last candidate tried is returned
|
||||
// as-is, still possibly colliding. That is deliberately not papered over with a
|
||||
// fabricated-unique name (a timestamp suffix, say) — silently handing back a name nobody
|
||||
// asked for is exactly the kind of thing that turns into a mystery file days later; a
|
||||
// caller that hits the bound should treat it as a real error, not swallow it here.
|
||||
//
|
||||
// Under `CollisionPolicy::overwrite`, `exists` and `max_attempts` are unused — `desired`
|
||||
// is returned unchanged.
|
||||
[[nodiscard]] std::string resolve_collision(std::string_view desired,
|
||||
const std::function<bool(std::string_view)> &exists,
|
||||
CollisionPolicy policy = CollisionPolicy::rename,
|
||||
int max_attempts = 1000);
|
||||
|
||||
} // namespace vdm::rules
|
||||
|
||||
#endif // VDM_RULES_COLLISION_HPP
|
||||
@@ -0,0 +1,49 @@
|
||||
// vdm/rules/filename.hpp — turn a raw, untrusted filename candidate into one safe to
|
||||
// create on a real filesystem, cross-platform.
|
||||
//
|
||||
// This is NOT the path-traversal security boundary — that's DAEMON's fs/safepath (the
|
||||
// process's one canonicalize-and-verify-against-allowed-roots gate; see its own header
|
||||
// comment). This runs earlier and is cooperative, not adversarial-proof on its own: turn
|
||||
// whatever `net::parse_content_disposition` or `net::url_filename` handed back (see their
|
||||
// headers — neither fully sanitizes for the filesystem, by design; this is where that
|
||||
// finishes) into a *reasonable* candidate so an ordinary download doesn't needlessly
|
||||
// collide with a reserved device name, get silently mangled by NTFS, or get truncated
|
||||
// mid-extension by safepath's own leaf check.
|
||||
//
|
||||
// Total on hostile input: never throws, never asserts, never returns empty (falls back to
|
||||
// a generic name). No JSON, no SQL, no Qt, no RPC, no filesystem access (CLAUDE.md §3) —
|
||||
// pure string transformation.
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_RULES_FILENAME_HPP
|
||||
#define VDM_RULES_FILENAME_HPP
|
||||
|
||||
#include <cstddef>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
namespace vdm::rules {
|
||||
|
||||
// Sanitizes `raw` into a single path component safe to create on ext4, APFS, and NTFS
|
||||
// alike:
|
||||
// - strips path separators ('/' always; '\' too — Windows treats it as one) and collapses
|
||||
// any run of '.' that would otherwise still read as a traversal attempt ("..", "...")
|
||||
// down to a single '.', so a stripped-separator name can't reconstitute one
|
||||
// - strips C0 control bytes (incl. NUL) and DEL (0x7F)
|
||||
// - replaces the other NTFS-illegal characters (`< > : " | ? *`) with '_', so a name
|
||||
// that's fine on Linux doesn't silently fail to sync/export to a Windows-formatted
|
||||
// drive or SMB share
|
||||
// - strips trailing '.' and ' ' (both are NTFS traps: silently dropped by the Win32 API,
|
||||
// so "name." and "name" would otherwise collide invisibly on export)
|
||||
// - a reserved Windows device name (CON, PRN, AUX, NUL, COM1–9, LPT1–9), matched
|
||||
// case-insensitively against the part before the first '.' (or the whole name if
|
||||
// there's no '.'), gets a trailing '_' so it stops shadowing a device
|
||||
// - clamps to `max_bytes` (default 255, the common ext4/APFS/NTFS component limit),
|
||||
// cutting on a UTF-8 boundary and preferring to keep a short trailing extension intact
|
||||
// - empty, or entirely stripped down to nothing, falls back to "download"
|
||||
[[nodiscard]] std::string sanitize_filename(std::string_view raw, std::size_t max_bytes = 255);
|
||||
|
||||
} // namespace vdm::rules
|
||||
|
||||
#endif // VDM_RULES_FILENAME_HPP
|
||||
@@ -0,0 +1,91 @@
|
||||
// vdm/rules/match.hpp — the pure evaluation half of the rules engine
|
||||
// (contracts/schema/types/Rule.schema.json): given a rule table and what's known about one
|
||||
// candidate download, decide which rule fires and what it says to do.
|
||||
//
|
||||
// DAEMON owns the rule table itself — storage, `rules.upsert`/`rules.list`, the wire
|
||||
// `Rule` type generated from the contract. It decodes its own JSON/SQL representation into
|
||||
// the plain structs below and calls in; core never sees JSON, SQL, or the generated
|
||||
// protocol types (CLAUDE.md §3) — these structs mirror the contract's shape in CORE's own
|
||||
// vocabulary, the same relationship `util/error.hpp`'s `Error` has to the wire error codes.
|
||||
//
|
||||
// Pure and total: no I/O, no throw, no crash on any input (an empty table, an empty
|
||||
// pattern, a rule with every match clause absent).
|
||||
//
|
||||
// This header compiles standalone.
|
||||
|
||||
#ifndef VDM_RULES_MATCH_HPP
|
||||
#define VDM_RULES_MATCH_HPP
|
||||
|
||||
#include <cstdint>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
#include <vector>
|
||||
|
||||
namespace vdm::rules {
|
||||
|
||||
enum class StartMode { now, later, queue }; // mirrors contracts' StartMode
|
||||
enum class CaptureVerdict { take, ignore }; // mirrors RuleAction.capture
|
||||
|
||||
// All present clauses must match; an absent clause is not a constraint
|
||||
// (contracts/schema/types/Rule.schema.json's own wording for RuleMatch).
|
||||
struct RuleMatch {
|
||||
std::optional<std::vector<std::string>> extensions; // no leading '.'; matched
|
||||
// case-insensitively
|
||||
std::optional<std::vector<std::string>> mime_types; // matched case-insensitively
|
||||
std::optional<std::string> host_pattern; // glob_match against the host
|
||||
std::optional<std::string> url_pattern; // glob_match against the whole URL
|
||||
std::optional<std::uint64_t> min_size_bytes; // inclusive
|
||||
std::optional<std::uint64_t> max_size_bytes; // inclusive
|
||||
};
|
||||
|
||||
// What to do with a matching download.
|
||||
struct RuleAction {
|
||||
std::optional<std::string> category_id;
|
||||
std::optional<std::string> save_dir;
|
||||
std::optional<std::string> queue_id;
|
||||
std::optional<std::uint32_t> segments;
|
||||
std::optional<StartMode> start_mode;
|
||||
std::optional<CaptureVerdict> capture;
|
||||
};
|
||||
|
||||
// One row of the rules engine.
|
||||
struct Rule {
|
||||
std::string rule_id;
|
||||
bool enabled = true;
|
||||
std::int64_t priority = 0; // lower runs first
|
||||
RuleMatch match;
|
||||
RuleAction action;
|
||||
};
|
||||
|
||||
// What's known about one candidate download, to match rules against. A field being empty
|
||||
// (not `std::nullopt` — these are plain strings, not optionals) means "unknown, matches no
|
||||
// non-empty clause that needs it" — e.g. `size_bytes` is absent pre-probe, so any rule with
|
||||
// a size clause simply doesn't match yet; the caller is expected to re-run matching once
|
||||
// the probe fills it in, same as DownloadSpec itself gets refined post-probe.
|
||||
struct MatchInput {
|
||||
std::string extension; // lowercased, no leading '.'; empty if none
|
||||
std::string mime_type; // lowercased; empty if unknown
|
||||
std::string host; // lowercased effective-URL host
|
||||
std::string url; // the whole effective URL
|
||||
std::optional<std::uint64_t> size_bytes;
|
||||
};
|
||||
|
||||
// Rules are tried in ascending `priority` order (ties broken by table order), skipping
|
||||
// disabled rows; the first whose every present match clause is satisfied wins.
|
||||
// `std::nullopt` means no rule matched — the caller falls back to its own default category
|
||||
// (this function has no notion of a default; that policy lives with the caller).
|
||||
[[nodiscard]] std::optional<RuleAction> match_rules(const std::vector<Rule> &rules,
|
||||
const MatchInput &input);
|
||||
|
||||
// Case-insensitive glob: '*' matches any run of characters including none, '?' matches
|
||||
// exactly one character. No character classes, no escaping — rule patterns are meant to
|
||||
// stay simple (contracts/schema/types/Rule.schema.json's own hostPattern/urlPattern
|
||||
// description gives only "*.example.com" as the example). Exposed on its own because
|
||||
// hostPattern and urlPattern are both just this against different text, and because it has
|
||||
// its own test table worth keeping separate from match_rules's.
|
||||
[[nodiscard]] bool glob_match(std::string_view pattern, std::string_view text) noexcept;
|
||||
|
||||
} // namespace vdm::rules
|
||||
|
||||
#endif // VDM_RULES_MATCH_HPP
|
||||
@@ -0,0 +1,40 @@
|
||||
#include "vdm/rules/collision.hpp"
|
||||
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
namespace vdm::rules {
|
||||
namespace {
|
||||
|
||||
// Split on the last '.', unless it's a leading dot (a dotfile like ".gitignore" has no
|
||||
// extension by this convention — matches sanitize_filename's own treatment). Returns
|
||||
// {stem, ext} where `ext` includes the leading '.' when present.
|
||||
std::pair<std::string_view, std::string_view> split_stem_ext(std::string_view name) {
|
||||
auto dot = name.rfind('.');
|
||||
if (dot == std::string_view::npos || dot == 0)
|
||||
return {name, {}};
|
||||
return {name.substr(0, dot), name.substr(dot)};
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string resolve_collision(std::string_view desired,
|
||||
const std::function<bool(std::string_view)> &exists,
|
||||
CollisionPolicy policy, int max_attempts) {
|
||||
if (policy == CollisionPolicy::overwrite)
|
||||
return std::string(desired);
|
||||
|
||||
if (!exists || !exists(desired))
|
||||
return std::string(desired);
|
||||
|
||||
auto [stem, ext] = split_stem_ext(desired);
|
||||
std::string candidate;
|
||||
for (int n = 1; n <= max_attempts; ++n) {
|
||||
candidate = std::string(stem) + " (" + std::to_string(n) + ")" + std::string(ext);
|
||||
if (!exists(candidate))
|
||||
return candidate;
|
||||
}
|
||||
return candidate; // still colliding; see the header comment on why this isn't hidden
|
||||
}
|
||||
|
||||
} // namespace vdm::rules
|
||||
@@ -0,0 +1,127 @@
|
||||
#include "vdm/rules/filename.hpp"
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <cctype>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
namespace vdm::rules {
|
||||
namespace {
|
||||
|
||||
bool is_control_byte(unsigned char c) { return c < 0x20 || c == 0x7F; }
|
||||
|
||||
bool is_ntfs_illegal(char c) {
|
||||
switch (c) {
|
||||
case '<':
|
||||
case '>':
|
||||
case ':':
|
||||
case '"':
|
||||
case '|':
|
||||
case '?':
|
||||
case '*':
|
||||
return true;
|
||||
default:
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// CON, PRN, AUX, NUL, COM1-9, LPT1-9 — Win32 device names, matched case-insensitively.
|
||||
// `stem` is already ASCII-only by the time this runs (everything else has been filtered),
|
||||
// so a byte-wise toupper is enough; no locale, no UTF-8 concerns.
|
||||
bool is_reserved_device_name(std::string_view stem) {
|
||||
static constexpr std::array<std::string_view, 4> kFixed = {"CON", "PRN", "AUX", "NUL"};
|
||||
std::string upper(stem);
|
||||
std::transform(upper.begin(), upper.end(), upper.begin(),
|
||||
[](unsigned char c) { return static_cast<char>(std::toupper(c)); });
|
||||
for (auto f : kFixed)
|
||||
if (upper == f)
|
||||
return true;
|
||||
if (upper.size() == 4 && (upper.starts_with("COM") || upper.starts_with("LPT")) &&
|
||||
upper[3] >= '1' && upper[3] <= '9')
|
||||
return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
// Backs `pos` up off any UTF-8 continuation bytes (10xxxxxx) so a byte-length cut never
|
||||
// splits a multi-byte codepoint. `pos` is a candidate cut index into `s`, 0 <= pos <=
|
||||
// s.size().
|
||||
std::size_t utf8_safe_cut(std::string_view s, std::size_t pos) {
|
||||
while (pos > 0 && (static_cast<unsigned char>(s[pos]) & 0xC0) == 0x80)
|
||||
--pos;
|
||||
return pos;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string sanitize_filename(std::string_view raw, std::size_t max_bytes) {
|
||||
// Pass 1: drop control bytes and path separators outright; fold the other
|
||||
// NTFS-illegal characters to '_'. Everything else (including non-ASCII UTF-8) passes
|
||||
// through untouched.
|
||||
std::string s;
|
||||
s.reserve(raw.size());
|
||||
for (char c : raw) {
|
||||
auto uc = static_cast<unsigned char>(c);
|
||||
if (is_control_byte(uc) || c == '/' || c == '\\')
|
||||
continue;
|
||||
s.push_back(is_ntfs_illegal(c) ? '_' : c);
|
||||
}
|
||||
|
||||
// Pass 2: collapse any run of 2+ '.' down to one, so stripped separators can't
|
||||
// reconstitute a ".." (or longer) traversal-looking sequence out of what's left.
|
||||
{
|
||||
std::string collapsed;
|
||||
collapsed.reserve(s.size());
|
||||
for (std::size_t i = 0; i < s.size(); ++i) {
|
||||
if (s[i] == '.' && i > 0 && collapsed.size() > 0 && collapsed.back() == '.')
|
||||
continue;
|
||||
collapsed.push_back(s[i]);
|
||||
}
|
||||
s = std::move(collapsed);
|
||||
}
|
||||
|
||||
// Pass 3: strip trailing '.' and ' ' — both are silently dropped by the Win32 API, so
|
||||
// leaving them lets two different requested names collide invisibly on export.
|
||||
while (!s.empty() && (s.back() == '.' || s.back() == ' '))
|
||||
s.pop_back();
|
||||
|
||||
if (s.empty())
|
||||
s = "download";
|
||||
|
||||
// Pass 4: reserved device name check, against the part before the first '.' (or the
|
||||
// whole name if there's none). The '_' goes right after the stem, before any
|
||||
// extension ("NUL.txt" -> "NUL_.txt"), so the result still looks like the same kind of
|
||||
// file rather than growing a spurious trailing character after its extension.
|
||||
{
|
||||
auto dot = s.find('.');
|
||||
std::string_view stem = dot != std::string::npos ? std::string_view(s).substr(0, dot)
|
||||
: std::string_view(s);
|
||||
if (is_reserved_device_name(stem))
|
||||
s.insert(stem.size(), "_");
|
||||
}
|
||||
|
||||
// Pass 5: clamp to max_bytes, UTF-8-safe, keeping a short trailing extension intact
|
||||
// where possible.
|
||||
if (s.size() > max_bytes) {
|
||||
std::string_view ext;
|
||||
if (auto dot = s.rfind('.'); dot != std::string::npos && dot > 0 && s.size() - dot <= 16)
|
||||
ext = std::string_view(s).substr(dot);
|
||||
if (ext.size() < max_bytes) {
|
||||
std::size_t stem_budget = max_bytes - ext.size();
|
||||
std::size_t cut = utf8_safe_cut(s, stem_budget);
|
||||
s = s.substr(0, cut) + std::string(ext);
|
||||
} else {
|
||||
s = s.substr(0, utf8_safe_cut(s, max_bytes));
|
||||
}
|
||||
// Re-strip: truncation can expose a new trailing '.'/' ' (e.g. the byte right
|
||||
// before the cut was itself a dot that pass 3 had no reason to touch).
|
||||
while (!s.empty() && (s.back() == '.' || s.back() == ' '))
|
||||
s.pop_back();
|
||||
if (s.empty())
|
||||
s = "download";
|
||||
}
|
||||
|
||||
return s;
|
||||
}
|
||||
|
||||
} // namespace vdm::rules
|
||||
@@ -0,0 +1,98 @@
|
||||
#include "vdm/rules/match.hpp"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cctype>
|
||||
#include <cstddef>
|
||||
#include <numeric>
|
||||
|
||||
namespace vdm::rules {
|
||||
namespace {
|
||||
|
||||
char lower_ascii(char c) { return static_cast<char>(std::tolower(static_cast<unsigned char>(c))); }
|
||||
|
||||
bool ieq(std::string_view a, std::string_view b) {
|
||||
return a.size() == b.size() &&
|
||||
std::equal(a.begin(), a.end(), b.begin(),
|
||||
[](char x, char y) { return lower_ascii(x) == lower_ascii(y); });
|
||||
}
|
||||
|
||||
// A rule's `extensions` entries are documented with no leading '.', but be lenient about
|
||||
// one showing up anyway (a hand-edited rule table, an older client) rather than let a
|
||||
// clause that never matches silently swallow a whole category.
|
||||
std::string_view strip_leading_dot(std::string_view s) {
|
||||
return (!s.empty() && s.front() == '.') ? s.substr(1) : s;
|
||||
}
|
||||
|
||||
bool match_clause(const RuleMatch &m, const MatchInput &in) {
|
||||
if (m.extensions) {
|
||||
bool any = std::any_of(m.extensions->begin(), m.extensions->end(), [&](const auto &e) {
|
||||
return ieq(strip_leading_dot(e), in.extension);
|
||||
});
|
||||
if (!any)
|
||||
return false;
|
||||
}
|
||||
if (m.mime_types) {
|
||||
bool any = std::any_of(m.mime_types->begin(), m.mime_types->end(),
|
||||
[&](const auto &t) { return ieq(t, in.mime_type); });
|
||||
if (!any)
|
||||
return false;
|
||||
}
|
||||
if (m.host_pattern && !glob_match(*m.host_pattern, in.host))
|
||||
return false;
|
||||
if (m.url_pattern && !glob_match(*m.url_pattern, in.url))
|
||||
return false;
|
||||
if (m.min_size_bytes) {
|
||||
if (!in.size_bytes || *in.size_bytes < *m.min_size_bytes)
|
||||
return false;
|
||||
}
|
||||
if (m.max_size_bytes) {
|
||||
if (!in.size_bytes || *in.size_bytes > *m.max_size_bytes)
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool glob_match(std::string_view pattern, std::string_view text) noexcept {
|
||||
// Classic iterative wildcard match (single backtrack point at the most recent '*'), not
|
||||
// the naive recursive version — bounded work on any input, including a pattern that is
|
||||
// nothing but repeated '*'s against a long `text`.
|
||||
std::size_t p = 0, t = 0;
|
||||
std::size_t star = std::string_view::npos, mark = 0;
|
||||
while (t < text.size()) {
|
||||
if (p < pattern.size() && (pattern[p] == '?' || lower_ascii(pattern[p]) == lower_ascii(text[t]))) {
|
||||
++p;
|
||||
++t;
|
||||
} else if (p < pattern.size() && pattern[p] == '*') {
|
||||
star = p++;
|
||||
mark = t;
|
||||
} else if (star != std::string_view::npos) {
|
||||
p = star + 1;
|
||||
t = ++mark;
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
while (p < pattern.size() && pattern[p] == '*') ++p;
|
||||
return p == pattern.size();
|
||||
}
|
||||
|
||||
std::optional<RuleAction> match_rules(const std::vector<Rule> &rules, const MatchInput &input) {
|
||||
std::vector<std::size_t> order(rules.size());
|
||||
std::iota(order.begin(), order.end(), 0);
|
||||
// Stable by construction: std::stable_sort keeps table order among equal priorities.
|
||||
std::stable_sort(order.begin(), order.end(), [&](std::size_t a, std::size_t b) {
|
||||
return rules[a].priority < rules[b].priority;
|
||||
});
|
||||
for (auto i : order) {
|
||||
const Rule &r = rules[i];
|
||||
if (!r.enabled)
|
||||
continue;
|
||||
if (match_clause(r.match, input))
|
||||
return r.action;
|
||||
}
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
} // namespace vdm::rules
|
||||
@@ -55,3 +55,7 @@ if(NOT EXISTS ${_testserver})
|
||||
message(STATUS "veloxcore: tools/testserver not present; net integration tests will "
|
||||
"skip their server-backed cases.")
|
||||
endif()
|
||||
|
||||
vdm_add_test(veloxcore_rules_filename_test rules/filename_test.cpp)
|
||||
vdm_add_test(veloxcore_rules_collision_test rules/collision_test.cpp)
|
||||
vdm_add_test(veloxcore_rules_match_test rules/match_test.cpp)
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
#include "vdm/rules/collision.hpp"
|
||||
|
||||
#include <functional>
|
||||
#include <set>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
#include "vtest.hpp"
|
||||
|
||||
using vdm::rules::CollisionPolicy;
|
||||
using vdm::rules::resolve_collision;
|
||||
|
||||
namespace {
|
||||
std::function<bool(std::string_view)> exists_in(const std::set<std::string> &names) {
|
||||
return [&names](std::string_view s) { return names.count(std::string(s)) > 0; };
|
||||
}
|
||||
} // namespace
|
||||
|
||||
VT_TEST(collision_no_collision_returns_desired) {
|
||||
std::set<std::string> existing = {"other.txt"};
|
||||
VT_CHECK_EQ(resolve_collision("file.txt", exists_in(existing)), std::string("file.txt"));
|
||||
}
|
||||
|
||||
VT_TEST(collision_renames_on_conflict) {
|
||||
std::set<std::string> existing = {"file.txt"};
|
||||
VT_CHECK_EQ(resolve_collision("file.txt", exists_in(existing)), std::string("file (1).txt"));
|
||||
}
|
||||
|
||||
VT_TEST(collision_finds_first_free_slot) {
|
||||
std::set<std::string> existing = {"file.txt", "file (1).txt", "file (2).txt"};
|
||||
VT_CHECK_EQ(resolve_collision("file.txt", exists_in(existing)), std::string("file (3).txt"));
|
||||
}
|
||||
|
||||
VT_TEST(collision_no_extension) {
|
||||
std::set<std::string> existing = {"README"};
|
||||
VT_CHECK_EQ(resolve_collision("README", exists_in(existing)), std::string("README (1)"));
|
||||
}
|
||||
|
||||
VT_TEST(collision_dotfile_treated_as_no_extension) {
|
||||
std::set<std::string> existing = {".gitignore"};
|
||||
VT_CHECK_EQ(resolve_collision(".gitignore", exists_in(existing)),
|
||||
std::string(".gitignore (1)"));
|
||||
}
|
||||
|
||||
VT_TEST(collision_overwrite_policy_ignores_existence) {
|
||||
std::set<std::string> existing = {"file.txt"};
|
||||
VT_CHECK_EQ(resolve_collision("file.txt", exists_in(existing), CollisionPolicy::overwrite),
|
||||
std::string("file.txt"));
|
||||
}
|
||||
|
||||
VT_TEST(collision_gives_up_after_max_attempts_without_fabricating) {
|
||||
auto always_exists = [](std::string_view) { return true; };
|
||||
auto out = resolve_collision("file.txt", always_exists, CollisionPolicy::rename, 3);
|
||||
VT_CHECK_EQ(out, std::string("file (3).txt")); // last attempted, still colliding
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
#include "vdm/rules/filename.hpp"
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "vtest.hpp"
|
||||
|
||||
using vdm::rules::sanitize_filename;
|
||||
|
||||
VT_TEST(filename_passthrough_when_already_clean) {
|
||||
VT_CHECK_EQ(sanitize_filename("report.pdf"), std::string("report.pdf"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_strips_path_separators) {
|
||||
VT_CHECK_EQ(sanitize_filename("a/b\\c.txt"), std::string("abc.txt"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_collapses_dotdot_after_separator_strip) {
|
||||
// "../../etc/passwd" -> separators stripped, then the resulting ".." runs collapse to
|
||||
// a single '.', which strip-trailing-dot then removes entirely.
|
||||
auto out = sanitize_filename("../../etc/passwd");
|
||||
VT_CHECK(out.find("..") == std::string::npos);
|
||||
}
|
||||
|
||||
VT_TEST(filename_strips_control_bytes) {
|
||||
std::string raw = "bad";
|
||||
raw.push_back('\0');
|
||||
raw += "name.txt";
|
||||
auto out = sanitize_filename(raw);
|
||||
VT_CHECK_EQ(out, std::string("badname.txt"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_replaces_ntfs_illegal_chars) {
|
||||
VT_CHECK_EQ(sanitize_filename("a<b>c:d\"e|f?g*h.txt"), std::string("a_b_c_d_e_f_g_h.txt"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_strips_trailing_dot_and_space) {
|
||||
VT_CHECK_EQ(sanitize_filename("name. "), std::string("name"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_empty_falls_back_to_download) {
|
||||
VT_CHECK_EQ(sanitize_filename(""), std::string("download"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_all_stripped_falls_back_to_download) {
|
||||
VT_CHECK_EQ(sanitize_filename("/\\"), std::string("download"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_reserved_device_name_bare) {
|
||||
VT_CHECK_EQ(sanitize_filename("CON"), std::string("CON_"));
|
||||
VT_CHECK_EQ(sanitize_filename("con"), std::string("con_"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_reserved_device_name_with_extension) {
|
||||
VT_CHECK_EQ(sanitize_filename("NUL.txt"), std::string("NUL_.txt"));
|
||||
VT_CHECK_EQ(sanitize_filename("com3.tar.gz"), std::string("com3_.tar.gz"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_reserved_device_name_not_a_false_positive) {
|
||||
// "CONTEST" is not "CON" — must not get mangled.
|
||||
VT_CHECK_EQ(sanitize_filename("CONTEST.txt"), std::string("CONTEST.txt"));
|
||||
VT_CHECK_EQ(sanitize_filename("COM99.txt"), std::string("COM99.txt")); // not COM1-9
|
||||
}
|
||||
|
||||
VT_TEST(filename_truncates_long_name_keeping_extension) {
|
||||
std::string stem(500, 'a');
|
||||
auto out = sanitize_filename(stem + ".txt", 255);
|
||||
VT_CHECK(out.size() <= 255);
|
||||
VT_CHECK(out.ends_with(".txt"));
|
||||
}
|
||||
|
||||
VT_TEST(filename_truncation_is_utf8_safe) {
|
||||
// Each "é" is 2 bytes (C3 A9); a 5-byte budget can fit 2 whole codepoints (4 bytes) but
|
||||
// not a 3rd (needs 6) -- an unguarded byte-length cut at 5 would split the 3rd
|
||||
// codepoint's C3 from its A9, leaving a dangling lead byte.
|
||||
std::string stem;
|
||||
for (int i = 0; i < 20; ++i) stem += "\xC3\xA9";
|
||||
auto out = sanitize_filename(stem, 5);
|
||||
VT_CHECK_EQ(out, std::string("\xC3\xA9\xC3\xA9")); // 2 whole codepoints, 4 bytes
|
||||
}
|
||||
|
||||
VT_TEST(filename_preserves_non_ascii) {
|
||||
VT_CHECK_EQ(sanitize_filename("caf\xC3\xA9.pdf"), std::string("caf\xC3\xA9.pdf"));
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
#include "vdm/rules/match.hpp"
|
||||
|
||||
#include <string>
|
||||
|
||||
#include "vtest.hpp"
|
||||
|
||||
using namespace vdm::rules;
|
||||
|
||||
namespace {
|
||||
|
||||
Rule make_rule(std::string id, std::int64_t priority, RuleMatch m, RuleAction a,
|
||||
bool enabled = true) {
|
||||
Rule r;
|
||||
r.rule_id = std::move(id);
|
||||
r.priority = priority;
|
||||
r.enabled = enabled;
|
||||
r.match = std::move(m);
|
||||
r.action = std::move(a);
|
||||
return r;
|
||||
}
|
||||
|
||||
// -Wmissing-field-initializers (part of -Wextra) flags a designated-initializer list that
|
||||
// skips a member, even one this repo's designated-init style would normally leave
|
||||
// implicit -- these small builders keep the tests below readable without tripping it.
|
||||
RuleAction action_with_category(std::string id) {
|
||||
RuleAction a;
|
||||
a.category_id = std::move(id);
|
||||
return a;
|
||||
}
|
||||
|
||||
MatchInput input_with_extension(std::string ext) {
|
||||
MatchInput in;
|
||||
in.extension = std::move(ext);
|
||||
return in;
|
||||
}
|
||||
|
||||
MatchInput input_with_host(std::string host) {
|
||||
MatchInput in;
|
||||
in.host = std::move(host);
|
||||
return in;
|
||||
}
|
||||
|
||||
MatchInput input_with_size(std::optional<std::uint64_t> size) {
|
||||
MatchInput in;
|
||||
in.size_bytes = size;
|
||||
return in;
|
||||
}
|
||||
|
||||
MatchInput input_with_extension_and_host(std::string ext, std::string host) {
|
||||
MatchInput in;
|
||||
in.extension = std::move(ext);
|
||||
in.host = std::move(host);
|
||||
return in;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
// --- glob_match ---------------------------------------------------------------------
|
||||
|
||||
VT_TEST(glob_exact_match) {
|
||||
VT_CHECK(glob_match("example.com", "example.com"));
|
||||
VT_CHECK(!glob_match("example.com", "example.org"));
|
||||
}
|
||||
|
||||
VT_TEST(glob_star_suffix) {
|
||||
VT_CHECK(glob_match("*.example.com", "cdn.example.com"));
|
||||
VT_CHECK(glob_match("*.example.com", "a.b.example.com"));
|
||||
VT_CHECK(!glob_match("*.example.com", "example.com")); // no room for the literal '.'
|
||||
}
|
||||
|
||||
VT_TEST(glob_star_matches_empty) {
|
||||
VT_CHECK(glob_match("file*.zip", "file.zip"));
|
||||
VT_CHECK(glob_match("file*.zip", "file123.zip"));
|
||||
}
|
||||
|
||||
VT_TEST(glob_question_mark) {
|
||||
VT_CHECK(glob_match("file?.txt", "file1.txt"));
|
||||
VT_CHECK(!glob_match("file?.txt", "file.txt"));
|
||||
VT_CHECK(!glob_match("file?.txt", "file12.txt"));
|
||||
}
|
||||
|
||||
VT_TEST(glob_case_insensitive) {
|
||||
VT_CHECK(glob_match("*.EXAMPLE.com", "cdn.example.COM"));
|
||||
}
|
||||
|
||||
VT_TEST(glob_multiple_stars) {
|
||||
VT_CHECK(glob_match("*foo*bar*", "xxfooyybarzz"));
|
||||
VT_CHECK(!glob_match("*foo*bar*", "xxbarzzfooyy")); // order matters
|
||||
}
|
||||
|
||||
VT_TEST(glob_pathological_stars_terminate) {
|
||||
// A pattern of nothing but '*' against a long text must not blow up (bounded work).
|
||||
std::string pattern(50, '*');
|
||||
std::string text(10000, 'x');
|
||||
VT_CHECK(glob_match(pattern, text));
|
||||
}
|
||||
|
||||
// --- match_rules ---------------------------------------------------------------------
|
||||
|
||||
VT_TEST(match_empty_table_yields_nullopt) {
|
||||
VT_CHECK(!match_rules({}, input_with_extension("zip")).has_value());
|
||||
}
|
||||
|
||||
VT_TEST(match_no_clause_matches_everything) {
|
||||
auto rules = {make_rule("r1", 0, RuleMatch{}, action_with_category("default"))};
|
||||
auto r = match_rules(rules, input_with_extension("anything"));
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK_EQ(*r->category_id, std::string("default"));
|
||||
}
|
||||
|
||||
VT_TEST(match_by_extension_case_insensitive) {
|
||||
RuleMatch m;
|
||||
m.extensions = std::vector<std::string>{"zip", "rar"};
|
||||
auto rules = {make_rule("r1", 0, m, action_with_category("archives"))};
|
||||
VT_CHECK(match_rules(rules, input_with_extension("ZIP")).has_value());
|
||||
VT_CHECK(!match_rules(rules, input_with_extension("txt")).has_value());
|
||||
}
|
||||
|
||||
VT_TEST(match_priority_order_lower_wins) {
|
||||
std::vector<Rule> rules = {
|
||||
make_rule("hi", 10, RuleMatch{}, action_with_category("first")),
|
||||
make_rule("lo", 0, RuleMatch{}, action_with_category("second")),
|
||||
};
|
||||
auto r = match_rules(rules, MatchInput{});
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK_EQ(*r->category_id, std::string("second")); // priority 0 runs first
|
||||
}
|
||||
|
||||
VT_TEST(match_ties_keep_table_order) {
|
||||
std::vector<Rule> rules = {
|
||||
make_rule("a", 5, RuleMatch{}, action_with_category("first")),
|
||||
make_rule("b", 5, RuleMatch{}, action_with_category("second")),
|
||||
};
|
||||
auto r = match_rules(rules, MatchInput{});
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK_EQ(*r->category_id, std::string("first"));
|
||||
}
|
||||
|
||||
VT_TEST(match_skips_disabled_rules) {
|
||||
std::vector<Rule> rules = {
|
||||
make_rule("a", 0, RuleMatch{}, action_with_category("disabled"), /*enabled=*/false),
|
||||
make_rule("b", 1, RuleMatch{}, action_with_category("enabled")),
|
||||
};
|
||||
auto r = match_rules(rules, MatchInput{});
|
||||
VT_REQUIRE(r.has_value());
|
||||
VT_CHECK_EQ(*r->category_id, std::string("enabled"));
|
||||
}
|
||||
|
||||
VT_TEST(match_host_pattern) {
|
||||
RuleMatch m;
|
||||
m.host_pattern = "*.cdn.example.com";
|
||||
auto rules = {make_rule("r1", 0, m, action_with_category("cdn"))};
|
||||
VT_CHECK(match_rules(rules, input_with_host("a.cdn.example.com")).has_value());
|
||||
VT_CHECK(!match_rules(rules, input_with_host("example.com")).has_value());
|
||||
}
|
||||
|
||||
VT_TEST(match_size_bounds) {
|
||||
RuleMatch m;
|
||||
m.min_size_bytes = 1000;
|
||||
m.max_size_bytes = 2000;
|
||||
auto rules = {make_rule("r1", 0, m, action_with_category("midsize"))};
|
||||
VT_CHECK(match_rules(rules, input_with_size(1500)).has_value());
|
||||
VT_CHECK(!match_rules(rules, input_with_size(500)).has_value());
|
||||
VT_CHECK(!match_rules(rules, input_with_size(5000)).has_value());
|
||||
}
|
||||
|
||||
VT_TEST(match_size_clause_with_unknown_size_does_not_match) {
|
||||
RuleMatch m;
|
||||
m.min_size_bytes = 1000;
|
||||
auto rules = {make_rule("r1", 0, m, action_with_category("big"))};
|
||||
// size_bytes left absent (pre-probe) -- a size clause must not match speculatively.
|
||||
VT_CHECK(!match_rules(rules, MatchInput{}).has_value());
|
||||
}
|
||||
|
||||
VT_TEST(match_all_clauses_must_hold) {
|
||||
RuleMatch m;
|
||||
m.extensions = std::vector<std::string>{"iso"};
|
||||
m.host_pattern = "*.trusted.example";
|
||||
auto rules = {make_rule("r1", 0, m, action_with_category("isos"))};
|
||||
VT_CHECK(match_rules(rules, input_with_extension_and_host("iso", "mirror.trusted.example"))
|
||||
.has_value());
|
||||
// Extension matches but host doesn't -- must not match.
|
||||
VT_CHECK(!match_rules(rules, input_with_extension_and_host("iso", "evil.example"))
|
||||
.has_value());
|
||||
}
|
||||
|
||||
VT_TEST(match_falls_through_to_default_when_nothing_matches) {
|
||||
RuleMatch m;
|
||||
m.extensions = std::vector<std::string>{"exe"};
|
||||
auto rules = {make_rule("r1", 0, m, action_with_category("installers"))};
|
||||
VT_CHECK(!match_rules(rules, input_with_extension("pdf")).has_value());
|
||||
}
|
||||
Reference in New Issue
Block a user