Files
vdm/tools/bench/support/testserver_client.hpp
T
samiandClaude Sonnet 5 b60d4e6f5b core: add tools/bench (throughput/load/alloc-check) and record the M7 baseline
Three subcommands in one binary, driving vdm::Engine directly (docs/04 §8):

- throughput: a single download against a fast local origin
  (support/local_server.hpp, busybox httpd), reporting Mbps/CPU%/RSS. Gates
  on --require-mbps/--max-cpu-pct only when passed, so the ctest smoke
  registration stays a correctness check, not a hardware-dependent
  perf gate -- the real 1-Gbit-link sign-off is a manual/CI job (see the
  file's header comment).
- load: N concurrent tasks against tools/testserver's `throttled` mode
  (support/testserver_client.hpp), reporting peak RSS via getrusage(). Paced
  externally rather than through the engine's own rate::RateLimiter or
  busybox: the limiter's pause/resume path allocates on every throttle event
  (would contaminate alloc-check's measurement) and under heavy segment
  contention was found to starve individual tasks indefinitely (see
  docs/adr/0016, added here); busybox couldn't sustain the DoD's ~160
  concurrent connections (20 tasks * default_segments=8) reliably. The
  ctest registration runs at reduced concurrency under sanitizer presets --
  see the CMakeLists.txt comment and the ADR's postscript.
- alloc-check: operator new/delete overridden process-wide, sampling the
  allocation count across a steady mid-transfer window against a paced
  tools/testserver origin. Caught a real bug in the same change (see the
  http_client.cpp commit) and, by dropping its Engine mid-download to end
  cleanly, also surfaced the quiesce() use-after-free (see that commit).

core/docs/m7-baseline.md records actual measured numbers against the M1/M7
DoD lines, including where they don't clear yet (RSS ~70 MB vs a 60 MB
target; throughput/CPU only measured on loopback, no 1 Gbit link available
here) rather than rounding them away.

docs/adr/0016 documents a rate::RateLimiter fairness gap found building the
load subcommand: a single shared TokenBucket under heavy segment contention
has no fairness ordering across its peek/commit race and can starve a
waiter well past what its configured rate implies. Filed as a follow-up
(it's a core/src/rate design question, not a tools/bench one) rather than
fixed here, along with a related TSan-only load-test straggler that could
not be root-caused in this environment.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
2026-09-11 13:12:57 +04:00

127 lines
4.6 KiB
C++

// tools/bench/support/testserver_client.hpp — spawn tools/testserver for benches that need
// a genuinely paced, external source (alloc-check's steady-state sampling window).
//
// LocalServer (busybox httpd) is right for the throughput/load numbers -- it's fast and
// honest, closer to "a real origin". But that speed is exactly wrong for alloc-check: at
// this server's loopback throughput a modest file transfers in well under the sampling
// window, leaving nothing to sample mid-transfer. Rather than pace the transfer with the
// engine's own rate::RateLimiter (whose curl-write-callback pause/resume path allocates a
// timer node + std::function per throttle event -- exactly the kind of cost alloc-check
// exists to catch, so using it to build the fixture would contaminate the measurement),
// pace it *externally*: tools/testserver's `throttled` mode sleeps between chunks
// server-side, so curl's write callback fires already spaced out and the engine never
// takes the pause/resume path at all.
//
// Linux-only (fork/exec/pipe/kill), same shape as core/tests/net/testserver_fixture.hpp
// (which this mirrors rather than includes -- that header lives under core/tests and pulls
// in VDM_TESTSERVER_PY via core/tests/CMakeLists.txt's own injection; tools/bench gets its
// own so the two test trees stay independently buildable).
#ifndef VDM_BENCH_TESTSERVER_CLIENT_HPP
#define VDM_BENCH_TESTSERVER_CLIENT_HPP
#include <fcntl.h>
#include <signal.h>
#include <sys/wait.h>
#include <unistd.h>
#include <cerrno>
#include <chrono>
#include <cstdlib>
#include <string>
#include <thread>
#ifndef VDM_TESTSERVER_PY
#define VDM_TESTSERVER_PY ""
#endif
namespace vdm::bench {
class TestServerProc {
public:
// `throttle_bps` sets the `throttled` mode's rate (testserver's own default is 1
// MiB/s, generally too slow to keep a ctest's TIMEOUT happy at a size big enough to
// span a sampling window -- callers pacing a specific --size/--window-s pair should
// pass one sized to match, same arithmetic cmd_load uses for its own rate limit).
explicit TestServerProc(std::uint64_t throttle_bps = 0) {
const char *script = VDM_TESTSERVER_PY;
if (!script || !*script || ::access(script, R_OK) != 0)
return;
int pipefd[2];
if (::pipe(pipefd) != 0)
return;
pid_ = ::fork();
if (pid_ < 0) {
::close(pipefd[0]);
::close(pipefd[1]);
pid_ = -1;
return;
}
if (pid_ == 0) {
::dup2(pipefd[1], STDOUT_FILENO);
::close(pipefd[0]);
::close(pipefd[1]);
int devnull = ::open("/dev/null", O_WRONLY);
if (devnull >= 0)
::dup2(devnull, STDERR_FILENO);
std::string bps_str = std::to_string(throttle_bps ? throttle_bps : 1048576ull);
::execlp("python3", "python3", script, "--port", "0", "--seed", "7",
"--throttle-bps", bps_str.c_str(), static_cast<char *>(nullptr));
::_exit(127);
}
::close(pipefd[1]);
std::string line;
char c = 0;
auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(10);
while (std::chrono::steady_clock::now() < deadline) {
ssize_t r = ::read(pipefd[0], &c, 1);
if (r == 1) {
if (c == '\n')
break;
line += c;
} else if (r == 0) {
break;
} else if (errno != EINTR) {
break;
}
}
::close(pipefd[0]);
if (!line.empty())
port_ = std::atoi(line.c_str());
std::this_thread::sleep_for(std::chrono::milliseconds(150));
}
~TestServerProc() {
if (pid_ > 0) {
::kill(pid_, SIGTERM);
int status = 0;
for (int i = 0; i < 50; ++i) {
if (::waitpid(pid_, &status, WNOHANG) == pid_)
return;
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
::kill(pid_, SIGKILL);
::waitpid(pid_, &status, 0);
}
}
TestServerProc(const TestServerProc &) = delete;
TestServerProc &operator=(const TestServerProc &) = delete;
[[nodiscard]] bool available() const { return port_ > 0; }
[[nodiscard]] std::string url(const std::string &path) const {
return "http://127.0.0.1:" + std::to_string(port_) + path;
}
private:
pid_t pid_ = -1;
int port_ = 0;
};
} // namespace vdm::bench
#endif // VDM_BENCH_TESTSERVER_CLIENT_HPP