samiandClaude Sonnet 5
f2ee45f818
Per-task engine state, downloaded/effective_segments, every segment's own state, and the engine's SegmentBudget snapshot -- printed once, when a task times out, instead of needing to re-run the bench under a debugger or add throwaway instrumentation to find out why. This is what actually diagnosed the SegmentBudget over-admission bug fixed in the previous commit: the dump showed budget.active pinned at max_active_segments while multiple tasks sat starved, which is what pointed straight at confirm_slot()'s missing engine-wide check rather than a per-task target bug. Also updates the vdm_bench_load20 ctest registration's comment: the straggler under --preset tsan that motivated running it at reduced concurrency (docs/adr/0016's postscript) is now suspected to have been the same SegmentBudget bug (docs/adr/0017), not the TSan-timing artifact first guessed -- not reverified at the DoD's full shape under TSan in this change, so the reduced-concurrency registration stays for now. Co-Authored-By: Claude Sonnet 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Q3QrF7rCt21bkAjt9BCDFQ
58 lines
3.5 KiB
CMake
58 lines
3.5 KiB
CMake
# tools/bench — the M1/M7 performance gates (docs/04-engine-design.md §8) and the
|
|
# sanitizer-clean 20-task load test. Lane CORE owns tools/bench.
|
|
#
|
|
# Self-guarding like every tools/* dir: the top-level CMakeLists.txt add_subdirectory()s
|
|
# this unconditionally, so it must opt out on its own if core/ hasn't landed yet.
|
|
|
|
if(NOT TARGET velox::core)
|
|
return()
|
|
endif()
|
|
|
|
add_executable(vdm_bench vdm_bench.cpp)
|
|
target_include_directories(vdm_bench PRIVATE ${CMAKE_CURRENT_SOURCE_DIR})
|
|
target_link_libraries(vdm_bench PRIVATE velox::core Threads::Threads)
|
|
|
|
# alloc-check paces its sampling window via tools/testserver's `throttled` mode (see
|
|
# support/testserver_client.hpp for why: the engine's own rate limiter isn't a clean
|
|
# substitute -- its pause/resume path is itself allocating, which would contaminate the
|
|
# very thing being measured). Same conditional-define pattern as core/tests/CMakeLists.txt.
|
|
set(_testserver ${CMAKE_SOURCE_DIR}/tools/testserver/testserver.py)
|
|
if(EXISTS ${_testserver})
|
|
target_compile_definitions(vdm_bench PRIVATE VDM_TESTSERVER_PY="${_testserver}")
|
|
endif()
|
|
|
|
# Regression tripwires only, mirroring tools/fuzz's smoke/campaign split: these must pass
|
|
# under every preset including dev/tsan, so they assert correctness (every task completes,
|
|
# no sanitizer error) and nothing about absolute throughput/CPU/RSS, which only mean what
|
|
# the DoD numbers say under --preset release on real (or at least unshared) hardware. The
|
|
# actual M1/M7 sign-off is a manual/CI perf job:
|
|
#
|
|
# cmake --preset release && cmake --build --preset release
|
|
# bin/vdm_bench throughput --size 5G --require-mbps 940 --max-cpu-pct 8 # against a
|
|
# # real 1 Gbit peer
|
|
# bin/vdm_bench load --tasks 20 --require-rss-kb 61440
|
|
# bin/vdm_bench alloc-check --size 512M
|
|
if(VELOX_BUILD_TESTS)
|
|
add_test(NAME vdm_bench_throughput_smoke COMMAND vdm_bench throughput --size 32M)
|
|
set_tests_properties(vdm_bench_throughput_smoke PROPERTIES LABELS "bench" TIMEOUT 120)
|
|
|
|
# --tasks 8 --segments 2 (not the DoD's 20 tasks * default_segments=8 = 160 concurrent
|
|
# segments): at the full shape under --preset tsan, a straggler task was observed not
|
|
# just slow but still incomplete past a 300s-per-task budget (docs/adr/0016's
|
|
# postscript, written before docs/adr/0017's SegmentBudget fix landed) -- now suspected
|
|
# to have been that same over-admission bug (a segment denied a slot with nothing to
|
|
# wake it), not a TSan-timing artifact as first guessed, since the symptom -- a task
|
|
# that simply never resumes -- matches exactly. Left at this reduced concurrency rather
|
|
# than reverting: re-verifying the full shape is clean under TSan wasn't done as part
|
|
# of that fix (see docs/adr/0017's "Consequences"), so this is still the bar that's
|
|
# actually known to hold. "Every task completes" is what this smoke test exists to
|
|
# check (see the split above); the DoD's real 20-task/default-segments/60MB-RSS shape
|
|
# is exercised by the manual/CI M7 sign-off run in this file's header comment, at
|
|
# --preset release, where it passes (core/docs/m7-baseline.md).
|
|
add_test(NAME vdm_bench_load20 COMMAND vdm_bench load --tasks 8 --task-size 2M --segments 2)
|
|
set_tests_properties(vdm_bench_load20 PROPERTIES LABELS "bench" TIMEOUT 300)
|
|
|
|
add_test(NAME vdm_bench_alloc_check COMMAND vdm_bench alloc-check --size 64M --window-s 1)
|
|
set_tests_properties(vdm_bench_alloc_check PROPERTIES LABELS "bench" TIMEOUT 60)
|
|
endif()
|