benchmarks/public_baseline.py's SOURCE_PATHS includes Cargo.toml, and the artifact's source_fingerprint is a hash over those paths. #6528 made _fingerprint_bytes strip perry's own version = "x.y.z" line before hashing, so a release bump no longer trips it — but it does nothing for dependency pins.
So any routine Cargo.toml churn (dependabot/taze bumps, adding a registry entry, a new dep) changes the fingerprint and fails "Public benchmark evidence freshness" on main, even when nothing about the benchmarked TS/JS suite changed.
Why this is worth fixing rather than living with
Clearing it is not a normal CI retry. ./benchmarks/run_public_baseline.sh requires a clean tree, Node exactly v22.23.1, Bun exactly 1.3.14, AC power, and a quiet CPU (≤25% for 60 consecutive seconds) — it monopolises the machine, by design, because it is a benchmark.
Observed cost of the current instance: the gate has been red on main since 2026-07-29. Because lint is a step sequence and a failing step skips the rest, that also silently skipped the eight later gate steps (File size limit, Binding upstream pins, GC store-site inventory, Address-classification audit, Gap snapshot self-test, Platform-aware parity allowlist self-test, Moving-GC gate wiring, GC matrix liveness gate) for days — in a required job that looked red for an unrelated reason. It is currently the sole blocker on #7278.
An attempt to regenerate it today was abandoned after ~55 minutes: no 60-second quiet window was reachable on a shared machine.
The concern
A gate that predictably goes red on unrelated merges, and whose only remedy is a hardware-gated manual ritual, is one that eventually gets admin-bypassed or ignored — which is the same hazard class the repo's own gate-design notes warn about.
Possible directions (not a prescription)
- Narrow
SOURCE_PATHS so Cargo.toml contributes only what actually affects benchmark output (e.g. the [profile.*] sections and perry's own source deps), rather than the whole file.
- Extend the
_fingerprint_bytes normalisation from perry's own version line to dependency version pins generally, keeping structural changes significant.
- Separate "the harness or suite changed, regenerate" from "an input moved, re-verify cheaply".
Related: #7278 (adds if: ${{ !cancelled() }} to its own lint step so it survives this skip), and separate in-flight work to guard or split the remaining lint steps.
benchmarks/public_baseline.py'sSOURCE_PATHSincludesCargo.toml, and the artifact'ssource_fingerprintis a hash over those paths. #6528 made_fingerprint_bytesstrip perry's ownversion = "x.y.z"line before hashing, so a release bump no longer trips it — but it does nothing for dependency pins.So any routine
Cargo.tomlchurn (dependabot/taze bumps, adding a registry entry, a new dep) changes the fingerprint and fails "Public benchmark evidence freshness" onmain, even when nothing about the benchmarked TS/JS suite changed.Why this is worth fixing rather than living with
Clearing it is not a normal CI retry.
./benchmarks/run_public_baseline.shrequires a clean tree, Node exactly v22.23.1, Bun exactly 1.3.14, AC power, and a quiet CPU (≤25% for 60 consecutive seconds) — it monopolises the machine, by design, because it is a benchmark.Observed cost of the current instance: the gate has been red on
mainsince 2026-07-29. Becauselintis a step sequence and a failing step skips the rest, that also silently skipped the eight later gate steps (File size limit, Binding upstream pins, GC store-site inventory, Address-classification audit, Gap snapshot self-test, Platform-aware parity allowlist self-test, Moving-GC gate wiring, GC matrix liveness gate) for days — in a required job that looked red for an unrelated reason. It is currently the sole blocker on #7278.An attempt to regenerate it today was abandoned after ~55 minutes: no 60-second quiet window was reachable on a shared machine.
The concern
A gate that predictably goes red on unrelated merges, and whose only remedy is a hardware-gated manual ritual, is one that eventually gets admin-bypassed or ignored — which is the same hazard class the repo's own gate-design notes warn about.
Possible directions (not a prescription)
SOURCE_PATHSsoCargo.tomlcontributes only what actually affects benchmark output (e.g. the[profile.*]sections and perry's own source deps), rather than the whole file._fingerprint_bytesnormalisation from perry's own version line to dependency version pins generally, keeping structural changes significant.Related: #7278 (adds
if: ${{ !cancelled() }}to its own lint step so it survives this skip), and separate in-flight work to guard or split the remaining lint steps.