Skip to content

chore(node-suite): stop committing derived overall aggregate - #6698

Merged
proggeramlug merged 1 commit into
mainfrom
chore/node-suite-drop-derived-overall
Jul 19, 2026
Merged

chore(node-suite): stop committing derived overall aggregate#6698
proggeramlug merged 1 commit into
mainfrom
chore/node-suite-drop-derived-overall

Conversation

@TheHypnoo

@TheHypnoo TheHypnoo commented Jul 19, 2026

Copy link
Copy Markdown
Member

Problem

test-parity/node_suite_baseline.json stored a top-level overall {pass,total,pct} block, but nothing reads it:

  • scripts/node_suite_regression_check.py reads only .get("modules", {}) — the gate is 100% per-module.
  • scripts/node_suite_run.py computes its own overall from the live run.
  • No CI workflow or doc parses the overall key.

Because every floor-bumping parity PR rewrote those same three overall lines, any two such PRs collided pairwise on that shared triple — currently blocking ~10 open parity PRs from rebasing cleanly.

The stored value had also drifted stale (2820/2919 vs the real module sum 2834/2959), confirming it as write-only bookkeeping nobody kept in sync.

Change

  • Remove the overall block from the baseline JSON (and drop the now-obsolete _schema.note sentence describing it).
  • Compute the overall from the per-module floors at report time in node_suite_regression_check.py, printed as baseline floor overall: <sum>/<sum> (<pct>%).

Per-module data — the exhaustive analysis — is untouched. The gate's pass/fail decision is unchanged (still per-module floors only). Each parity PR now edits only its own module block, so the conflict class is gone permanently.

Notes

  • No JSON-schema validation or test asserts the key's presence; removal is safe.
  • Version bump + CHANGELOG entry follow separately at merge time (maintainer convention).

Summary by CodeRabbit

  • New Features

    • Added an overall baseline floor summary showing aggregate passing and total test counts with the corresponding percentage.
  • Documentation

    • Updated baseline notes to reflect the revised summary description.

The baseline stored a top-level `overall` {pass,total,pct} block that no
code reads — the regression gate is 100% per-module (node_suite_regression_check.py
reads only `modules`). Every floor-bumping parity PR rewrote the same three
`overall` lines, so any two such PRs collided pairwise on that shared triple.

The stored value had also drifted stale (2820/2919 vs the real module sum
2834/2959), confirming it as write-only bookkeeping nobody maintained.

Remove the block and compute the overall from the per-module floors at report
time in node_suite_regression_check.py instead. Per-module data (the exhaustive
analysis) is untouched; the gate's pass/fail decision is unchanged.
@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b2e9c423-e4ce-43b5-adff-e837bf6a2d6e

📥 Commits

Reviewing files that changed from the base of the PR and between e7626a1 and 69a4fbe.

📒 Files selected for processing (2)
  • scripts/node_suite_regression_check.py
  • test-parity/node_suite_baseline.json

📝 Walkthrough

Walkthrough

The regression check now computes and prints an aggregate node suite baseline floor. The baseline JSON note no longer states that the overall summary equals the sum of per-module floors.

Changes

Node suite baseline

Layer / File(s) Summary
Aggregate baseline reporting
scripts/node_suite_regression_check.py, test-parity/node_suite_baseline.json
The check sums baseline pass and total values to print an overall floor, while the baseline note removes its previous aggregate-summary statement.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: proggeramlug

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description covers the motivation and change, but it does not use the required template sections like Summary, Changes, Related issue, Test plan, or Checklist. Rewrite it to match the template with Summary, Changes, Related issue, Test plan, and Checklist sections, plus optional screenshots/output if relevant.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: removing the derived overall aggregate from node-suite baseline handling.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/node-suite-drop-derived-overall

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants