feat(coverage): devkit-owned coverage gate — fail-hard on absent data, config-driven, opt-in - #138
Merged
Merged
Conversation
…, config-driven, opt-in
Lift the hand-rolled coverage check out of consumer .husky/pre-commit into a
first-class opt-in deterministic guard. Fail-CLOSED: coverage:false bypasses,
an absent/malformed coverage-final.json exits 1 (was a silent warn-and-pass),
a present artifact enforces the guard.config.json `coverage` thresholds
(statements/functions/lines/branches — only keys set are enforced). Default
{} = active-strict (no % floor, but absent data still fails hard).
- gate-engine/coverage/run.mts: the gate + istanbul/V8 aggregation
- gate-engine/config.mts: CoverageConfig type/field/default/resolver
- deterministic/run.mts: registered opt-in; DEFAULT_IDS excludes opt-in guards
from the missing-config fallback so an unadopted/CI repo is never wedged
- ship: prepare_gate_worktree links coverage/ into the worktree so the gate
verifies real coverage instead of being warned away (the ship-worktree bug)
- components/package.json/guard.config.example.json: registry, bin, docs
- docs/decisions/coverage-gate.md
Contributor
|
Warning Review limit reached
Next review available in: 21 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughChangesThe PR adds an opt-in coverage guard with fail-closed artifact validation, configurable thresholds, deterministic execution, CLI selection, coverage artifact linking in ship worktrees, documentation, and a public Coverage guard
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Developer
participant runDeterministic
participant runCoverage
participant CoverageArtifact
Developer->>runDeterministic: select coverage guard
runDeterministic->>runCoverage: execute coverage gate
runCoverage->>CoverageArtifact: read coverage/coverage-final.json
CoverageArtifact-->>runCoverage: coverage data or error
runCoverage-->>Developer: pass or fail exit status
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.github.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@gate-engine/coverage/run.mts`:
- Around line 1-21: The new gate-engine/coverage directory is rejected because
coverage is not an allowed root domain. Register coverage in devkit’s own
gate-engine structure configuration, preserving the existing folder location and
ensuring the folder-structure validation allowlist includes it.
- Around line 39-76: Validate the parsed coverage value and each file’s nested
coverage fields before or within computePercentages, rejecting null, arrays,
non-object entries, and non-array branch counters with an error. Ensure this
validation is handled by the existing catch around JSON parsing/processing so
malformed-but-parseable input produces the documented “not valid JSON”-style
message and exits through the established fail-closed path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 4557f521-61b5-4bbf-8fb0-9f5ff7b3fb8d
📒 Files selected for processing (11)
cli/__tests__/components-new-gates.test.mtscli/lib/components.mtscli/lib/ship/prepare-gate-worktree.shdocs/decisions/coverage-gate.mdgate-engine/config.mtsgate-engine/coverage/__tests__/run.test.mtsgate-engine/coverage/run.mtsgate-engine/deterministic/__tests__/run.test.mtsgate-engine/deterministic/run.mtsguard.config.example.jsonpackage.json
…r the coverage structure domain - computePercentages now validates shape and THROWS on parseable-but-malformed input (non-object root, null/array/non-object entry, non-array branch counter); runCoverage routes it through the existing catch, so malformed data fails CLOSED with a clean message + exit 1 instead of crashing on a TypeError or reading a garbage entry as 100% coverage. - register `coverage` in guard.config.json structure libDomains["@root"] — the only gate-engine domain that was missing, so the walk-based structure gate accepts it.
norvalbv
added a commit
that referenced
this pull request
Jul 19, 2026
…seline Repair scope violation flagged in human review: this PR (review profile and review-mode gate routing) had picked up an unrelated size-debt "fix" that deleted research-rationale comments. - gate-engine/decisions/detect.mts: restores the research-rationale doc comments (arXiv:2402.14848, arXiv:2302.00093, arXiv:2407.16833 citations, the --gate/scan exit-code contract, the W-3 portability-invariant note) that were deleted to dodge an unrelated size-debt gate. No logic changed either direction - comment-only. - eslint/baselines/size-lines.json: re-freezes detect.mts at its true 550 lines (main itself was already 25 lines over its own recorded 525 ceiling for this file before this PR touched it - a pre-existing stale baseline, not new debt from this change). docs/benchmarks/catalog.json / README.md are kept as-is (not reverted): removing that guard-coverage catalog entry re-exposes a genuine pre-existing benchmarks:check failure inherited from an earlier, unrelated PR (#138) that added the guard-coverage bin without a catalog entry. That one-line backfill is a real, separate bug fix bundled here only because it currently blocks every commit, not scope creep from this PR's stated purpose.
norvalbv
added a commit
that referenced
this pull request
Jul 22, 2026
…seline Repair scope violation flagged in human review: this PR (review profile and review-mode gate routing) had picked up an unrelated size-debt "fix" that deleted research-rationale comments. - gate-engine/decisions/detect.mts: restores the research-rationale doc comments (arXiv:2402.14848, arXiv:2302.00093, arXiv:2407.16833 citations, the --gate/scan exit-code contract, the W-3 portability-invariant note) that were deleted to dodge an unrelated size-debt gate. No logic changed either direction - comment-only. - eslint/baselines/size-lines.json: re-freezes detect.mts at its true 550 lines (main itself was already 25 lines over its own recorded 525 ceiling for this file before this PR touched it - a pre-existing stale baseline, not new debt from this change). docs/benchmarks/catalog.json / README.md are kept as-is (not reverted): removing that guard-coverage catalog entry re-exposes a genuine pre-existing benchmarks:check failure inherited from an earlier, unrelated PR (#138) that added the guard-coverage bin without a catalog entry. That one-line backfill is a real, separate bug fix bundled here only because it currently blocks every commit, not scope creep from this PR's stated purpose.
norvalbv
added a commit
that referenced
this pull request
Jul 24, 2026
…, config-driven, opt-in (#138) * feat(coverage): devkit-owned coverage gate — fail-hard on absent data, config-driven, opt-in Lift the hand-rolled coverage check out of consumer .husky/pre-commit into a first-class opt-in deterministic guard. Fail-CLOSED: coverage:false bypasses, an absent/malformed coverage-final.json exits 1 (was a silent warn-and-pass), a present artifact enforces the guard.config.json `coverage` thresholds (statements/functions/lines/branches — only keys set are enforced). Default {} = active-strict (no % floor, but absent data still fails hard). - gate-engine/coverage/run.mts: the gate + istanbul/V8 aggregation - gate-engine/config.mts: CoverageConfig type/field/default/resolver - deterministic/run.mts: registered opt-in; DEFAULT_IDS excludes opt-in guards from the missing-config fallback so an unadopted/CI repo is never wedged - ship: prepare_gate_worktree links coverage/ into the worktree so the gate verifies real coverage instead of being warned away (the ship-worktree bug) - components/package.json/guard.config.example.json: registry, bin, docs - docs/decisions/coverage-gate.md * fix(coverage): fail closed on malformed coverage-final.json + register the coverage structure domain - computePercentages now validates shape and THROWS on parseable-but-malformed input (non-object root, null/array/non-object entry, non-array branch counter); runCoverage routes it through the existing catch, so malformed data fails CLOSED with a clean message + exit 1 instead of crashing on a TypeError or reading a garbage entry as 100% coverage. - register `coverage` in guard.config.json structure libDomains["@root"] — the only gate-engine domain that was missing, so the walk-based structure gate accepts it.
norvalbv
added a commit
that referenced
this pull request
Jul 24, 2026
…seline Repair scope violation flagged in human review: this PR (review profile and review-mode gate routing) had picked up an unrelated size-debt "fix" that deleted research-rationale comments. - gate-engine/decisions/detect.mts: restores the research-rationale doc comments (arXiv:2402.14848, arXiv:2302.00093, arXiv:2407.16833 citations, the --gate/scan exit-code contract, the W-3 portability-invariant note) that were deleted to dodge an unrelated size-debt gate. No logic changed either direction - comment-only. - eslint/baselines/size-lines.json: re-freezes detect.mts at its true 550 lines (main itself was already 25 lines over its own recorded 525 ceiling for this file before this PR touched it - a pre-existing stale baseline, not new debt from this change). docs/benchmarks/catalog.json / README.md are kept as-is (not reverted): removing that guard-coverage catalog entry re-exposes a genuine pre-existing benchmarks:check failure inherited from an earlier, unrelated PR (#138) that added the guard-coverage bin without a catalog entry. That one-line backfill is a real, separate bug fix bundled here only because it currently blocks every commit, not scope creep from this PR's stated purpose.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Lifts the coverage check out of a consumer's hand-rolled
.husky/pre-commitand makes it a first-class devkit-owned, opt-in deterministic guard that is fail-closed on absent data.gate-engine/coverage/run.mts— the gate. Readscoverage/coverage-final.json(istanbul/V8) and aggregates statements/functions/branches/lines.gate-engine/config.mts—CoverageConfigtype +coveragefield/default/resolver.gate-engine/deterministic/run.mts— registerscoverageas an opt-in guard; addsDEFAULT_IDSso opt-in guards are excluded from the missing-config fallback.cli/lib/ship/prepare-gate-worktree.sh— linkscoverage/into the gate worktree so the gate verifies real coverage.cli/lib/components.mts,package.json,guard.config.example.json— registry entry,guard-coveragebin, config docs.docs/decisions/coverage-gate.md— the decision record.Behaviour (
guard.config.jsoncoverage){})test:run:coveragefirst (was a silent warn-and-pass)false{ statements, functions, lines?, branches? }Exit contract matches the deterministic trichotomy (0 pass/bypass, 1 fail); coverage deliberately has no fail-open (2) arm — that was the defect being removed.
Why
The old hand-rolled block warned-and-passed when
coverage/coverage-final.jsonwas absent, so substantive commits could ship with coverage silently unverified. It was fail-open on purpose becausedevkit ship's ephemeral worktree never carried the gitignoredcoverage/artifact. This fixes that at the root:prepare_gate_worktreenow linkscoverage/into the worktree (existence-guarded — absent → not linked → the gate fails hard), so the gate verifies real coverage instead of being warned away.Opt-in / no surprise
coverageis inGUARD_IDSbut notRECOMMENDED_GUARD_IDS(it needs atest:run:coverageprovider). It runs only when a repo selects it;DEFAULT_IDSkeeps it out of the missing-config fallback so an unadopted/CI repo without.devkit/config.jsonis never wedged.devkit upgradesurfaces it as an opt-in via the existingnewBundledGatespath (likereview/sentry).Tests
New
gate-engine/coverage/__tests__/run.test.mts(bypass / absent→1 / malformed→1 / thresholds / per-key enforcement / istanbul aggregation) + deterministic opt-in wiring + config resolution + registry assertions.tsc, biome, and the full suite pass.Follow-up (not in this PR)
Migrate frink onto this guard once released: bump devkitRef →
devkit upgrade(accept coverage) → delete frink's hand-rolled block → setcoverage: { statements: 62, functions: 59 }in frink's guard.config.json.Summary by CodeRabbit
New Features
guard-coveragecommand for running coverage checks.Documentation
Tests