feat(workflows): overhaul PR review workflows around /review - #1758
Merged
Conversation
|
Azure Pipelines: 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Replaces the two near-duplicate Rust reviewers with a five-way `/review` fan-out modelled on githubnext/gh-aw, and adds a sous-chef loop that keeps open PRs moving. The old pair (`rust-pr-reviewer.md` on every push, `rust-review-command.md` on `/rust-review`) shared ~95% of their prompt, covered only Rust, and posted summary comments. Reviews are now inline line comments. New reviewers, each owning a distinct concern so comments do not overlap: - review-rust Rust engineering quality - review-typescript scripts/ado-script/ quality - review-tests test quality beyond coverage - review-compiler-contract ado-aw domain contracts and drift - review-security diff-scoped security regressions `review-rust` and `review-typescript` are deliberately language-generic; all ado-aw-specific invariants live in `review-compiler-contract`, which also detects ncc bundle drift — a gap nothing else in the repo covers, since ado-script.yml rebuilds the bundles but never diffs them. Only the two code-quality reviewers re-run on every push. All five carry `draft: false` so pushes to drafts cost nothing, and gh-aw's generated per-PR `cancel-in-progress` concurrency supersedes runs on rapid pushes. Shared components: - shared/pr-review-base.md tools, network and review safe-outputs - shared/pr-diff-data-fetch.md pre-agent-steps diff/meta/comment prefetch - pr-data-prefetch.yml engine-less cache warmer `submit-pull-request-review` pins `allowed-events: [COMMENT, REQUEST_CHANGES]` in the shared base: the Actions actor cannot approve a PR, so permitting it would only fail at runtime. `supersede-older-reviews` keeps repeated pushes from stacking stale REQUEST_CHANGES reviews, and the prefetched `pr-review-comments.json` lets each reviewer skip feedback it already gave. `pr-sous-chef.md` runs every 15 minutes and on `/souschef`, posting one marker'd `@copilot` nudge per PR listing unresolved threads and failed checks, resolving answered threads, dismissing stale bot reviews and pushing `cargo fmt --all` fixes. It does not rebuild the TypeScript bundles — that needs `npm ci` plus two `cargo run` invocations, far too costly at 96 runs/day — so bundle drift is reported by review-compiler-contract instead. Command changes: - `/rust-review` is replaced by `/review` - `/change-risk` is renamed to `/risk` (change-risk.md -> risk.md) - `/scout`, `/plan` and `/risk` move to `strategy: centralized` All four commands now route through the generated `agentic_commands.yml`, resolving the advisory `gh aw validate` was emitting. `gh aw compile` reports 27 workflows, 0 errors; every added bash body passes shellcheck. agentics-maintenance.yml is a v0.84.0 regeneration artefact. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 475bd62b-12cb-41aa-88b6-39a3d75e3c50
jamesadevine
force-pushed
the
feat/ai-reviewer-overhaul
branch
from
July 31, 2026 19:52
da535b5 to
b910f81
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Stacked on #1757 (the gh-aw v0.84.0 upgrade). Targets
chore/upgrade-gh-aw-v0.84.0, so this diff shows only the review-workflow changes. Merge #1757 first; this will retarget tomainautomatically.Replaces the two near-duplicate Rust reviewers with a five-way
/reviewfan-out modelled on githubnext/gh-aw, and adds a sous-chef loop that keeps open PRs moving.The problem
rust-pr-reviewer.md(every push) andrust-review-command.md(/rust-review) shared ~95% of their prompt, covered only Rust, ignored the TypeScript workspace entirely, and posted summary comments rather than line comments. Nothing closed the loop back to Copilot when a PR stalled.What gh-aw actually does
/reviewthere is not one workflow — it is a fan-out dispatched by a generated central router, hittingpr-code-quality-reviewer,test-quality-sentinelanddesign-decision-gate. Reviewers post inline comments batched into a single review, and dedupe across pushes via a shared pre-fetch of the diff and the existing review comments, cached by head SHA.The five reviewers
review-rustanyhowcontext,unwrapon user paths, lossy casts, cross-platform paths, async correctnessreview-typescriptscripts/ado-script/— unhandled rejections,anyleakage, unvalidated external input, secret handlingreview-testsreview-compiler-contractreview-securityAll five answer
/review. Only the two code-quality reviewers re-run per push, which is the main cost control.review-rustandreview-typescriptare deliberately language-generic; every ado-aw-specific invariant lives inreview-compiler-contract. That keeps the five from duplicating each other's comments, and gives one obvious home for future domain rules.A CI gap this closes
review-compiler-contractdetects ncc bundle drift — a change toscripts/ado-script/src/**without the rebuilt*.js. Those bundles are what actually execute inside customer pipelines, so a source change without a rebuild is a no-op at runtime.ado-script.ymlrebuilds the bundles but never diffs them, so nothing in the repo currently catches this. Its drift guard only coverstypes.gen.tsandfact-catalog.gen.json.Anti-spam, since two reviewers run on every push
draft: falseon all five — gh-aw's first-class filter, so pushes to drafts cost nothing.supersede-older-reviews— stops repeated pushes stacking staleREQUEST_CHANGESreviews.pr-review-comments.json— each reviewer reads what it already said and skips it.cancel-in-progressconcurrency, already generated by gh-aw, supersedes in-flight runs.pr-data-prefetch.yml— an engine-less warmer that populates thepr-prefetch-<sha>cache in ~30-60s, so all five get a cache hit instead of five identical API pulls per push.submit-pull-request-reviewpinsallowed-events: [COMMENT, REQUEST_CHANGES]in the shared base: the Actions actor cannot approve a PR, so permitting it would only fail at runtime.PR Sous Chef
Runs every 15 minutes and on
/souschef. Per PR it posts one@copilotnudge carrying a hidden marker, listing unresolved review threads and failed checks, then resolves answered threads, dismisses stale bot reviews, refreshes the branch and pushescargo fmt --allfixes.Guards against nagging: 30-minute marker cooldown; skip when the latest comment is already ours (overridden when the branch is
CONFLICTING, since nobody else will resolve it); skip while checks run, with checks running over an hour treated as stale so long agentic jobs cannot block nudges forever; cap of four nudges per run. A marker comment that does not mention@copilotis informational and counts toward neither rule.It deliberately does not rebuild the TypeScript bundles — that needs
npm ciplus twocargo runinvocations, far too costly at 96 runs/day. Bundle drift is reported byreview-compiler-contractinstead.Command changes
/rust-review/review/change-risk/risk(change-risk.md→risk.md)/scout,/planand/riskalso move tostrategy: centralized, so all commands now route through the generatedagentic_commands.yml. This resolves the advisorygh aw validatewas emitting.Router now reads:
Verification
gh aw compile— 27 workflows, 0 errors.gh aw validate— clean; centralized-routing advisory resolved./rust-reviewor/change-riskreferences outsideCHANGELOG.md.main: byte-identical output, so no drift.Two gotchas, now documented in
AGENTS.mdBoth cost a compile cycle and will bite the next person:
permissions:cannot be inherited from an import. Every workflow must declare its own, even when importing a shared base.${{ }}in a markdown body is parsed as an expression and fails compilation with "unauthorized expressions". Describe the syntax in prose instead.AGENTS.mdalso gains a "Repository agentic workflows" section documenting all five commands, the fan-out, the shared components and the sous chef.