Skip to content

ci: auto-label PRs "Work in progress" from review state - #10294

Open
igorpecovnik wants to merge 1 commit into
mainfrom
ci-wip-label-on-review
Open

ci: auto-label PRs "Work in progress" from review state#10294
igorpecovnik wants to merge 1 commit into
mainfrom
ci-wip-label-on-review

Conversation

@igorpecovnik

@igorpecovnik igorpecovnik commented Jul 27, 2026

Copy link
Copy Markdown
Member

Keeps the Work in progress label in sync with a PR's review state: present while a review requests changes or any conversation is unresolved, cleared once neither holds.

Why hybrid

Two hard GitHub constraints rule out a single pull_request_review-triggered workflow (thanks @coderabbitai for catching both):

  1. No trigger for conversation resolution. pull_request_review_thread is a webhook event but not a valid Actions trigger — actionlint rejects it. So nothing fires when a thread is resolved/unresolved.
  2. Fork PRs get a read-only GITHUB_TOKEN on pull_request_review, so direct addLabels/removeLabel would silently fail on any fork PR.

So this uses an event path (fast, for the common cases) plus a cron backstop (for the case that has no event):

File Role
.github/scripts/wip-label.js Shared logic — reviewDecision + paginated unresolved reviewThreads → add/remove label. Single source of truth so the two paths can't drift.
maintenance-listen-review-wip.yml Listens on pull_request_review with a read-only token (fork-safe); only records the PR number as an artifact.
maintenance-label-wip.yml workflow_run worker fired by the listener — runs in the default-branch context with a write token, checks out the trusted script, syncs that one PR.
maintenance-label-wip-sweep.yml Scheduled backstop (*/30), recomputes the label for every open PR — catches conversation resolution and any missed events.

Work in progress = reviewDecision == CHANGES_REQUESTED OR any review conversation unresolved.

How this addresses the review

  • Invalid pull_request_review_thread trigger → removed entirely; conversation-resolution is handled by the cron sweep instead.
  • Fork read-only token → the fork event only records the PR number (read-only, safe); all label writes happen in the workflow_run/schedule context, which has a write token even for fork PRs. The worker checks out the default branch, so the label logic is never PR-authored code.
  • Concurrency → the worker uses a per-PR concurrency group (keyed on the reviewed head repo + branch) with cancel-in-progress; the sweep serialises with its own group.

Also

  • Dropped Work in progress from the blanket removal list in maintenance-label-on-approval.yml (now owned by this automation, so an approved-but-unresolved PR keeps the label until resolved), and quoted $GITHUB_OUTPUT there (shellcheck SC2086).
  • Work in progress label already exists in .github/labels.yml.
  • Validated locally with actionlint — clean.

Trade-off / tunables

The event path is instant for "changes requested" and re-reviews. The one case with no event — resolving the last conversation without submitting a review — is picked up by the sweep within its cron interval (*/30, tunable). Set it looser to reduce Actions runs, tighter for faster resolution latency.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The pull request adds review-event and scheduled workflows that synchronize the “Work in progress” label using review decisions and unresolved threads. Approval cleanup no longer removes that label, and the shared script supports both single-PR and all-open-PR synchronization.

Changes

Work-in-progress label synchronization

Layer / File(s) Summary
WIP evaluation and label reconciliation
.github/scripts/wip-label.js
Evaluates review decisions and unresolved threads, then adds or removes the “Work in progress” label for individual or all open PRs.
Review event synchronization pipeline
.github/workflows/maintenance-listen-review-wip.yml, .github/workflows/maintenance-label-wip.yml
Passes reviewed PR numbers through artifacts and invokes single-PR label synchronization after successful review-event processing.
Scheduled sweep and label ownership
.github/workflows/maintenance-label-wip-sweep.yml, .github/workflows/maintenance-label-on-approval.yml
Periodically or manually synchronizes all open PRs while approval cleanup excludes the WIP label and safely quotes the GitHub output path.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PullRequestReview
  participant ReviewListener
  participant WipWorkflow
  participant WipLabelScript
  participant GitHubGraphQL
  participant PullRequestLabels
  PullRequestReview->>ReviewListener: Submit, edit, or dismiss review
  ReviewListener->>WipWorkflow: Upload PR number artifact
  WipWorkflow->>WipLabelScript: Call syncOne for the reviewed PR
  WipLabelScript->>GitHubGraphQL: Query review decision and unresolved threads
  GitHubGraphQL-->>WipLabelScript: Return review state and thread count
  WipLabelScript->>PullRequestLabels: Add or remove Work in progress label
Loading

Suggested labels: Framework

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: automatically syncing the "Work in progress" label from PR review state.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci-wip-label-on-review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size/medium PR with more then 50 and less then 250 lines 08 Milestone: Third quarter release Needs review Seeking for review GitHub Actions GitHub Actions code labels Jul 27, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/maintenance-label-wip.yml:
- Around line 17-18: Replace the unsupported pull_request_review_thread trigger
in the workflow’s on configuration with a supported GitHub Actions trigger, or
remove the workflow if no suitable trigger is needed. If synchronization for
resolved and unresolved review threads must remain, use a cron/polling trigger
or external dispatch mechanism.
- Line 38: Update the workflow’s pull_request_review handling to address
fork-origin PRs: either authenticate label operations with a trusted
write-capable identity instead of the read-only fork-event GITHUB_TOKEN, or add
an explicit repository/branch condition that excludes fork PRs before label
add/remove steps run.
- Around line 25-30: Update the wip-label job to define a concurrency group
scoped to the pull request number and enable cancellation of superseded runs,
ensuring only the latest workflow run for each PR performs WIP label
synchronization.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9d737439-5eab-400a-8057-afdda66cf24e

📥 Commits

Reviewing files that changed from the base of the PR and between 4451999 and 34a447d.

📒 Files selected for processing (2)
  • .github/workflows/maintenance-label-on-approval.yml
  • .github/workflows/maintenance-label-wip.yml

Comment thread .github/workflows/maintenance-label-wip.yml Outdated
Comment thread .github/workflows/maintenance-label-wip.yml Outdated
Comment thread .github/workflows/maintenance-label-wip.yml
@igorpecovnik
igorpecovnik force-pushed the ci-wip-label-on-review branch from 34a447d to 2581851 Compare July 27, 2026 08:14
@github-actions github-actions Bot added the GitHub GitHub-related changes like labels, templates, ... label Jul 27, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
.github/workflows/maintenance-label-wip-sweep.yml (3)

25-28: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Reduce pull-request permission to read-only.

The helper reads review state and manages PR labels through Issues endpoints; it does not modify pull requests. Use pull-requests: read and verify the reviewThreads query still succeeds. GitHub exposes label management through Issues endpoints and review APIs with read-level pull-request access. (docs.github.com)

Proposed fix
       issues: write
-      pull-requests: write
+      pull-requests: read
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/maintenance-label-wip-sweep.yml around lines 25 - 28,
Update the permissions block in the maintenance-label WIP sweep workflow to
change pull-requests access from write to read, while retaining contents read
and issues write. Verify the existing reviewThreads query continues to succeed
with the reduced permission.

31-31: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Disable checkout credential persistence.

The token is persisted for Git commands by default, but this job already passes it explicitly to github-script. Prevent checked-out code from accessing unnecessary Git credentials:

Proposed fix
       - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/maintenance-label-wip-sweep.yml at line 31, Update the
actions/checkout step in the maintenance-label workflow to disable credential
persistence by setting persist-credentials to false; leave the existing checkout
reference and github-script token handling unchanged.

Source: Linters/SAST tools


33-39: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Surface per-PR synchronization failures.

syncAllOpen currently warns and continues when an individual PR fails, so this step can be green while labels remain stale. Configure retries and have the helper aggregate failed PRs, then call core.setFailed after the sweep completes. github-script otherwise performs no request retries by default. (github.com)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/maintenance-label-wip-sweep.yml around lines 33 - 39,
Update the “Sync Work in progress” step invoking syncAllOpen to configure
github-script request retries. Modify syncAllOpen to aggregate individual PR
synchronization failures while completing the sweep, then have the workflow call
core.setFailed with the aggregated failure details after syncAllOpen returns.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/maintenance-label-wip-sweep.yml:
- Around line 3-8: Update the workflow’s actions/checkout configuration to
explicitly use the repository’s default branch ref for manual workflow_dispatch
runs, ensuring the sweep always executes the default-branch version of
.github/scripts/wip-label.js. Leave the scheduled trigger behavior and remaining
workflow logic unchanged.

In @.github/workflows/maintenance-label-wip.yml:
- Around line 34-54: Change the artifact handling before the “Sync "Work in
progress" for the PR” step to download into $RUNNER_TEMP rather than the trusted
workspace. Update the pr step to read only the expected $RUNNER_TEMP/pr.txt
artifact, validating the PR number before exporting it, so the checkout’s
.github/scripts/wip-label.js cannot be overwritten by PR-origin content.

---

Nitpick comments:
In @.github/workflows/maintenance-label-wip-sweep.yml:
- Around line 25-28: Update the permissions block in the maintenance-label WIP
sweep workflow to change pull-requests access from write to read, while
retaining contents read and issues write. Verify the existing reviewThreads
query continues to succeed with the reduced permission.
- Line 31: Update the actions/checkout step in the maintenance-label workflow to
disable credential persistence by setting persist-credentials to false; leave
the existing checkout reference and github-script token handling unchanged.
- Around line 33-39: Update the “Sync Work in progress” step invoking
syncAllOpen to configure github-script request retries. Modify syncAllOpen to
aggregate individual PR synchronization failures while completing the sweep,
then have the workflow call core.setFailed with the aggregated failure details
after syncAllOpen returns.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 672f8d88-e913-4727-95d2-56ccb0ed63f3

📥 Commits

Reviewing files that changed from the base of the PR and between 34a447d and 2581851.

📒 Files selected for processing (5)
  • .github/scripts/wip-label.js
  • .github/workflows/maintenance-label-on-approval.yml
  • .github/workflows/maintenance-label-wip-sweep.yml
  • .github/workflows/maintenance-label-wip.yml
  • .github/workflows/maintenance-listen-review-wip.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/maintenance-label-on-approval.yml

Comment thread .github/workflows/maintenance-label-wip-sweep.yml
Comment thread .github/workflows/maintenance-label-wip.yml
Adds automation that keeps the "Work in progress" label in sync with a PR's
review state: present while a review requests changes or any conversation is
unresolved, cleared once neither holds.

GitHub Actions has no trigger for a review conversation being resolved, and
pull_request_review from a fork gets a read-only GITHUB_TOKEN, so a single
pull_request_review-triggered workflow cannot do this. Use a hybrid instead:

- .github/scripts/wip-label.js -- shared logic (reviewDecision + paginated
  unresolved reviewThreads -> add/remove the label), so the event and cron
  paths can never drift.
- maintenance-listen-review-wip.yml -- listens on pull_request_review with a
  read-only token (fork-safe) and just records the PR number as an artifact.
- maintenance-label-wip.yml -- workflow_run worker fired by the listener; runs
  in the default-branch context with a write token, checks out the trusted
  script and syncs the one PR. Per-PR concurrency, newer reviews supersede.
- maintenance-label-wip-sweep.yml -- scheduled backstop that recomputes the
  label for every open PR, catching conversation resolution (which has no
  event) and any missed events.

Hardening: the listener artifact is fork-controlled, so the worker downloads it
into RUNNER_TEMP (never the checkout) so it cannot overwrite the trusted script
before github-script runs it; the sweep pins checkout to the default branch so a
workflow_dispatch on another branch cannot run that branch's script; both set
persist-credentials: false.

Also drop "Work in progress" from the blanket removal list in
maintenance-label-on-approval.yml (now owned by this automation) and quote
$GITHUB_OUTPUT there (shellcheck SC2086).

Signed-off-by: Igor Pecovnik <igor@armbian.com>
@igorpecovnik
igorpecovnik force-pushed the ci-wip-label-on-review branch from 2581851 to 0b00307 Compare July 27, 2026 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

08 Milestone: Third quarter release GitHub Actions GitHub Actions code GitHub GitHub-related changes like labels, templates, ... Needs review Seeking for review size/medium PR with more then 50 and less then 250 lines

Development

Successfully merging this pull request may close these issues.

1 participant