Skip to content

Add reusable codex-review workflow - #43

Open
brenthoneybone wants to merge 18 commits into
mainfrom
ID-114-create-reusable-codex-review-github-action
Open

Add reusable codex-review workflow#43
brenthoneybone wants to merge 18 commits into
mainfrom
ID-114-create-reusable-codex-review-github-action

Conversation

@brenthoneybone

@brenthoneybone brenthoneybone commented Jul 8, 2026

Copy link
Copy Markdown

Description

Reusable github workflow for running a codex review against a PR inside plugin repos when adding the codex-review tag

Trialed on AIBrandInsights repo

Checklist

  • [✔] I have understood, reviewed, and tested all AI outputs before use
  • [✔] All AI instructions respect security, IP, and privacy rules

Review

The PR checkout uses persist-credentials: false so the token is not left
in pr/.git/config for Codex to read. The follow-up `git fetch origin` then
had no credentials and failed non-interactively against the private repo
("could not read Username for 'https://github.com'"). Supply auth for that
single fetch via an in-memory `git -c http.extraheader`, which is never
written to .git/config, mirroring how actions/checkout authenticates.
Expand the README Security Model section into a threat-model-based
explanation: PR content is treated as untrusted, and the OPENAI_API_KEY
and GITHUB_TOKEN are kept from the agent by several independent layers
(trigger gating, trusted-code separation, least-privilege token,
persist-credentials: false plus in-memory fetch auth, env stripping,
read-only sandbox, prompt-injection policy, and review/posting separation).
formatSeverityCounts had no callers and was not exported. Drop it.
@brenthoneybone
brenthoneybone force-pushed the ID-114-create-reusable-codex-review-github-action branch from 49167cc to 46c99cc Compare July 9, 2026 16:07
Pass an explicit model (default gpt-5.5) and reasoning effort (default
xhigh) to openai/codex-action so runs no longer fall through to the
action's unavailable default model. Both are optional workflow_call
inputs and are documented in review/README.md.
- Always symlink the PR checkout into the core plugin path, replacing any
  core-bundled copy, so Codex reviews the PR and not stale core code.
- Tolerate a malformed PR plugin.json instead of aborting the codex job under
  set -e.
- Only run the review jobs on the 'labeled' action, so removing the trigger
  label can no longer start a review.
The CODEX_REVIEW_MARKER drift guard compares post-review.js against
codex-review.yml, so a change to the workflow's copy of the marker must
trigger the test job.
- unplaced_findings path/line are optional (nullable), matching validateReview.
- Reject whitespace-only strings via a \\S pattern, matching the validator's
  trim check, so both layers agree on what is valid.
- parsePatchLines no longer skips diff lines whose source begins with ++/--;
  that previously desynchronised inline-comment line numbers.
- highest_severity is recomputed from the severest actual finding, so a
  blocking inline comment is never posted as a non-blocking COMMENT.
- Create the new review before dismissing previous ones, so a failed create
  cannot leave the PR with no Codex review.
- A listFiles failure degrades to a PR comment instead of crashing the step.
- Add coverage for the above plus multi-hunk parsing, unplaced->inline
  promotion, the 403 fallback, and dismissal error paths.
requiredEnv/optionalEnv were duplicated in render-review-prompt.js and
post-review.js with a subtle difference (post-review tolerates empty strings
for always()-passthrough outputs). Move them to a shared module with an
allowEmpty option so the two callers share one implementation.
- Reuse env-utils and a FINDING_SEVERITIES constant instead of repeating the
  ['low','medium','blocking'] list and the requiredEnv implementation.
- Extract placeInlineComment so the inline_comments and unplaced_findings
  placement loops share one diff-mapping implementation, and reuse pluralize.
- Delete inline comments left by earlier Codex runs (matched by a hidden marker
  and the bot identity, excluding the review just posted) so they no longer
  accumulate across runs, since dismissing a review does not remove them.
@brenthoneybone
brenthoneybone marked this pull request as ready for review July 13, 2026 15:41
@AltamashShaikh

Copy link
Copy Markdown
Contributor

@lachiebol @snake14 Can you guy check if we can re-use this in plugins easily ?

@snake14

snake14 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@lachiebol @snake14 Can you guy check if we can re-use this in plugins easily ?

@AltamashShaikh Yes. I think it would be very useful. We'd simply need to define a workflow in each plugin repo and ensure that the secret is available to each repo. 👍

Comment thread .github/workflows/codex-review.yml
@brenthoneybone

Copy link
Copy Markdown
Author

@lachiebol @snake14 Can you guy check if we can re-use this in plugins easily ?

@AltamashShaikh Yes. I think it would be very useful. We'd simply need to define a workflow in each plugin repo and ensure that the secret is available to each repo. 👍

@snake14 yeah you only need the workflow file in the repo - copy paste and only one line needs changing: plugin-name:. The keys are already setup on both Matomo and Innocraft orgs so won't need to set it at repo level.

@maxtaube
maxtaube self-requested a review July 24, 2026 10:32

@maxtaube maxtaube left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small comments, but nothing blocking

env:
CODEX_OUTPUT_DIR: ${{ runner.temp }}/codex-review
CODEX_RESULT: ${{ needs.codex.result }}
PREFLIGHT_RESULT: ${{ needs.preflight.result }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PREFLIGHT_RESULT is never used

skip_reason: ${{ steps.inspect.outputs.skip_reason || steps.request.outputs.skip_reason }}
skip_message: ${{ steps.inspect.outputs.skip_message || steps.request.outputs.skip_message }}
changed_files: ${{ steps.inspect.outputs.changed_files || steps.request.outputs.changed_files }}
automation_files: ${{ steps.inspect.outputs.automation_files || steps.request.outputs.automation_files }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't used downstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants