Skip to content

Add an autonomous issue planner (GitHub Action) to generate test-first plans on issue creation #2256

Description

@acoliver

Goal

Add a self-hosted, GitHub-Action-driven issue planner that automatically generates implementation plans when issues are filed or edited, replacing CodeRabbit's issue-planner capability for the switch to ocr. It runs the LLxprt CLI agentic (same pattern as pr-review.yml) and posts a single idempotent, tagged plan comment.

Background / motivation

  • We are switching PR diff review from CodeRabbit to ocr (open-code-review): ocr finds more PR problems and is more cost-effective.
  • Gap: ocr has no planner — it is review / scan / rules / config / llm only. CodeRabbit's issue planner today produces (a) a semantically-matched Related PRs block, (b) a phased Coding Plan, and (c) a per-phase "Prompt for AI agents" block. This issue closes that gap with a self-hosted equivalent.
  • The existing .github/workflows/pr-review.yml already runs the LLxprt CLI in agentic --yolo --prompt mode, builds a context.md/instructions.md, captures agent output to a file, and posts a tagged comment. The planner is that same pipeline pointed at issues instead of PRs.

What the planner should do

The planner must be adaptive, not one-size-fits-all.

1. Size the issue (big vs small)

  • Estimate magnitude from: number of acceptance criteria, span of packages/files the issue likely touches, whether the body already implies phases, and whether it is a subissue of an epic.
  • Classify as small (single package / few criteria / small LoC magnitude) or large (multi-package / multi-phase / epic-scale).
  • Classification selects the output format (see o3 isn't working #3). Per repo policy, sizing must use LoC/magnitude only — never time estimates.
  • The plan must document the sizing basis so it is auditable, not a black box.

2. Always write test-first plans

Every plan, regardless of size, must be test-first:

  • Identify the tests that must exist before implementation, stated up front.
  • Favor adjusting/extending existing tests over creating new test files — unless the feature is genuinely new (no existing test file covers the area).
  • Name the concrete existing test files to extend (e.g. packages/core/src/.../__tests__/foo.test.ts) and the specific new cases to add.
  • Respect the test runner/framework conventions of the touched packages (vitest in this repo).
  • When a new test file is warranted, justify why no existing test file can absorb the cases.

3. Format the plan by size

  • Small issue — lightweight (mirrors CodeRabbit's compact output): a Summary bullet list, a Test plan (existing files to extend + new cases), short implementation steps, and a single "Prompt for AI agents" block.
  • Large issue — a GitHub-adapted version of dev-docs/PLAN.md / dev-docs/PLAN-TEMPLATE.md:
    • Phased structure with phase IDs, prerequisites, and a Phase 0.5 preflight verification (dependency/type/call-path/test-infrastructure checks against the actual repo).
    • stub -> TDD -> impl cycles per feature slice.
    • Integration analysis: which existing code will USE the feature, which existing code is REPLACED, how users ACCESS it, and migration needs (no isolated features — per the "STOP BUILDING ISOLATED FEATURES" rule in PLAN.md).
    • Per-phase verification (deferred-implementation detection: TODO/HACK/STUB/empty returns; behavioral checks).
    • Per-phase "Prompt for AI agents" blocks.
    • Adapted to a single GitHub issue comment (collapsible <details> sections) rather than the dev-docs project-plans/<feature>/ file tree.
  • Always include a Related PRs/issues block (CodeRabbit parity): semantically related merged PRs and issues, found by the agent during its run or via a heuristic precompute step.

4. Ground plans in repo conventions

  • Encode the repo's lint/complexity policy invariance: no new suppression directives (eslint-disable*, @ts-ignore/@ts-expect-error/@ts-nocheck), no ESLint severity downgrades, no increases to complexity/size thresholds, no new ignores: blocks.
  • Reference the real verification scripts the implementer must satisfy: npm run lint:ci, npm run lint:eslint-guard, npm run typecheck, npm run test.
  • Honor any explicit constraints stated in the issue body (e.g. no-console handling, specific design decisions).

How it runs

  • New workflow: .github/workflows/issue-planner.yml.
  • Triggers:
    • on: issues: [opened, edited, reopened, labeled] for automatic planning.
    • on: issue_comment: [created], gated on a /plan (and /plan <feedback>) slash command, to replicate CodeRabbit's @coderabbitai plan replan UX.
  • Job modeled on pr-review.yml: checkout -> gh issue view > issue.json -> build issue-context.md (issue body + linked parent/sibling issues + repo conventions + candidate related PRs) -> build planning-instructions.md (encoding rules 1-4 above) -> run llxprt --yolo --prompt "..." writing plan.md -> post the plan.
  • Idempotent: comment tag <!-- llxprt-issue-plan -->; replans edit the existing comment rather than stacking new ones.
  • Permissions: issues: write, contents: read (less than the PR review job needs; on: issues is repo-scoped, so the pull_request_target/fork-safety concerns do not apply).
  • Key budget: reuse scripts/ci-quota-check.js key rotation shared with pr-review.yml. No new external service or secret beyond existing repo vars.

Out of scope / non-goals

  • Does not replace ocr on PRs — ocr remains the PR diff reviewer. The planner and ocr are complementary: ocr = review, LLxprt-Action = plan.
  • Does not block issue flow; planning is advisory.
  • Does not run the dev-docs project-plans/<feature>/ file-tree flow on disk; that stays a separate local workflow. The issue planner adapts that structure into an in-comment form.

Acceptance criteria

  • New workflow issue-planner.yml triggers on issue open/edit/reopen/label and on a /plan (and /plan <feedback>) comment.
  • Planner sizes each issue (small/large) and documents the basis (criteria count, package span, magnitude — no time estimates).
  • All plans are test-first and name the existing test files to extend, or justify a new test file.
  • Large issues receive a PLAN.md/PLAN-TEMPLATE.md-adapted phased plan in GitHub comment form (preflight, stub->TDD->impl, integration analysis, per-phase verification); small issues receive the lightweight format.
  • Every plan includes a Related PRs/issues block.
  • Plans are grounded in repo lint/complexity policy and name the verification scripts to satisfy.
  • Plans post as a single idempotent tagged comment; replan edits in place.
  • Shares key/quota logic with pr-review.yml; no new external service and no new secrets beyond existing vars.
  • The new workflow passes actionlint/shellcheck consistent with the other workflows; the repo's lint:ci, lint:eslint-guard, typecheck, and test all remain green.

Notes / decisions to confirm

  • Sizing thresholds for small vs large (propose: criteria count + package span + LoC magnitude).
  • Whether PLAN.md-style code markers (@plan:/@requirement:) should be emitted inside the in-comment plan — they are designed for the on-disk file-tree flow and may be adapted or omitted for issue comments.
  • Replan UX confirmation: edit-in-place vs. appending a versioned history.
  • Whether to gate the initial rollout behind a label (e.g. only plan issues labeled plan) before enabling for all issues.

Metadata

Metadata

Assignees

Labels

ci/cdIssues with github and the workflow scripts and CI CD environment.planPlanning work / issue-planner output

Type

No type

Projects

Status
Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions