Agent/issue agent - #6
Merged
Merged
Conversation
![cubic-dev-ai[bot]](https://avatars.github.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
4 issues found across 67 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/issue-agent/README.md">
<violation number="1" location="packages/issue-agent/README.md:206">
P2: A fresh production setup remains unable to call Gemini because the Usage configuration omits the required `GOOGLE_API_KEY`. Including it alongside the GitHub settings makes the run instructions self-contained.</violation>
</file>
<file name="packages/issue-agent/src/issue_agent/agent.py">
<violation number="1" location="packages/issue-agent/src/issue_agent/agent.py:163">
P0: Production completion has no exactly-once GitHub issue guarantee: the model may skip/repeat this tool, and a later 429 can replay the run after creation. Creating one issue in host code after validating the report, with an incident idempotency key and surfaced API failure, would make the side effect reliable.</violation>
</file>
<file name="README.md">
<violation number="1" location="README.md:125">
P2: The documented E2E command fails on a fresh checkout because `packages/evaluator/.venv` is neither tracked nor created anywhere in this Quickstart. Consider adding the venv creation/install steps before these commands or using a documented environment-independent runner.</violation>
</file>
<file name="packages/issue-agent/tests/scenarios/tool_call_anomaly/evaluator.json">
<violation number="1" location="packages/issue-agent/tests/scenarios/tool_call_anomaly/evaluator.json:4">
P3: Evaluator reasoning doesn't match incident reasoning pattern. All other evaluator.json files use the first sentence verbatim from incident.json's evaluation.reasoning, but this file uses a completely rewritten shorter version.</violation>
</file>
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
| @@ -0,0 +1,323 @@ | |||
| import json | |||
There was a problem hiding this comment.
P0: Production completion has no exactly-once GitHub issue guarantee: the model may skip/repeat this tool, and a later 429 can replay the run after creation. Creating one issue in host code after validating the report, with an incident idempotency key and surfaced API failure, would make the side effect reliable.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/issue-agent/src/issue_agent/agent.py, line 163:
<comment>Production completion has no exactly-once GitHub issue guarantee: the model may skip/repeat this tool, and a later 429 can replay the run after creation. Creating one issue in host code after validating the report, with an incident idempotency key and surfaced API failure, would make the side effect reliable.</comment>
<file context>
@@ -0,0 +1,323 @@
+ if not os.environ.get("VOID_DEV_MODE"):
+ @agent.tool
+ def create_github_issue(ctx: RunContext[IssueAgentDeps], title: str, body: str) -> str:
+ url = ctx.deps.repo.create_issue(title, body)
+ ctx.deps.tokens_used += 1
+ if url:
</file context>
Member
Author
|
@coderabbitai review |
Member
Author
Member
Author
|
@coderabbitai please review the PR |
|
✅ Action performedReview finished.
|
📝 WalkthroughWalkthroughThe PR introduces an ChangesIssue Agent implementation
Estimated code review effort: 4 (Complex) | ~60 minutes 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.github.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 11
🧹 Nitpick comments (2)
packages/issue-agent/src/issue_agent/agent.py (1)
152-173: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd docstrings to the registered tools.
search_repo,read_file, andbuild_code_graphhave no docstrings. Per pydantic-ai's tool registration model, docstrings are inspected to build the tool/parameter descriptions sent to the model — omitting them likely means the LLM receives no guidance beyond bare type hints, which can hurt tool-selection quality for an agent whose entire job is investigative tool use.✏️ Example
`@agent.tool` def search_repo(ctx: RunContext[IssueAgentDeps], query: str) -> str: + """Search the repository for files/symbols matching `query`.""" result = ctx.deps.repo.search_symbol(query)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/issue-agent/src/issue_agent/agent.py` around lines 152 - 173, Add concise docstrings to the registered tools search_repo, read_file, and build_code_graph describing each tool’s purpose and its parameters, so pydantic-ai can expose useful tool and parameter descriptions to the model. Preserve their existing behavior and signatures.packages/issue-agent/tests/e2e_monitor.py (1)
182-332: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMultiple Ruff lint errors (F541 f-strings without placeholders, E741 ambiguous
l).Static analysis flags many
f"..."strings with no interpolation (Lines 182, 187, 190, 194, 197, 212, 223, 228, 246, 261, 270, 285, 288, 291, 293, 297, 300, 306, 316, 319, 323, 328, 332) and ambiguous loop variablel(Lines 214, 221, 229) at[error]severity — these are likely to fail a Ruff-gated CI check even though they're harmless at runtime.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/issue-agent/tests/e2e_monitor.py` around lines 182 - 332, Clean up the reporting block around the evaluator output, issue-agent logs, LLM conversation, and engineering report: replace f-strings without interpolated values with plain strings, while preserving interpolation where present. Rename the ambiguous loop variable l in the log-processing loops to a descriptive name and update its references consistently, without changing output or behavior.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/issue-agent/README.md`:
- Line 18: Update the README’s tool descriptions and flow diagram to list only
the three tools actually registered by _build_agent: search_repo, read_file, and
build_code_graph. Remove create_issue/create_github_issue from the tools table
and the “not registered” wording, while preserving the later description of
create_github_issue_from_report as a separate host-code function.
In `@packages/issue-agent/src/issue_agent/__main__.py`:
- Around line 39-45: Update the main execution flow around run_issue_agent and
the report handling so production mode calls create_github_issue_from_report()
after a valid report is produced, while dev mode retains the existing JSON-print
behavior. Preserve the current failure path for missing reports and use the
existing helper from agent.py rather than duplicating issue-creation logic.
In `@packages/issue-agent/src/issue_agent/repository.py`:
- Around line 72-79: Update the repository search logic at
packages/issue-agent/src/issue_agent/repository.py lines 72-79 and 151-159 so
content matching is independent of filename matching: read and inspect every
repository blob or allowed text file, then record content matches separately
from filename matches while preserving existing path-filter behavior for
filename results. Use the existing _search_source_content flow and ensure both
affected sites support symbols absent from the file path.
- Around line 64-66: Update the shared request helper containing the GitHub tree
request to catch httpx.RequestError and return the helper’s established error
result instead of propagating the exception. Ensure search_symbol, read_file,
search_files, and create_issue continue converting that result into their
documented empty or None fallbacks, without adding separate exception handling
to each method.
- Around line 107-112: Update Repository.search_files to call GitHub’s global
search endpoint using an absolute API URL rather than the repo-scoped base URL.
Pass the search query through params with the filename: qualifier and retain the
existing response-error handling and result mapping.
In `@packages/issue-agent/tests/e2e_monitor.py`:
- Around line 238-240: Update the tool_call handling in the monitor’s
message-printing logic to read the logged tool_args_count field instead of the
nonexistent tool_args field. Display the count while preserving the existing
no-raw-arguments behavior and tool name output.
In `@packages/issue-agent/tests/issue-agent/looping/evaluation.json`:
- Around line 3-20: Update the evaluation.json summary and reasoning to describe
repeated retries until the internal retry limit, not an infinite loop. Preserve
the finding that identical search_api calls ignored rate-limit errors and lacked
backoff or adaptive behavior, while removing any wording that overstates the
liveness failure.
In `@packages/issue-agent/tests/run_all.sh`:
- Around line 22-24: Remove or update the nonexistent test_issue_agent_e2e.py
invocation in run_all.sh. Ensure the E2E test section runs only an existing test
module from this PR, or remove that section if no E2E test is available, so the
script completes without failing under set -euo pipefail.
In `@packages/issue-agent/tests/scenarios/planner_bug/incident.json`:
- Around line 57-75: Reconcile
packages/issue-agent/tests/scenarios/planner_bug/incident.json lines 57-75 with
the actual 47-call trace: update the call count, latency arithmetic, telemetry
fields, and embedded evaluation so they consistently reflect that trace. Then
regenerate the evaluator narrative in
packages/issue-agent/tests/scenarios/planner_bug/evaluator.json lines 2-4 from
the corrected trace and metadata.
In `@packages/issue-agent/tests/test_helpers.py`:
- Around line 162-177: Update the INSUFFICIENT_EVIDENCE validation in the test
helper to enforce must_not_suggest_fix and must_not_identify_root_cause without
length thresholds. Require the corresponding fields to be empty, or match only
the predefined insufficient-evidence boilerplate, and preserve the existing
confidence and summary checks.
- Around line 179-205: The validate_report() REAL_INCIDENT validation must
enforce the expected evidence_must_contain and relevant_files_must_contain
assertions. Add case-insensitive containment checks against report.evidence and
report.relevant_files, appending validation errors consistent with the existing
suspected-components, investigation, and tests checks; alternatively remove
those unsupported expected keys, but keep the scenario contract and validator
behavior aligned.
---
Nitpick comments:
In `@packages/issue-agent/src/issue_agent/agent.py`:
- Around line 152-173: Add concise docstrings to the registered tools
search_repo, read_file, and build_code_graph describing each tool’s purpose and
its parameters, so pydantic-ai can expose useful tool and parameter descriptions
to the model. Preserve their existing behavior and signatures.
In `@packages/issue-agent/tests/e2e_monitor.py`:
- Around line 182-332: Clean up the reporting block around the evaluator output,
issue-agent logs, LLM conversation, and engineering report: replace f-strings
without interpolated values with plain strings, while preserving interpolation
where present. Rename the ambiguous loop variable l in the log-processing loops
to a descriptive name and update its references consistently, without changing
output or behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: d3cb4252-7660-4b86-b5e4-a0f904b4c957
📒 Files selected for processing (60)
README.mdpackages/issue-agent/README.mdpackages/issue-agent/pyproject.tomlpackages/issue-agent/src/issue_agent/__init__.pypackages/issue-agent/src/issue_agent/__main__.pypackages/issue-agent/src/issue_agent/agent.pypackages/issue-agent/src/issue_agent/evidence.pypackages/issue-agent/src/issue_agent/repository.pypackages/issue-agent/src/issue_agent/schemas.pypackages/issue-agent/tests/__init__.pypackages/issue-agent/tests/e2e_monitor.pypackages/issue-agent/tests/issue-agent/context-overflow/evaluation.jsonpackages/issue-agent/tests/issue-agent/context-overflow/expected.jsonpackages/issue-agent/tests/issue-agent/crash-loop/evaluation.jsonpackages/issue-agent/tests/issue-agent/crash-loop/expected.jsonpackages/issue-agent/tests/issue-agent/critical-escalation/evaluation.jsonpackages/issue-agent/tests/issue-agent/critical-escalation/expected.jsonpackages/issue-agent/tests/issue-agent/example/evaluation.jsonpackages/issue-agent/tests/issue-agent/example/expected.jsonpackages/issue-agent/tests/issue-agent/false-positive/evaluation.jsonpackages/issue-agent/tests/issue-agent/false-positive/expected.jsonpackages/issue-agent/tests/issue-agent/handoff-failure/evaluation.jsonpackages/issue-agent/tests/issue-agent/handoff-failure/expected.jsonpackages/issue-agent/tests/issue-agent/insufficient-evidence/evaluation.jsonpackages/issue-agent/tests/issue-agent/insufficient-evidence/expected.jsonpackages/issue-agent/tests/issue-agent/looping/evaluation.jsonpackages/issue-agent/tests/issue-agent/looping/expected.jsonpackages/issue-agent/tests/issue-agent/mixed-labels/evaluation.jsonpackages/issue-agent/tests/issue-agent/mixed-labels/expected.jsonpackages/issue-agent/tests/issue-agent/rate-limit/evaluation.jsonpackages/issue-agent/tests/issue-agent/rate-limit/expected.jsonpackages/issue-agent/tests/issue-agent/recurring/evaluation.jsonpackages/issue-agent/tests/issue-agent/recurring/expected.jsonpackages/issue-agent/tests/issue-agent/silent-hallucination/evaluation.jsonpackages/issue-agent/tests/issue-agent/silent-hallucination/expected.jsonpackages/issue-agent/tests/issue-agent/tool-anomaly/evaluation.jsonpackages/issue-agent/tests/issue-agent/tool-anomaly/expected.jsonpackages/issue-agent/tests/issue-agent/transient-error/evaluation.jsonpackages/issue-agent/tests/issue-agent/transient-error/expected.jsonpackages/issue-agent/tests/run_all.shpackages/issue-agent/tests/scenarios/context_overflow/evaluator.jsonpackages/issue-agent/tests/scenarios/context_overflow/expected.jsonpackages/issue-agent/tests/scenarios/context_overflow/incident.jsonpackages/issue-agent/tests/scenarios/handoff_failure/evaluator.jsonpackages/issue-agent/tests/scenarios/handoff_failure/expected.jsonpackages/issue-agent/tests/scenarios/handoff_failure/incident.jsonpackages/issue-agent/tests/scenarios/planner_bug/evaluator.jsonpackages/issue-agent/tests/scenarios/planner_bug/expected.jsonpackages/issue-agent/tests/scenarios/planner_bug/incident.jsonpackages/issue-agent/tests/scenarios/tool_call_anomaly/evaluator.jsonpackages/issue-agent/tests/scenarios/tool_call_anomaly/expected.jsonpackages/issue-agent/tests/scenarios/tool_call_anomaly/incident.jsonpackages/issue-agent/tests/scenarios/tool_timeout/evaluator.jsonpackages/issue-agent/tests/scenarios/tool_timeout/expected.jsonpackages/issue-agent/tests/scenarios/tool_timeout/incident.jsonpackages/issue-agent/tests/test_agent.pypackages/issue-agent/tests/test_helpers.pypackages/issue-agent/tests/test_mapper.pypackages/issue-agent/tests/test_repository.pypackages/issue-agent/tests/test_schemas.py
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/issue-agent/src/issue_agent/__main__.py`:
- Around line 39-45: Update the VOID_DEV_MODE parsing that determines the
dev_mode value before the LocalRepo/GitHubRepo selection to use strict boolean
semantics, enabling dev mode only when the environment value is explicitly "1"
(or otherwise valid true). Ensure values such as "0", "false", empty, or unset
select GitHubRepo and preserve the existing report handling.
In `@packages/issue-agent/src/issue_agent/agent.py`:
- Around line 153-164: The search_repo contract does not match
GitHubRepo.search_symbol behavior because path-only matching misses symbols
inside files. Update GitHubRepo.search_symbol to search repository file contents
independently of filename while preserving filename matches and result
formatting, then add a regression test verifying a symbol such as OrderService
is found in a differently named source file.
- Around line 162-164: Rename the usage counter currently updated in
search_repo, read_file, and build_code_graph from tokens_used to
tool_calls_used, including all related reads and reporting, while preserving its
increment once per repository tool invocation. Keep usage.total_tokens as the
separate model-token metric.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: f8e8331e-2a07-4dda-b639-75b667f27f95
📒 Files selected for processing (10)
packages/issue-agent/src/issue_agent/__main__.pypackages/issue-agent/src/issue_agent/agent.pypackages/issue-agent/src/issue_agent/repository.pypackages/issue-agent/tests/e2e_monitor.pypackages/issue-agent/tests/issue-agent/looping/evaluation.jsonpackages/issue-agent/tests/run_all.shpackages/issue-agent/tests/scenarios/planner_bug/evaluator.jsonpackages/issue-agent/tests/scenarios/planner_bug/incident.jsonpackages/issue-agent/tests/test_helpers.pypackages/issue-agent/tests/test_mapper.py
🚧 Files skipped from review as they are similar to previous changes (7)
- packages/issue-agent/tests/scenarios/planner_bug/evaluator.json
- packages/issue-agent/tests/scenarios/planner_bug/incident.json
- packages/issue-agent/tests/issue-agent/looping/evaluation.json
- packages/issue-agent/tests/e2e_monitor.py
- packages/issue-agent/tests/test_mapper.py
- packages/issue-agent/tests/test_helpers.py
- packages/issue-agent/src/issue_agent/repository.py
This was referenced Jul 24, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by cubic
Add the
issue-agentpackage to turn evaluated incidents into engineering-ready reports and optionally create GitHub issues. Adds a CLI, repository investigation, structured schemas, tests, and E2E monitoring with scenario runner.New Features
packages/issue-agentwith CLIissue-agent(reads IncidentSnapshot JSON from stdin, outputs EngineeringReport JSON; dev usesLocalRepo; production can create a GitHub issue whenGITHUB_TOKENandDEMO_REPOSITORYare set).pydantic-aiwith tools to search the repo, read files, build a simple code graph, and prepare/create a GitHub issue payload.GitHubRepo(GitHub API viahttpx) andLocalRepo.pydantic,pydantic-ai,httpx.Migration
GOOGLE_API_KEY(orGEMINI_API_KEY).VOID_DEV_MODE=1. GitHub mode: setGITHUB_TOKENandDEMO_REPOSITORY.packages/evaluator-epackages/issue-agent; then run the E2E monitor orpackages/issue-agent/tests/run_all.sh.Written for commit b74239f. Summary will update on new commits.
Summary by CodeRabbit