Skip to content

fix(runner): emit a single real error frame instead of appending "produced no output" - #5321

Merged
mmabrouk merged 1 commit into
release/v0.105.0from
fix-pi-duplicate-error-frame
Jul 14, 2026
Merged

fix(runner): emit a single real error frame instead of appending "produced no output"#5321
mmabrouk merged 1 commit into
release/v0.105.0from
fix-pi-duplicate-error-frame

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Jul 14, 2026

Copy link
Copy Markdown
Member

Context

PR #5317 stopped Pi's swallowed provider errors (out-of-credit, bad key, rate limit) from
being silently swallowed into "The agent produced no output." On the fixed build, though, a
user hitting an out-of-credit key still saw only "The agent produced no output." in the UI,
with no trace of the real error.

Reproduced against the QA deployment (docs/design/agent-workflows/projects/qa/scripts/qa_product.py --cell C3, a Pi/local cell with a vault OpenAI key that is genuinely out of
credit). One /invoke turn streamed THREE error frames, in this order:

error  "pi_core: the model provider account has insufficient credit (check the project's OpenAI key)."
error  "Agent run failed: pi_core: the model provider account has insufficient credit (check the project's OpenAI key)."
error  "The agent produced no output."

Root cause is two compounding bugs in sdks/python/agenta/sdk/agents/adapters/vercel/stream.py,
not in services/runner (the runner's findSwallowedPiError recovery, sandbox_agent.ts, is
working correctly and is unchanged by this PR):

  1. The runner both live-streams the recovered error as an error event (run.emitEvent,
    frame 1) AND fails the turn's terminal result ({ok:false, error}). On the streaming path,
    that terminal failure is parsed by result_from_wire, which raises RuntimeError("Agent run failed: ..."). The exception propagates out of the event iterator uncaught (no except
    around agent_event_stream's async for) and is caught here as a second, differently
    worded error frame (frame 2) for the exact same failure. This dual-signal shape (live event
    • ok:false) is not specific to Pi; it's how the runner reports every failure, so any runner
      error duplicates today.
  2. The zero-content-parts backstop that synthesizes "The agent produced no output." (added to
    stop a genuinely blank ok:true turn from rendering an empty bubble) never checked whether
    an error frame had already gone out. So it fired unconditionally whenever no text/tool/data
    content was emitted, piling a third, generic frame on top of two real ones (frame 3).

The generic string exists only on this SSE wire: it is never recorded to a trace span (the
zero-content backstop runs after the handler generator is exhausted and never touches OTel;
trace-side errors come from the runner's recordError() spans, which carry the real message).
So the UI showed it from its SSE-side error capture, which keeps the turn's LAST error frame.
Removing the trailing generic frame fixes the displayed message. Verdict: this is a real
masking bug
, not a stale pre-#5317 image.

Changes

stream.py now tracks whether a real error frame already went out this turn (whether from a
live etype == "error" event or from the except clause) and:

  • Skips the except clause's frame when one already fired live, so the SAME underlying failure
    only ever surfaces once instead of twice with different wording.
  • Skips the zero-content-parts "produced no output" backstop whenever any real error frame went
    out, live or raised.

The backstop still fires exactly as before for a genuinely empty, non-erroring ok:true turn (no
behavior change there). Applied to both the live projection (_agent_stream_to_vercel_stream_impl)
and its dev-only twin (_agent_run_to_vercel_parts_impl), which share this shape.

Before (QA C3, out-of-credit): 3 error frames, ending in the generic message.
After: exactly 1 error frame, "pi_core: the model provider account has insufficient credit (check the project's OpenAI key)."

Tests / notes

  • Added test_swallowed_provider_error_emits_exactly_one_error_frame (+ dev-twin counterpart) in
    test_vercel_stream_conformance.py, replaying the exact live-event-then-failed-result shape and
    asserting exactly one error frame with the real message, and that the generic frame never
    appears.
  • uv run --no-sync python -m pytest oss/tests/pytest/unit/agents/ (sdks/python): 660 passed.
  • ruff format + ruff check: clean.
  • No services/runner code changed; its unit suite (pnpm test, 1158 tests) and pnpm run typecheck pass unmodified, confirming this is purely an SDK-side fix.

https://claude.ai/code/session_01Hyn9365BLPXDmNZShrQkmH

…ed no output"

A swallowed-provider-error recovery (e.g. an out-of-credit key) both live-streams
the real error AND fails the run's terminal result for the same failure. The
zero-content-parts backstop never counted an error frame as content, so it piled
a third, generic "The agent produced no output." frame on top of the real one(s)
-- burying the actionable message. Track whether a real error already went out
this turn and skip both the duplicate exception-derived frame and the generic
backstop when it did.
@mmabrouk

Copy link
Copy Markdown
Member Author

@coderabbitai review

@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jul 14, 2026
@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Ready Ready Preview, Comment Jul 14, 2026 8:40pm

Request Review

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Changes

Vercel error handling

Layer / File(s) Summary
Run adapter error suppression
sdks/python/agenta/sdk/agents/adapters/vercel/stream.py, sdks/python/oss/tests/pytest/unit/agents/adapters/test_vercel_stream_conformance.py
The run-based adapter tracks emitted error events, suppresses duplicate exception and no-output frames, and tests strict-schema output for swallowed provider errors.
Stream adapter error suppression
sdks/python/agenta/sdk/agents/adapters/vercel/stream.py, sdks/python/oss/tests/pytest/unit/agents/adapters/test_vercel_stream_conformance.py
The stream-based adapter applies the same error suppression to exception and finalization paths, with corresponding conformance coverage.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title matches the core fix: emitting one real error frame and suppressing the generic "produced no output" frame.
Description check ✅ Passed The description clearly describes the adapter bug, the fix, and the added tests, and it matches the changeset.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-pi-duplicate-error-frame

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@dosubot dosubot Bot added bug Something isn't working python Pull requests that update Python code labels Jul 14, 2026
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Railway Preview Environment

Status Destroyed (PR closed)

Updated at 2026-07-14T20:46:12.643Z

@mmabrouk
mmabrouk merged commit 14895df into release/v0.105.0 Jul 14, 2026
40 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working python Pull requests that update Python code size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant