fix(runner): emit a single real error frame instead of appending "produced no output" - #5321
Merged
Merged
Conversation
…ed no output" A swallowed-provider-error recovery (e.g. an out-of-credit key) both live-streams the real error AND fails the run's terminal result for the same failure. The zero-content-parts backstop never counted an error frame as content, so it piled a third, generic "The agent produced no output." frame on top of the real one(s) -- burying the actionable message. Track whether a real error already went out this turn and skip both the duplicate exception-derived frame and the generic backstop when it did.
Member
Author
|
@coderabbitai review |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughChangesVercel error handling
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
✅ Action performedReview finished.
|
Contributor
Railway Preview Environment
Updated at 2026-07-14T20:46:12.643Z |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
PR #5317 stopped Pi's swallowed provider errors (out-of-credit, bad key, rate limit) from
being silently swallowed into "The agent produced no output." On the fixed build, though, a
user hitting an out-of-credit key still saw only "The agent produced no output." in the UI,
with no trace of the real error.
Reproduced against the QA deployment (
docs/design/agent-workflows/projects/qa/scripts/qa_product.py --cell C3, a Pi/local cell with a vault OpenAI key that is genuinely out ofcredit). One
/invoketurn streamed THREE error frames, in this order:Root cause is two compounding bugs in
sdks/python/agenta/sdk/agents/adapters/vercel/stream.py,not in
services/runner(the runner'sfindSwallowedPiErrorrecovery,sandbox_agent.ts, isworking correctly and is unchanged by this PR):
errorevent (run.emitEvent,frame 1) AND fails the turn's terminal result (
{ok:false, error}). On the streaming path,that terminal failure is parsed by
result_from_wire, which raisesRuntimeError("Agent run failed: ..."). The exception propagates out of the event iterator uncaught (noexceptaround
agent_event_stream'sasync for) and is caught here as a second, differentlyworded
errorframe (frame 2) for the exact same failure. This dual-signal shape (live eventok:false) is not specific to Pi; it's how the runner reports every failure, so any runnererror duplicates today.
stop a genuinely blank
ok:trueturn from rendering an empty bubble) never checked whetheran
errorframe had already gone out. So it fired unconditionally whenever no text/tool/datacontent was emitted, piling a third, generic frame on top of two real ones (frame 3).
The generic string exists only on this SSE wire: it is never recorded to a trace span (the
zero-content backstop runs after the handler generator is exhausted and never touches OTel;
trace-side errors come from the runner's
recordError()spans, which carry the real message).So the UI showed it from its SSE-side error capture, which keeps the turn's LAST error frame.
Removing the trailing generic frame fixes the displayed message. Verdict: this is a real
masking bug, not a stale pre-#5317 image.
Changes
stream.pynow tracks whether a realerrorframe already went out this turn (whether from alive
etype == "error"event or from theexceptclause) and:exceptclause's frame when one already fired live, so the SAME underlying failureonly ever surfaces once instead of twice with different wording.
out, live or raised.
The backstop still fires exactly as before for a genuinely empty, non-erroring
ok:trueturn (nobehavior change there). Applied to both the live projection (
_agent_stream_to_vercel_stream_impl)and its dev-only twin (
_agent_run_to_vercel_parts_impl), which share this shape.Before (QA C3, out-of-credit): 3 error frames, ending in the generic message.
After: exactly 1 error frame,
"pi_core: the model provider account has insufficient credit (check the project's OpenAI key)."Tests / notes
test_swallowed_provider_error_emits_exactly_one_error_frame(+ dev-twin counterpart) intest_vercel_stream_conformance.py, replaying the exact live-event-then-failed-result shape andasserting exactly one
errorframe with the real message, and that the generic frame neverappears.
uv run --no-sync python -m pytest oss/tests/pytest/unit/agents/(sdks/python): 660 passed.ruff format+ruff check: clean.services/runnercode changed; its unit suite (pnpm test, 1158 tests) andpnpm run typecheckpass unmodified, confirming this is purely an SDK-side fix.https://claude.ai/code/session_01Hyn9365BLPXDmNZShrQkmH