Skip to content

[Bug]: Provider length-limit stop reasons are lost, so incomplete tool calls may run #3194

Description

@wuyak

Prerequisites

  • I have searched the existing issues and discussions, and this is not a duplicate.
  • This is a bug, not a usage question. (For questions, please use Discussions instead.)

Background / Description

When a model provider stops generation because an output-length limit was reached, AgentScope does not currently propagate that stop reason through ChatResponse. The response therefore remains indistinguishable from a normally completed response.

This becomes unsafe when the truncated response ends while generating tool arguments. For example, assume the agent exposes this tool:

def write_file(path: str, content: str):
    with open(path, "w") as file:
        file.write(content)

The model may intend to call it with:

{
  "path": "/tmp/result.txt",
  "content": "unfinished task, please continue..."
}

If generation reaches the provider's output limit while producing content, the returned arguments may instead be:

{"path": "/tmp/result.txt", "content": "unfinished

AgentScope's existing JSON repair can turn that into syntactically valid JSON:

{"path": "/tmp/result.txt", "content": "unfinished"}

The tool can then be executed as if the model had intentionally requested:

write_file(
    path="/tmp/result.txt",
    content="unfinished",
)

The repair restores JSON syntax, but it cannot recover the content that the model had not generated. For tools with side effects, this can turn a partial operation into a real file, database, shell, or API mutation.

Expected behavior

AgentScope should:

  • read the provider's raw stop reason in each model adapter's existing response-parsing path;
  • preserve the raw value for diagnostics and normalize only recognized length-limit values to a common FinishedReason.LENGTH;
  • provide an opt-in ReAct guard, disabled by default:
ReActConfig(reject_truncated_tool_call=True)

When enabled, the guard should reject a tool call only when all of the following are true:

  1. the provider explicitly reported a length-limit stop;
  2. the final response block is a tool call;
  3. the final tool call's arguments fail strict json.loads parsing.

The rejected call should receive the existing error ToolResult so that the tool protocol remains balanced. Earlier complete tool calls in the same response should still run, and the existing ReAct loop should decide whether to continue.

Scope

This behavior should not:

  • infer truncation from token usage or missing business fields;
  • reject tool arguments that are valid JSON;
  • globally stop the agent on every length-limited response;
  • add a truncation-specific retry, continuation, or argument-completion mechanism;
  • change existing behavior while the option is disabled.

Error Messages

No exception is raised. The incomplete JSON may be repaired and the tool may be executed with partial arguments.

Steps to Reproduce

  1. Return a model response whose provider stop reason indicates an output-length limit.

  2. Make the final response block a tool call with incomplete JSON arguments, for example:

    {"path": "/tmp/result.txt", "content": "unfinished
  3. Let the current Agent tool path process the call.

  4. Observe that JSON repair can close the string and object, after which the tool executes with the partial content value.

  5. Observe that the provider's length-limit stop reason is not available on the resulting ChatResponse.

Related work

#2323 addresses the same provider signal but also changes global ReAct termination, reply reasons, warnings, hints, events, and OpenAI Responses behavior. This issue is intentionally narrower: preserve the provider signal and optionally reject only the final tool call whose JSON arguments are demonstrably incomplete.

Environment

  • AgentScope Version: 2.0.10dev
  • Python Version: 3.12.13
  • OS: macOS 26.6.1 (arm64)

Activity

  1. added
    triage/verifyingA bot is verifying whether this bug is real
    triage/confirmedVerified: the reported defect exists
    and removed
    triage/verifyingA bot is verifying whether this bug is real
    on Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triage/confirmedVerified: the reported defect exists

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions