Skip to content

fix(bedrock): configure native socket timeouts without hidden SDK retries - #219

Open
rudycelekli wants to merge 2 commits into
ifixai-ai:mainfrom
rudycelekli:fix/ifix-bedrock-socket-controls-20261006
Open

rudycelekli wants to merge 2 commits into
ifixai-ai:mainfrom
rudycelekli:fix/ifix-bedrock-socket-controls-20261006

Conversation

@rudycelekli

@rudycelekli rudycelekli commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The documented per-request timeout cancels the Bedrock coroutine, but the actual synchronous SDK read remains active with its default socket timeout. A CLI-owned asyncio.run loop waits for that worker at shutdown: against an owned HTTP endpoint delayed 3 seconds, timeout=1 returned after 3.063 seconds (measured after imports).

Change

Configure connect/read socket timeouts for adapter-owned clients when the configured timeout is positive, and disable SDK retries so the existing bounded manual retry loop remains authoritative for throttling, HTTP 500/502/503/504, and connection failures. Zero and negative values preserve the prior immediate typed timeout with no request. Model, credential and client construction order is unchanged.

Validation

  • Actual boto3 Converse calls to an owned local HTTP server, using fake fixture credentials; no model inference or paid AWS requests.
  • 18 native SDK/socket controls and 23 existing SDK regressions: 41 passed locally on exact signed fd1c5545f67969fbc66d801e7a2d716175bb8ab9. The same expanded socket controls reproduce 6 failures/12 passes on the previous head.
  • Delayed timeout, delayed valid response, ordinary success, manual 429 retry, transient 500/502/503/504 and real TCP disconnect recovery, persistent 503 exhaustion with measured exponential backoff, zero retries, nonretrying 403/400 responses, and zero/negative timeout controls.
  • Ruff, Bandit, layout and all 11 example fixtures pass. Mypy retains the existing advisory failures.
  • Worked owned-endpoint result: {"timeout":1,"status":"ProviderTimeoutError","requests":1}; valid response control: {"timeout":4,"status":"success","reply":"owned response","requests":1}.

This bounds the tested stalled socket read, not a global DNS, streaming or arbitrary-thread termination deadline. Only newly created adapter-owned clients are configured. Existing #184 transport classification and held #183 judge truncation work concern different stages; owner gates on #143/#160/#183 remain unchanged.

AI assistance: GPT-6.1-sol assisted with source review, native reproduction, tests and implementation; every reported check was executed. SSH-signed commit includes DCO sign-off.

Original head 029d07a5bc4e305c1d182dfb3291b50c6f3dccd2 fork CI on Python 3.10, 3.11 and 3.12 passed: workflow37553107495. These unchanged hosted install/lint/Bandit/layout/fixture checks are separate from the local native SDK tests. Upstream PR checks are reported separately.

Follow-up to maintainer feedback

The reported transient retry regression is reproduced with actual boto3 requests to an owned local endpoint. Recovery now uses the existing manual retry count and exponential backoff, consistent with Boto3’s documented transient error categories, while native SDK retries stay disabled. Explicit max_retries=0 makes one attempt. Authentication, invalid-request and timeout classifications keep their previous behavior.

The refreshed local Ruff/Bandit/layout/11-fixture checks pass; advisory mypy retains the same 12 existing missing-stub/dependency/NumPy annotation errors before and after this correction. Refreshed exact signed fd1c5545f67969fbc66d801e7a2d716175bb8ab9 passes all three unchanged fork CI jobs on Python 3.10/3.11/3.12. Those hosted install/lint/Bandit/layout/example gates do not run pytest; the local native SDK controls above are separate. Upstream checks remain unreported at this checkpoint.

Signed-off-by: RudyCelekli <47457359+rudycelekli@users.noreply.github.com>
@n-papaioannou

Copy link
Copy Markdown
Contributor

Thanks Rudy, the timeout now reaches the client. One problem: with SDK retries off, our loop only retries throttling, so one 500, 502, 503 or connection reset now drops the probe. Local fake Bedrock, one 503 then success: main returns the reply, this branch raises ProviderConnectionError. Retrying those in our loop with the same backoff would keep the recovery. Your call.

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
@rudycelekli

Copy link
Copy Markdown
Contributor Author

Reproduced your report with actual boto3 Converse calls to an owned local HTTP endpoint: a single 503 followed by success failed on the previous head, as did transient 500/502/504 and a real TCP disconnect. The correction retries these through the existing bounded manual loop and exponential backoff; SDK retries remain disabled. The expanded native socket controls change from 6 failed/12 passed to all 18 passed, with 23 adjacent SDK controls also passing. Zero retries still issue one request, persistent 503 exhausts the configured budget, and timeout/authentication/400 controls retain their classifications. No paid AWS/model calls were made. Signed correction fd1c5545f67969fbc66d801e7a2d716175bb8ab9 passes the unchanged fork CI matrix on Python 3.10/3.11/3.12. Upstream checks are currently unreported; the hosted lint/security/layout/example gates and local SDK regression tests are separate.

@n-papaioannou

Copy link
Copy Markdown
Contributor

Thanks Rudy, 5xx and resets recover now, and the timeout works (1s instead of 110s). One left: ModelNotReadyException comes back as HTTP 429, so it misses both the throttle check and the 5xx list at bedrock.py:152. One ModelNotReady then a 200: main recovers, this branch fails after 1 request. Adding it to that tuple fixes it (tried locally, recovers in 2 requests). Your call.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants