Repository navigation
fix(bedrock): configure native socket timeouts without hidden SDK retries - #219
rudycelekli wants to merge 2 commits into
Conversation
Signed-off-by: RudyCelekli <47457359+rudycelekli@users.noreply.github.com>
|
Thanks Rudy, the timeout now reaches the client. One problem: with SDK retries off, our loop only retries throttling, so one 500, 502, 503 or connection reset now drops the probe. Local fake Bedrock, one 503 then success: main returns the reply, this branch raises ProviderConnectionError. Retrying those in our loop with the same backoff would keep the recovery. Your call. |
Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
|
Reproduced your report with actual boto3 Converse calls to an owned local HTTP endpoint: a single 503 followed by success failed on the previous head, as did transient 500/502/504 and a real TCP disconnect. The correction retries these through the existing bounded manual loop and exponential backoff; SDK retries remain disabled. The expanded native socket controls change from 6 failed/12 passed to all 18 passed, with 23 adjacent SDK controls also passing. Zero retries still issue one request, persistent 503 exhausts the configured budget, and timeout/authentication/400 controls retain their classifications. No paid AWS/model calls were made. Signed correction |
|
Thanks Rudy, 5xx and resets recover now, and the timeout works (1s instead of 110s). One left: ModelNotReadyException comes back as HTTP 429, so it misses both the throttle check and the 5xx list at bedrock.py:152. One ModelNotReady then a 200: main recovers, this branch fails after 1 request. Adding it to that tuple fixes it (tried locally, recovers in 2 requests). Your call. |
Problem
The documented per-request timeout cancels the Bedrock coroutine, but the actual synchronous SDK read remains active with its default socket timeout. A CLI-owned
asyncio.runloop waits for that worker at shutdown: against an owned HTTP endpoint delayed 3 seconds,timeout=1returned after 3.063 seconds (measured after imports).Change
Configure connect/read socket timeouts for adapter-owned clients when the configured timeout is positive, and disable SDK retries so the existing bounded manual retry loop remains authoritative for throttling, HTTP 500/502/503/504, and connection failures. Zero and negative values preserve the prior immediate typed timeout with no request. Model, credential and client construction order is unchanged.
Validation
fd1c5545f67969fbc66d801e7a2d716175bb8ab9. The same expanded socket controls reproduce 6 failures/12 passes on the previous head.{"timeout":1,"status":"ProviderTimeoutError","requests":1}; valid response control:{"timeout":4,"status":"success","reply":"owned response","requests":1}.This bounds the tested stalled socket read, not a global DNS, streaming or arbitrary-thread termination deadline. Only newly created adapter-owned clients are configured. Existing #184 transport classification and held #183 judge truncation work concern different stages; owner gates on #143/#160/#183 remain unchanged.
AI assistance: GPT-6.1-sol assisted with source review, native reproduction, tests and implementation; every reported check was executed. SSH-signed commit includes DCO sign-off.
Original head
029d07a5bc4e305c1d182dfb3291b50c6f3dccd2fork CI on Python 3.10, 3.11 and 3.12 passed: workflow37553107495. These unchanged hosted install/lint/Bandit/layout/fixture checks are separate from the local native SDK tests. Upstream PR checks are reported separately.Follow-up to maintainer feedback
The reported transient retry regression is reproduced with actual boto3 requests to an owned local endpoint. Recovery now uses the existing manual retry count and exponential backoff, consistent with Boto3’s documented transient error categories, while native SDK retries stay disabled. Explicit
max_retries=0makes one attempt. Authentication, invalid-request and timeout classifications keep their previous behavior.The refreshed local Ruff/Bandit/layout/11-fixture checks pass; advisory mypy retains the same 12 existing missing-stub/dependency/NumPy annotation errors before and after this correction. Refreshed exact signed
fd1c5545f67969fbc66d801e7a2d716175bb8ab9passes all three unchanged fork CI jobs on Python 3.10/3.11/3.12. Those hosted install/lint/Bandit/layout/example gates do not run pytest; the local native SDK controls above are separate. Upstream checks remain unreported at this checkpoint.