You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Every CI run since Aug 1 fails on Unit Tests (ubuntu-latest, 26.x) with the same single failure:
FAIL tests/api.test.mts > threads crud > update state while run in flight
AssertionError: expected 'error' to be 'interrupted'
This hits every branch, including dependabot PRs that only bump frontend deps. The last green run on main (Jul 29) used Node 26.5.0; runners now resolve 26.x to 26.5.1. Nothing in the repo changed.
Reproduction
On unmodified main, CI=1 npx vitest run tests/api.test.mts in libs/langgraph-api:
Node
undici (bundled)
Result
26.5.0
8.7.0
pass (2/2)
26.5.1
8.9.0
fail (3/3, same assertion)
The failure needs the full file's concurrent load; the test passes in isolation via -t.
Root cause
The regression is in undici, which jumped 8.7.0 → 8.9.0 inside the Node patch release. Cross-checking by overriding the SDK's fetch with standalone undici builds:
Node
fetch from
Result
26.5.0
undici@8.9.0
fail
26.5.0
undici@8.8.0
fail (2/2)
26.5.1
undici@8.7.0
pass
So the change landed in undici 8.8.0. A diagnostics_channel trace of the failing POST /threads/:id/state request shows where the time goes:
undici:request:create +0ms
undici:client:sendHeaders +1560ms <- request sat in the dispatcher
undici:request:headers +1563ms
The server handled the request in 1ms (hono log). Under the suite's load (67 concurrent tests sharing one client, several long-lived SSE joins) requests intermittently stall ~1.5–2.6s client-side before their headers are written, with delays quantizing around 500ms. A candidate is the idle-socket validation change in 8.8.0 (nodejs/undici#5499), but I haven't bisected undici commits and don't yet have a standalone repro to report upstream.
Why this test in particular
It schedules a run with afterSeconds: 2 and assumes create + 500ms sleep + update-state + cancel complete inside that window. The dispatch stall pushes the cancel past it: the queue picks the run up at the 2s mark, the cancel finds a running run instead of a pending one (Attempted to cancel non-pending run in the log), and the abort surfaces in the queue worker as an error, so the run ends "error".
Alternatively the workflow could pin node-version: 26.5.0, but that freezes the whole matrix over one test and needs a manual unpin later.
One thing worth a maintainer's opinion: cancelling a run that has already started always ends it as "error" — ops.runs.cancel only sets "interrupted" while the run is still pending, and the queue worker classifies the cancel abort like any other failure. If "interrupted" is the intended status for a cancelled running run, I'm happy to follow up with that change separately.
Every CI run since Aug 1 fails on
Unit Tests (ubuntu-latest, 26.x)with the same single failure:This hits every branch, including dependabot PRs that only bump frontend deps. The last green run on
main(Jul 29) used Node 26.5.0; runners now resolve26.xto 26.5.1. Nothing in the repo changed.Reproduction
On unmodified
main,CI=1 npx vitest run tests/api.test.mtsinlibs/langgraph-api:The failure needs the full file's concurrent load; the test passes in isolation via
-t.Root cause
The regression is in undici, which jumped 8.7.0 → 8.9.0 inside the Node patch release. Cross-checking by overriding the SDK's fetch with standalone undici builds:
So the change landed in undici 8.8.0. A
diagnostics_channeltrace of the failingPOST /threads/:id/staterequest shows where the time goes:The server handled the request in 1ms (hono log). Under the suite's load (67 concurrent tests sharing one client, several long-lived SSE joins) requests intermittently stall ~1.5–2.6s client-side before their headers are written, with delays quantizing around 500ms. A candidate is the idle-socket validation change in 8.8.0 (nodejs/undici#5499), but I haven't bisected undici commits and don't yet have a standalone repro to report upstream.
Why this test in particular
It schedules a run with
afterSeconds: 2and assumes create + 500ms sleep + update-state + cancel complete inside that window. The dispatch stall pushes the cancel past it: the queue picks the run up at the 2s mark, the cancel finds a running run instead of a pending one (Attempted to cancel non-pending runin the log), and the abort surfaces in the queue worker as an error, so the run ends"error".Fixes
node-version: 26.5.0, but that freezes the whole matrix over one test and needs a manual unpin later.One thing worth a maintainer's opinion: cancelling a run that has already started always ends it as
"error"—ops.runs.cancelonly sets"interrupted"while the run is still pending, and the queue worker classifies the cancel abort like any other failure. If"interrupted"is the intended status for a cancelled running run, I'm happy to follow up with that change separately.