Conversation
| /// Starts at a high offset to avoid collision with glide-core managed pool IDs | ||
| /// (which start at 1 and increment monotonically from POOL_REGISTRY::register_pool). | ||
| /// The two namespaces are disjoint as long as fewer than 2^62 glide-core pools are created. | ||
| static NEXT_POOL_ID: AtomicU64 = AtomicU64::new(1 << 62); // 4_611_686_018_427_387_904 |
There was a problem hiding this comment.
pool_allocate_id() is exposed to TypeScript as a number, but 2^62 is above JavaScript's Number.MAX_SAFE_INTEGER (2^53 - 1). At this magnitude adjacent fetch_add(1) results round to the same JS value (the ULP is 1024), so many ClientPool instances receive an indistinguishable poolId; their CLIENT_TO_POOL, monitor-handle, and discard-queue entries then overwrite or drain one another. Keep this allocator within the safe integer range (or expose the ID as a BigInt) and separate the two pool kinds without relying on an unsafe numeric offset.
There was a problem hiding this comment.
Resolved by the Node pool migration. pool_allocate_id() no longer exists — ClientPool.ts now calls createPool() which delegates to glide-core::register_pool(), same as Java, Go, and Python. No separate TS-side pool ID is needed.
| @@ -227,20 +234,22 @@ pub unsafe extern "C" fn glide_pool_create( | |||
| if pool.state.load(AtomicOrdering::Acquire) != POOL_RUNNING { | |||
| return; | |||
There was a problem hiding this comment.
If the pool is destroyed while this client is being created, this branch returns without releasing adapter_ptr. create_pool_client() has already transferred an Arc<ClientAdapter> into the raw pointer and get_pool_clients() is populated only below, so glide_pool_destroy() has no way to find/drop it. The same early-return shape exists in the two on-demand creation paths below. Reconstruct and drop the raw Arc before returning when POOL_RUNNING is false.
There was a problem hiding this comment.
Fixed. All three early-return paths after create_pool_client() now reconstruct and drop the Arc before returning:
unsafe { drop(Arc::from_raw(adapter_ptr as *const ClientAdapter)); }| // Guard arms immediately on task entry — flag was already set true before spawn. | ||
| // This ensures the flag is cleared on every exit path: normal completion, | ||
| // early ?-return inside the inner async block, and task cancellation. | ||
| let _unmark_guard = UnmarkOnDrop(pre_blocking_arc.clone()); |
There was a problem hiding this comment.
A single AtomicBool plus one UnmarkOnDrop per request is not safe when requests overlap on the same multiplexed pooled client. For example, two concurrent BLPOPs both store true; when the first future completes, its guard stores false while the second is still blocking, allowing the abandon monitor to reclaim the client. Node and FFI have the same last-writer-wins behavior. Track an active-blocking count (increment before dispatch, decrement on drop, and let the monitor test count > 0) rather than independently clearing a shared boolean.
There was a problem hiding this comment.
Safe by design: the pool enforces exclusive borrow — only one caller holds a pooled client at a time via try_acquire/release_client_async. Concurrent blocking commands on the same pooled client are not possible through the pool API. This invariant is documented in the BLOCKING_FLAG_REGISTRY doc comment.
881aee4 to
a9bcb70
Compare
a9bcb70 to
27c3a48
Compare
27c3a48 to
32b6dea
Compare
32b6dea to
b5034ab
Compare
b5034ab to
07539a7
Compare
07539a7 to
5a51afc
Compare
d8e608d to
5dcd472
Compare
…ssue #6971) Signed-off-by: affonsov <67347924+affonsov@users.noreply.github.com>
…ace (#6971) The abandon monitor can reclaim a pooled client parked on a blocking command (BLPOP, XREAD BLOCK, etc.) because mark_client_blocking() used try_lock() on the pool TokioMutex. Under lock contention the flag was never set, so the monitor treated the client as idle-abandoned. Fix: at borrow time, try_acquire() now returns the Arc<AtomicBool> for the client alongside its client_id. jni_pool stores this Arc in a new lockless DashMap (handle_id -> Arc<AtomicBool>). The JNI dispatch path sets is_blocking directly via .store() — no pool lock, no try_lock, no race. mark_client_blocking() and get_client_blocking_flag() are retained for the non-Java FFI path (ffi/src/lib.rs) which still uses try_lock(). Java JNI callers now use the lock-free Arc path exclusively. Adds: JNI_POOL_BLOCKING_FLAG_MAP populated at borrow time. Signed-off-by: affonsov <67347924+affonsov@users.noreply.github.com>
…e pool to core (#6971, #6887) Fixes #6971: BLOCKING_FLAG_REGISTRY in glide-core eliminates the try_lock race. Fixes #6887: Node ClientPool.ts now uses glide-core::ClientPool via createPool/ poolTryAcquire/poolAcquireBlocking/poolRelease/poolMetrics/poolDestroy, removing all TS-managed idle/active/waiter state. Key changes: - Add BLOCKING_FLAG_REGISTRY and CLIENT_TO_POOL to glide-core/src/pool.rs - Extract create_handle_for_client() and run_worker_message() from create_direct_client() in node/rust-client/src/lib.rs - Add closeForPoolRelease() to GlideClientHandle (stops worker without removing from scope registry, enabling safe client reuse across pool cycles) - Add pool_build_handle() to node/rust-client/src/pool.rs (JIT wraps a pool-acquired client in a fresh N-API handle with per-acquire wake callback) - Remove CLIENT_ACTIVITY, DISCARDED_CLIENTS, TS_MONITOR_HANDLES, NEXT_POOL_ID, pool_allocate_id(), pool_register_client(), pool_unregister_client(), pool_start_monitor(), pool_stop_monitor(), start_ts_abandon_monitor(), refresh_ts_activity() from node/rust-client/src/pool.rs - Add serializeConnectionRequest(), createClientFromHandle() to BaseClient - Add serializeConfig(), createFromHandle() to GlideClient and GlideClusterClient - Rewrite ClientPool.ts to delegate all pool state to glide-core - C#, PHP, Ruby and future clients get the fix for free via get_blocking_flag() Signed-off-by: affonsov <67347924+affonsov@users.noreply.github.com>
Signed-off-by: affonsov <67347924+affonsov@users.noreply.github.com>
… notification race The refactoring of create_direct_client moved Client::new onto the glide-core multi-thread runtime, creating a window where PubSub push notifications (subscription confirmations) arrived before the push listener task was running. Messages buffered silently with no wake_callback fired, causing PubSub tests to time out. Restore the original ordering: Client::new runs inside spawn_pinned so the push listener is set up atomically with the connection. create_handle_for_client is retained for the pool path only, where clients are pre-built and never have PubSub subscriptions. Signed-off-by: affonsov <67347924+affonsov@users.noreply.github.com>
…Sub race) - Move create_test_glide_client before mod tests to satisfy clippy::items-after-test-module - Fix broken rustdoc intra-doc links in node/rust-client - Fix UnmarkOnDrop scope for MIRI compilation in ffi/src/lib.rs - Move deferred.resolve after spawn_local(push listener) in create_direct_client to eliminate race where cluster-mode push notifications arrive before the listener task is scheduled Signed-off-by: affonsov <67347924+affonsov@users.noreply.github.com>
5dcd472 to
e02732b
Compare
Summary
This PR fixes a race condition in the
ClientPoolabandon monitor (#6971) and migrates the Node.js pool to useglide-core::ClientPooldirectly (#6887), removing the TypeScript-side state duplication that existed since the pool was first introduced.Bug (#6971): The abandon monitor could silently reclaim a pooled client executing a blocking command (
BLPOP,XREAD BLOCK, etc.), causing the in-flight future to fail with aClosingException. Root cause:mark_client_blocking()usedtry_lock()on the poolsTokioMutex`. Under lock contention the flag was never set, and the monitor treated the client as idle-abandoned.Refactor (#6887): Since the pool was introduced in #6338, Node
sClientPoolmaintained its own TypeScript-side idle stack, active map, waiter queue, and abandon monitor — duplicating state thatglide-core::ClientPoolalready tracks for Java, Go, and Python. This PR removes that duplication by wiringClientPool.ts` to delegate all pool state to Rust.Issue link
Closes #6971 — ClientPool abandon monitor can reclaim a client parked on a blocking command
Closes #6887 — Node: Refactor pool state management into Rust core
Features / Behaviour Changes
ClientPoolabandon monitor now correctly skips clients executing blocking commands under all conditions, including high pool mutex contention, for Java, Go, Python, and Node.mark_client_blocking()(which usedtry_lock) is removed. All bindings use the lock-freeget_blocking_flag(client_id)API fromglide-core.ClientPool.tsnow delegates all pool state toglide-core::ClientPoolvia the standard Rust pool APIs (createPool,poolTryAcquire,poolAcquireBlocking,poolRelease,poolMetrics,poolDestroy). The TypeScript-side idle stack, active map, waiter queue, and abandon monitor are removed.create_handle_for_client()extracted fromcreate_direct_client()innode/rust-client/src/lib.rs. Pool-managed Rust connections are wrapped as fullGlideClientHandleN-API objects at acquire time, preserving the complete command API surface for TypeScript callers.glide_core::pool::get_blocking_flag(client_id)in the dispatch path is sufficient.ClientPoolis unchanged.Implementation
Root cause and fix (
glide-core/src/pool.rs)mark_client_blocking()acquired the poolsTokioMutexviatry_lock(). If the lock was held at the moment a blocking command was dispatched,try_lock()returnedErrand theis_blockingflag was silently never set. The abandon monitors next scan sawis_blocking = falseand discarded the client.Fix: a
BLOCKING_FLAG_REGISTRY: DashMap<u64, Arc<AtomicBool>>is added toglide-core/src/pool.rs. TheArcis created when a client enters the pool (add_client) and is shared betweenPooledClient.is_blocking(read by the monitor) and the registry (read by bindings viaget_blocking_flag(client_id)). Each binding callsarc.store(true, Ordering::Release)synchronously on the calling thread, before anyspawn()or channel send. AnUnmarkOnDropRAII guard inside the async task clears the flag on all exit paths.Node pool migration (
node/rust-client/src/pool.rs,node/src/ClientPool.ts)create_handle_for_client(client, push_rx, wake_tsfn, inflight_limit, provided_client_id)is extracted fromcreate_direct_client(). It accepts aprovided_client_id: Option<u64>— whenSome(id)is given (pool path), the handlesclient_idmatches the pools tracking key; whenNone(standalone path), a new ID is allocated fromNEXT_CLIENT_ID. This ensurespoolRelease(pool_id, client.getClientId())finds the correctin_useentry.pool_build_handle(client_id, wake_callback)retrieves the existing RustClientfrom the glide-core scope registry and wraps it as a fullGlideClientHandlewith a fresh worker thread and response buffer — no new connection is opened.close_for_pool_release()stops the worker thread without unregistering from the scope registry, so the same underlying Rust connection can be reused across pool borrow cycles.On-demand connection creation is triggered in both
pool_try_acquireandpool_acquire_blockingvia a sharedmaybe_spawn_on_demand_creation()helper, preserving themaxSizepool growth behaviour.Key attention areas for reviewers
glide-core/src/pool.rs—BLOCKING_FLAG_REGISTRYandCLIENT_TO_POOLlifecycle (add on creation, reset-to-false on return-to-idle, remove on permanent discard and destroy)java/src/lib.rs—pre_blocking_arcset beforeget_runtime().spawn();UnmarkOnDropguard handles all exit paths including routing errors via?and task cancellationffi/src/lib.rs— same pre-spawn pattern;_arc_guardfor single-command pathnode/rust-client/src/lib.rs—create_handle_for_clientextraction;close_for_pool_releasenode/rust-client/src/pool.rs—maybe_spawn_on_demand_creation;pool_build_handle; on-demand creation in both acquire paths; discard drainingnode/src/ClientPool.ts— all TS-side state removed; delegates to Rust pool APIsLimitations
abandon_monitor_skips_blocking_clients,abandon_monitor_evicts_non_blocking_abandoned_clients) cover the core invariant without a Valkey server.pool_acquire_blockinguses a 5 ms polling loop rather than a condvar wait (pre-existing limitation, not introduced by this PR).Testing
New unit tests added (
glide-core/src/pool.rs— no Valkey server required):abandon_monitor_skips_blocking_clients— regression test for Java: ClientPool abandon monitor can reclaim a client parked on a blocking command (best-effort is_blocking marking race) #6971: inserts a client intopool.in_usewith backdatedborrowed_at, setsis_blocking = trueunder pool mutex contention, runs the monitor for 3× the abandon timeout, asserts the client is not evicted.abandon_monitor_evicts_non_blocking_abandoned_clients— companion: same setup withis_blocking = false, asserts the client is evicted, confirming the monitor is active.Verified:
cargo clippyclean across all 4 crates (glide-core, java/glide-rs, ffi, node/rust-client)npm run build:ts) cleanChecklist