Improve concurrent database access by JanJakes · Pull Request #361 · WordPress/sqlite-database-integration

JanJakes · 2026-04-13T14:59:33Z

Summary

Reduces lock contention under concurrent access by avoiding unnecessary write locks for read-only queries.

Skip the wrapper transaction for SELECT statements. A standalone SELECT translates to a single SQLite query, so the implicit autocommit transaction provides sufficient consistency — no explicit BEGIN/COMMIT needed.
Use deferred BEGIN for other read-only wrapper transactions. SHOW and DESCRIBE queries still use a wrapper transaction (they may translate to multiple SQLite queries), but now use a deferred BEGIN (SHARED lock) instead of BEGIN IMMEDIATE (RESERVED lock).

Background

The wrapper transaction previously used BEGIN IMMEDIATE for all queries, including read-only SELECTs. This acquired a RESERVED (write) lock on every query — but SQLite only allows one RESERVED lock at a time. Under concurrent load (e.g., multiple PHP-FPM workers handling requests simultaneously), all processes competed for the single write lock. When the 10-second busy timeout was exceeded, SQLite returned SQLITE_BUSY ("database is locked", error 5).

SHARED locks, acquired by a deferred BEGIN, are compatible with RESERVED locks — multiple readers can proceed concurrently alongside an active writer.

Fixes #318

1. Use deferred BEGIN (SHARED lock) instead of BEGIN IMMEDIATE (RESERVED lock) for read-only wrapper transactions (SHOW, DESCRIBE). 2. Skip the wrapper transaction entirely for SELECT statements. Fixes: #318

bgrgicak · 2026-04-16T04:18:13Z

+			 * [GRAMMAR]
+			 * simpleStatement: selectStatement | showStatement | utilityStatement | ...
+			 */
+			$statement_node = $child_node->get_first_child_node();


Should we wrap this in if ( null === $child_node ) because it seems like $child_node can be null?

👍 At this point in the AST, there is probably always a child node, but it's a good idea to add it, also for consistency with the code above. Fixed in 14dc77a.

bgrgicak · 2026-04-16T04:22:34Z

+/**
+ * Tests for concurrent access to the same SQLite database file.
+ */
+class WP_SQLite_Driver_Concurrency_Tests extends TestCase {


Should we add a test for showStatement queries to confirm that it uses the correct BEGIN?

👍 Added tests for SHOW and DESCRIBE in c559b3a.

JanJakes · 2026-04-16T10:32:02Z

@bgrgicak Thanks! Addressed both issues.

Pin BEGIN IMMEDIATE for write statements (INSERT, UPDATE, DELETE, REPLACE, CREATE/ALTER/DROP/TRUNCATE TABLE) so a future regression that accidentally flags a write as read-only — downgrading the lock to a deferred BEGIN — is caught immediately. Mirror the existing SELECT two-connection contention test for SHOW and DESCRIBE, proving they succeed under a concurrent RESERVED lock instead of only asserting the logged BEGIN string. Factor out a couple of small helpers to remove the repeated driver and query-logger setup boilerplate.

## Release `3.0.0-rc.1` Version bump and changelog update for release `3.0.0-rc.1`. **Changelog draft:** * Improve concurrent database access ([#361](#361)) * Remove legacy SQLite driver ([#358](#358)) **Full changelog:** v2.2.23...release/v3.0.0-rc.1 ## Next steps 1. **Review** the changes in this pull request. 2. **Push** any additional edits to this branch (`release/v3.0.0-rc.1`). 3. **Merge** this pull request to complete the release. Merging will automatically build the plugin ZIP and create a [GitHub release](https://github.com/WordPress/sqlite-database-integration/releases). > [!NOTE] > This is a **pre-release**. It will not be deployed to [WordPress.org](https://wordpress.org/plugins/sqlite-database-integration/).

@Unlink

…" on connect) (#443) Related to STU-1821 ## What changed Every WordPress request runs `SET SESSION sql_mode = …` at bootstrap (via `wpdb::set_sql_mode()`). This statement changes only session state and writes nothing to the database, but the driver acquires an SQLite write lock for it. Under concurrent access, this makes an otherwise read-only request fatal with `database is locked` (`SQLITE_BUSY`) the moment any other connection holds the write lock. This PR fixes the two places the `SET` path takes a write lock (`WP_PDO_MySQL_On_SQLite`): 1. **The wrapper transaction.** `query()` opens `BEGIN IMMEDIATE` (a RESERVED/write lock) for every non-`SELECT` statement, and `SET` was treated as a write. → `SET` is now detected as read-only, so it opens a deferred `BEGIN` (SHARED) instead — the same treatment `SHOW`/`DESCRIBE` already receive (#361). 2. **The empty-result builder.** `create_result_statement_from_data()` fabricates a 0-column result with a no-op `INSERT … WHERE FALSE` against a regular table. SQLite acquires a write lock for *any* DML, even one that inserts zero rows, so this re-took a write lock right after the wrapper committed. → The 0-column statement is now built with a no-op `INSERT` into a connection-private **`TEMP` table**, whose temp database has no shared lock. Both changes are required: with only (1), the fatal simply moves from the `BEGIN IMMEDIATE` to the no-op `INSERT`. ## Why This is a direct follow-up to #361 (which stopped `SELECT`/`SHOW`/`DESCRIBE` from taking write locks and closed #318). `SET` was left on the write path. Because `set_sql_mode()` runs on every connection, the impact is broad: a single slow request holding a write transaction (a background WooCommerce/Jetpack write, an import, a long render on a heavy site) makes every concurrent request fail at connect, not only concurrent writers. SQLite is single-writer by design, but statements that perform no writes should never contend for the write lock. Journal mode does not address this on its own: it reproduces identically under both `DELETE` and `WAL` (WAL still serialises writers). ## Reproduction Deterministic, single process, two driver connections to one file: ```php $path = tempnam( sys_get_temp_dir(), 'sqlite' ); @Unlink( $path ); $make = fn() => new WP_SQLite_Driver( new WP_SQLite_Connection( array( 'path' => $path ) ), 'db' ); // Build both connections first, so B's schema setup doesn't run under the lock. $a = $make(); $a->query( 'CREATE TABLE t ( id INTEGER PRIMARY KEY )' ); $b = $make(); $b->execute_sqlite_query( 'PRAGMA busy_timeout = 1500' ); // fail fast on a lock // A holds a write lock. $a->query( 'START TRANSACTION' ); $a->query( 'INSERT INTO t ( id ) VALUES ( 1 )' ); // B runs the connect-time statement. It writes nothing, so it must not block. $b->query( "SET SESSION sql_mode = 'NO_ENGINE_SUBSTITUTION'" ); // Before this PR: throws "database is locked". After: returns immediately. ``` **Before:** `WP_SQLite_Driver_Exception: … database is locked`, first in `begin_wrapper_transaction()`, or after fixing only (1), in `create_result_statement_from_data()`. **After:** the `SET` returns immediately while A still holds its write lock. This also reproduces end-to-end in a multi-worker WordPress install (eg. PHP-FPM with more than one workers) under concurrent requests, where one slow request holding a write transaction makes concurrent page loads fatal at `set_sql_mode`. ## Tests Two regression tests added to `WP_SQLite_Driver_Concurrency_Tests`: - `testSetQueryOpensReadOnlyTransaction`: `SET` opens `BEGIN`, not `BEGIN IMMEDIATE`. - `testSetQuerySucceedsWhileAnotherConnectionHoldsWriteLock`: with another connection holding `BEGIN IMMEDIATE` and `timeout = 0`, `SET` completes instead of throwing `SQLITE_BUSY`. Both fail on `trunk` and pass with this change. ## Notes / risk - **The 0-column contract is preserved.** `WP_SQLite_Driver::query()` returns `fetchAll()` when `columnCount() > 0`, otherwise `rowCount()`. The replacement empty-result statement keeps `columnCount() === 0`, so writes and `SET` still return their affected-row count (not an empty array). Verified: running a representative `CREATE`/`INSERT`/`SELECT`/`COUNT`/`UPDATE`/`DELETE`/`SET` sequence produces byte-identical results before and after. - **Bonus:** real writes (`INSERT`/`UPDATE`/`DELETE`) also fabricate their empty result through this path, which previously meant a second, redundant write-lock acquisition after the real write. That redundant lock is now gone, slightly reducing lock pressure for all writes. - **Temp-table lifetime.** The `TEMP` table is created once per connection (guarded by a flag); `TEMP` tables live for the connection's lifetime. A reviewer preferring maximum robustness over the micro-optimisation could drop the flag and always issue the idempotent `CREATE TEMP TABLE IF NOT EXISTS`. --------- Co-authored-by: Adam Zieliński <adam@adamziel.com>

Improve concurrent database access

752a8c4

1. Use deferred BEGIN (SHARED lock) instead of BEGIN IMMEDIATE (RESERVED lock) for read-only wrapper transactions (SHOW, DESCRIBE). 2. Skip the wrapper transaction entirely for SELECT statements. Fixes: #318

JanJakes marked this pull request as ready for review April 13, 2026 15:29

JanJakes requested review from a team and bgrgicak April 14, 2026 14:42

bgrgicak reviewed Apr 16, 2026

View reviewed changes

JanJakes added 2 commits April 16, 2026 12:21

Guard against null child node in read-only detection

14dc77a

Add tests for SHOW and DESCRIBE using a deferred BEGIN transaction

c559b3a

JanJakes requested a review from bgrgicak April 16, 2026 10:31

bgrgicak approved these changes Apr 17, 2026

View reviewed changes

JanJakes merged commit 2224d1a into trunk Apr 17, 2026
16 checks passed

JanJakes deleted the db-locked-error branch April 17, 2026 12:36

JanJakes mentioned this pull request Apr 17, 2026

Release 3.0.0-rc.1 #364

Merged

gcsecsey mentioned this pull request Jun 29, 2026

Don't take a write lock for SET statements (fixes "database is locked" on connect) #443

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improve concurrent database access#361

Improve concurrent database access#361
JanJakes merged 4 commits into
trunkfrom
db-locked-error

JanJakes commented Apr 13, 2026

Uh oh!

bgrgicak Apr 16, 2026

Uh oh!

JanJakes Apr 16, 2026

Uh oh!

bgrgicak Apr 16, 2026

Uh oh!

JanJakes Apr 16, 2026

Uh oh!

JanJakes commented Apr 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

JanJakes commented Apr 13, 2026

Summary

Background

Uh oh!

bgrgicak Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

JanJakes Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

bgrgicak Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

JanJakes Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

JanJakes commented Apr 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants