perf(persistence): bound JDBC hasChildren existence check with LIMIT#4973
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves performance in the relational JDBC persistence layer by turning an unbounded “fetch all children rows” existence check into a bounded query using LIMIT 1, reducing unnecessary row transfer/materialization when determining whether a parent entity has any children.
Changes:
- Add a new
QueryGenerator.generateSelectQuery(..., int limit)overload to append aLIMITclause for bounded existence checks. - Update
JdbcBasePersistenceImpl.hasChildrento use the new overload withlimit = 1. - Add a unit test asserting the generated SQL includes
LIMIT 1and preserves parameter binding order.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/QueryGenerator.java | Adds a LIMIT-capable SELECT generator overload and threads limit through the internal builder. |
| persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/JdbcBasePersistenceImpl.java | Uses LIMIT 1 for hasChildren to bound result size for existence checks. |
| persistence/relational-jdbc/src/test/java/org/apache/polaris/persistence/relational/jdbc/QueryGeneratorTest.java | Adds a test verifying LIMIT 1 is appended and parameters are ordered correctly. |
JdbcBasePersistenceImpl.hasChildren generated a SELECT over all entity columns with no row limit and then materialized the entire ResultSet into a list, only to return whether it was empty. Checking whether a parent has children therefore fetched, transferred, and deserialized every child row in full, including the large properties and internal_properties JSON columns. This path runs when dropping a catalog or a namespace, so dropping a namespace holding many tables loaded all of them into memory just to decide NAMESPACE_NOT_EMPTY, adding needless latency and allocation and risking OOM on large namespaces. Add a LIMIT-aware generateSelectQuery overload in QueryGenerator and use it from hasChildren with a limit of 1, mirroring the existing generateEntityTableExistQuery pattern. LIMIT is standard SQL accepted by all supported backends (PostgreSQL, CockroachDB, H2). Behavior is unchanged: the method still returns true when at least one child exists and false otherwise. Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
anxkhn
force-pushed
the
perf/jdbc-haschildren-limit
branch
from
July 16, 2026 06:14
dfb7e53 to
d8067a0
Compare
Contributor
Author
|
Pushed a revision addressing the latest review/CI feedback. |
1 similar comment
Contributor
Author
|
Pushed a revision addressing the latest review/CI feedback. |
dimas-b
approved these changes
Jul 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JdbcBasePersistenceImpl.hasChildrenonly needs to know whether a parent entityhas at least one child, but it does far more work than that. It builds a SELECT
over every entity column with no row limit, then hands the result to
DatasourceOperations.executeSelect, which collects the entireResultSetintoa list (
stream.forEach(results::add)) before the method simply returnsresults != null && !results.isEmpty(). Every matching child row is thereforefetched, transferred over the wire, and deserialized in full, including the
potentially large
propertiesandinternal_propertiesJSON columns, only to bediscarded.
This path runs when dropping a catalog or a namespace
(
AtomicOperationMetaStoreManager.hasChildren). When dropping a namespace thecheck matches all child types (tables and views included), so dropping a
namespace that holds many tables loads every one of them into memory just to
decide
NAMESPACE_NOT_EMPTY. That adds needless latency and allocation and risksOutOfMemoryErroron large namespaces. The JDBC layer already has a boundedexistence-check pattern next door:
QueryGenerator.generateEntityTableExistQueryuses
LIMIT 1for the table-level check.This change bounds the existence check:
LIMIT-awaregenerateSelectQuery(projections, tableName, whereClause, int limit)overload in
QueryGenerator. It mirrors the existing 4-arg public overload andthreads an optional limit through a new private builder, so every existing caller
keeps emitting no
LIMIT(the previous private builder now delegates withnull).The
LIMITis appended after anyORDER BY, which is valid clause order for allsupported backends. The shared builder rejects non-positive limits.
hasChildrenwith a limit of1. Behavior is unchanged: with atmost one row returned,
!results.isEmpty()still means "has at least one child",so the method still returns
truewhen a child exists andfalseotherwise.LIMITis standard SQL accepted by all supported backends (PostgreSQL,CockroachDB, H2). The projection is intentionally left unchanged:
ModelEntityreads every column when converting a row, so narrowing the projection would need
a separate converter and a larger change.
LIMIT 1alone removes the full scan,transfer, and materialization.
I kept the projection full and the interface surface additive rather than changing
any existing method signature, per the project's evolution guidelines.
Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(if needed)