Validate id column uniqueness in MariaDBSchemaValidator - #6708
Open
dev-xong wants to merge 1 commit into
Open
Conversation
INSERT ... ON DUPLICATE KEY UPDATE only behaves as an upsert when the id column is backed by a single-column PRIMARY KEY or UNIQUE constraint. Without that, duplicate rows kept getting inserted instead of updated, and this misconfiguration went undetected by schema validation. Add isSoleUniqueOrPrimaryKeyColumn, which queries INFORMATION_SCHEMA.STATISTICS to verify the id column is the sole column of a PRIMARY KEY or UNIQUE index, and fail schema validation when it isn't. Add MariaDBSchemaValidatorIT covering sole/composite PRIMARY KEY and UNIQUE constraint scenarios. Signed-off-by: dev-xong <songyoona1209@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the TODO left in
MariaDBSchemaValidator#validateTableSchema(ensure id is a primary key for batch update).MariaDBVectorStore's batch upsert usesINSERT ... ON DUPLICATE KEY UPDATE, which only behaves as "insert or update on duplicate" when theidcolumn is backed by a single-column PRIMARY KEY or UNIQUE constraint. Without that, duplicate rows kept getting inserted instead of updated. This change catches that misconfiguration during schema validation.Changes
isSoleUniqueOrPrimaryKeyColumn, which queriesINFORMATION_SCHEMA.STATISTICSto check whether theidcolumn is the sole column of a PRIMARY KEY or UNIQUE index.validateTableSchemathat throwsIllegalStateExceptionwhen this condition isn't met.Why not COLUMNS.COLUMN_KEY?
An earlier approach considered simply checking whether
INFORMATION_SCHEMA.COLUMNS.COLUMN_KEYequals'PRI', but that was rejected for two reasons:ON DUPLICATE KEY UPDATEworks the same way for UNIQUE constraints as for the PRIMARY KEY, so checking only'PRI'would wrongly reject valid schemas.COLUMN_KEYreports'PRI'/'UNI'even when a column is merely part of a composite PRIMARY KEY/UNIQUE constraint. For example, in a table withPRIMARY KEY (id, tenant_id),idshowsCOLUMN_KEY = 'PRI', even thoughidalone doesn't guarantee uniqueness — this case should fail validation but wouldn't have.To address this, the check was rewritten as a query against
INFORMATION_SCHEMA.STATISTICSthat directly verifies "does a unique index made up of exactly this one column exist?"NON_UNIQUE = 0: includes both PRIMARY and UNIQUE indexesHAVING COUNT(*) = 1: only single-column indexes (excludes composite keys)SUM(COLUMN_NAME = ?) = 1: confirms the matched column is theidunder validationNotes
idFieldNameis already a quoted identifier, so a separate raw (unquoted) column name (rawIdColumnName) is kept around to compare againstINFORMATION_SCHEMA.STATISTICS.COLUMN_NAME, which returns raw names.Behavior change
When schema validation is enabled (
schemaValidation = true), existing tables without a single-column PRIMARY KEY or UNIQUE constraint onidwill now fail schema validation at startup instead of silently duplicating rows at runtime during batch upsert.Tests
Added 5 scenarios to
MariaDBSchemaValidatorITidis the sole PRIMARY KEY → passes (validatesSuccessfullyWhenIdIsSolePrimaryKey)idis a sole UNIQUE column → passes (validatesSuccessfullyWhenIdIsSoleUniqueColumn)idhas no unique constraint at all → throws (rejectsWhenIdHasNoUniqueConstraint)idis part of a composite PRIMARY KEY → throws (rejectsWhenIdIsPartOfCompositePrimaryKey)idis part of a composite UNIQUE index → throws (rejectsWhenIdIsPartOfCompositeUniqueConstraint)