Skip to content

[FEATURE]: Unify parsing semantics for geospatial checks #1513

Description

@ghanse

Is there an existing issue for this?

  • I have searched the existing issues

Problem statement

Spark 4.2 introduced native geometry and geography types for DataFrames. DQX offers a module for validating properties of geospatial data. However, this module does not:

  • Consistently accept and handle geometry and geography columns as check function inputs
  • Consistently generate metadata when checks are skipped due to invalid WKB/WKT/GeoJSON parsing; These currently appear in results as NULL when the input values are invalid

Proposed Solution

  • Update the geospatial check functions to allow geometry and geography columns
  • Create metadata messages when checks are skipped because the input WKB/WKT/GeoJSON values are invalid

Additional Context

These changes were introduced for is_geo_within_distance in #1510 . This issue expands that work to other geospatial check functions.

Activity

  1. added
    enhancementNew feature or request
    DQX CoreFeature/Bug Related to DQX Core functionality
    on Sep 6, 2026
  2. self-assigned this
    on Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

DQX CoreFeature/Bug Related to DQX Core functionalityenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions