Skip to content

Add exact-fit strict-inequality/range aggregate check functions (is_aggr_greater_than, is_aggr_less_than, is_aggr_in_range, is_aggr_not_in_range) #1495

Description

@Christopher-Lawford

Context

Follow-up from review on PR #1485 (ODCS type: library quality metric support): comment.

DQX's type: library data-contract rule generator maps ODCS quality metrics (rowCount, nullValues, missingValues, invalidValues, duplicateValues) onto DQX aggregate checks where possible. Four of the eight ODCS threshold fields have no exact-fit DQX aggregate check today, so the generator falls back to a dataset-level sql_query check instead:

  • mustBeGreaterThan (metric > X)
  • mustBeLessThan (metric < X)
  • mustBeBetween (lo < metric < hi, both bounds exclusive per ODCS)
  • mustNotBeBetween (metric <= lo OR metric >= hi, per ODCS's exclusive-bounds semantics)

Proposal

Add native DQX check functions for these, so the data-contract generator (and any other caller) can use a structured, exact-fit check instead of a raw SQL string:

  • is_aggr_greater_than(column, limit, aggr_type=..., ...)
  • is_aggr_less_than(column, limit, aggr_type=..., ...)
  • is_aggr_in_range(column, min_limit, max_limit, aggr_type=..., ...) (exclusive bounds)
  • is_aggr_not_in_range(column, min_limit, max_limit, aggr_type=..., ...) (exclusive bounds)

These would mirror the existing is_aggr_equal / is_aggr_not_equal / is_aggr_not_less_than / is_aggr_not_greater_than family in check_funcs.py (same aggr_type, group_by, row_filter, tolerance parameters, etc.), and the DataContractRulesGenerator in contract_rules_generator.py could then replace its sql_query fallback for these four ODCS threshold fields with the new structured checks.

Out of scope for this issue

The generated sql_query fallback that's in place today is reviewed and correct — this is a nice-to-have follow-up to make the generated rules more structured, not a bug fix.

Activity

  1. added
    enhancementNew feature or request
    DQX CoreFeature/Bug Related to DQX Core functionality
    on Sep 6, 2026
  2. amising6 commented on Sep 18, 2026

    @amising6
    Contributor

    Hi @ghanse, I’d be interested in picking this up if it’s still available. I can add the four aggregate comparison checks following the existing patterns, cover the exclusive-boundary behavior with tests, and update the data-contract generator to use them instead of the SQL fallback.

    Is anyone already working on this, and would you prefer one PR for the checks and generator updates, or separate PRs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    DQX CoreFeature/Bug Related to DQX Core functionalityenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions