Pr/aoi builder

[sdrichards-bc]

AOI: validate requests, harden normalization, and return structured build results

Summary

This PR strengthens the AOI builder's existing normalization, part-building, inspection, and validation workflow. It converts a raw GeoDataFrame into a normalized AreaOfInterest and returns the AOI together with its validation findings and normalization audit report.

The main caller-facing change is the move from from_gdf(...) -> AreaOfInterest to build_from_request(...) -> AOIBuildResult. A completed build can contain validation errors; downstream callers must explicitly decide whether those errors prevent processing.

Release highlights:

• Validate and normalize request values before spatial processing, including a projected target CRS using metres.
• Handle nested geometry collections and standardize output to one active geometry column.
• Retain null dissolve groups and define which attributes survive each dissolve policy.
• Produce an immutable normalization report with populated policy counts and effects.
• Preserve part CRS and retained attributes, and add geometry-complexity metadata.
• Accept a valid custom metric CRS without requiring an EPSG code.
• Identify failed build stages through structured exceptions and logging.
• Preserve explicitly injected components, including objects that evaluate as false.

Motivation

The previous implementation already supported geometry repair, reprojection, three dissolve policies, overlap checks, singlepart AOIs, footprint and summed-part areas, and baseline validation. This release makes those capabilities more reliable and their contracts more explicit.

Area	Previous implementation	New implementation and benefit
Request configuration	Dataclass annotations without construction-time validation.	Validates types, required text, dissolve policies, field combinations, duplicates, CRS, and overlap policy. Invalid configuration fails earlier.
Geometry handling	Repair wrote to a literal geometry column; collection extraction examined only immediate members.	Retains the active geometry, standardizes its name, removes secondary geometry columns, and recursively extracts polygon components.
Grouped dissolve	Dissolved the full table using GeoPandas defaults.	Keeps only grouping fields and geometry, and uses dropna=False to retain null groups.
Normalization reporting	Mutated a dictionary; final report construction left policy-applied and policy-count fields at their defaults.	Uses a dedicated report builder, requires populated values, and freezes the completed report.
Overlap detection	Compared geometry pairs in nested loops.	Uses spatial-index candidates before checking intersection area. No performance benchmark is claimed.
AOI parts	Reconstructed each row as a new GeoDataFrame without explicitly carrying CRS metadata.	Slices the exploded GeoDataFrame and explicitly preserves active geometry, CRS, and retained attributes.
Inspection	Required a resolvable EPSG code; exposed basic area and bounds properties.	Allows crs_epsg=None and adds CRS text, counts, area ratio, vertices, and Z/M flags.
Build outcome	Returned AreaOfInterest; validation errors raised by default through raise_errors=True.	Returns AOIBuildResult for completed builds; callers evaluate validation findings.
Diagnostics	Separate exception types with no shared AOI base or consistent stage boundary.	Adds AOIError, stage-specific errors, chained AOIBuildError, and build summaries.

Key changes

Request validation

AOIRequest now validates input types before normalization. It:

• trims the AOI ID and name and rejects empty values;
• accepts a string target CRS, rejects invalid or geographic CRSs, and requires metre units;
• normalizes dissolve-mode case and whitespace, then checks the supported modes;
• accepts lists or tuples of dissolve-field strings and converts them to a trimmed tuple;
• removes blank dissolve fields and rejects duplicates after normalization;
• requires fields for by_fields and rejects non-empty fields for the other modes;
• requires allow_overlaps to be a boolean;
• exposes the parsed CRS through target_crs_obj, target_epsg, and is_projected.

The defaults remain EPSG:3005, full_union, no dissolve fields, and allow_overlaps=False.

AOIBuildRequest combines spec and raw_gdf and rejects None for either field. It does not perform complete runtime type validation; the normalizer checks whether the raw data is a usable GeoDataFrame.

Normalization

The normalizer checks raw input, cleans geometry, conforms CRS, applies policy, and checks the final geometry contract. Important changes include:

• Work on a copy of the input GeoDataFrame.
• Keep the active geometry and rename it to geometry when needed.
• Remove secondary columns with a geometry dtype and record their removal in report notes.
• Fail explicitly if a non-geometry column named geometry prevents the active-column rename.
• Remove null and empty geometry, repair invalid geometry with make_valid, and recursively retain polygon components from nested collections.
• Count individual non-empty polygon, line, and point components rather than collection containers.
• Reproject only when the input CRS differs from the requested metric projected CRS.
• Apply strict checks for valid, non-empty polygonal geometry before and after dissolve policy.

The three policies already existed. Their current output contracts are:

Policy	Geometry behavior	Retained attributes	Change from the previous implementation
full_union	Union all cleaned features into one normalized feature, which may be multipart.	Geometry only.	Basic policy behavior is retained.
by_fields	Dissolve within each configured group; null grouping values are retained.	Dissolve fields and geometry only.	Other attributes are deliberately removed; null groups are no longer excluded by the default grouping behavior.
preserve_features	Retain cleaned features without dissolving across rows.	Non-geometry attributes, plus the standardized active geometry.	Secondary geometry columns are removed and the active column name is standardized.

Overlap checks now use the spatial index to identify candidate pairs, then test positive intersection area. Shared edges and corners do not count as overlaps. Normalization uses the helper's default area tolerance of zero; no request-level tolerance setting has been added.

If overlaps remain and allow_overlaps=False, normalization fails and the builder raises AOIBuildError. This prohibition already existed; the error hierarchy and reporting are improved. Allowing overlaps permits them to remain but does not force a dissolve policy to preserve them.

Normalization reporting

AOINormalizationReportBuilder owns mutable reporting state during normalization. Its final AOINormalizationReport is frozen and records:

• input, cleaned, and output feature counts;
• input/output CRS and whether reprojection occurred;
• removed rows, repaired features, and retained/discarded geometry components;
• dissolve mode, fields, overlap policy, and whether policy application completed;
• feature counts and overlap state before and after policy application;
• whether policy resolved overlaps;
• notes about reprojection and geometry-column changes.

The reporter checks that required values have been populated. The old normalizer did not populate policy_applied, policy_input_feature_count, or policy_output_feature_count when constructing the final report; those values are now recorded.

The polygon_extract_*_feature_count names describe component counts, not input rows. Also, null_or_empty_removed_count includes rows removed after polygon extraction leaves no usable geometry. These counts should not be added together as if each represented a separate set of source rows.

AOI parts

AOIPartBuilder validates normalized input, explodes multipart geometry, and verifies that every resulting part is a valid single polygon. It constructs parts through AOIPart.from_gdf, preserving CRS and the attributes retained by the selected dissolve policy.

Part IDs now use zero-padded indexes, such as aoi_001_part_0001. They follow normalized/exploded row order; they are not persistent geometry identifiers and are not guaranteed to remain attached to the same geometry if input or dissolve ordering changes.

Each part adds vertex_count, has_z, and has_m, plus geometry and crs convenience properties. Vertex counts include exterior and interior rings, including each ring's repeated closing coordinate.

AOI inspection

AOIInspector now records:

• CRS string and optional EPSG code;
• unioned footprint area and summed parts area;
• parts-to-footprint area ratio;
• bounds, normalized feature count, and singlepart count;
• geometry type, total vertex count, and maximum vertices in one part;
• presence of Z or M coordinates.

overlay_area_ha is renamed to parts_area_ha; the underlying distinction from unique footprint area already existed. When overlapping features are preserved, parts area can exceed footprint area. The new ratio makes that difference easier to inspect.

A valid custom metric CRS may return crs_epsg=None. Consumers should use the CRS object or crs_string where possible and enforce an EPSG requirement only at a boundary that needs it.

Structured validation

AOIValidationResult.is_valid is derived from its issues instead of being stored independently. Both validation and build results expose has_errors, has_warnings, errors, warnings, and infos.

ValidationIssue normalizes severity to lowercase, codes to uppercase, and surrounding message whitespace. It rejects unsupported severity and blank codes or messages.

The current validator retains these checks:

Finding	Code	Severity
Disallowed overlaps indicated by the normalization report	OVERLAPS_PRESENT	Error
Null, empty, or non-polygon geometry removed	NULL_OR_NON_POLYGONS_REMOVED	Warning
Invalid geometry	INVALID_GEOMETRY	Error
Non-positive footprint or parts area	ZERO_AREA	Error
Recorded part count differs from the parts supplied	PART_COUNT_MISMATCH	Error
Part area is at or below 0.1 ha	ZERO_AREA_OR_SLIVER_PART	Error
Part area is at or above 10,000 ha	LARGE_PART	Error

The area thresholds and their error severity are unchanged. They report findings; they do not remove small parts or split large parts. Allowed overlaps are recorded in the normalization report without generating an overlap warning solely because they are allowed.

The latest validator no longer emits NO_GDF, NO_PROPERTIES, NO_PARTS, or NO_NORMALIZATION_REPORT. The normal build workflow establishes these inputs through earlier stages; direct calls to the validator must supply them. The uploaded README still lists these four checks and should be aligned with the implementation.

Build-result contract

The builder returns:

AOIBuildResult(
    aoi=...,
    validation=...,
    normalization_report=...,
)

Here, “success-only” means that construction completed. It does not mean that all validation rules passed.

Outcome	Behavior
Invalid request configuration or a missing build-request field	Raise AOIRequestError during request construction.
Failure within normalization, part building, inspection, or validation execution	Raise AOIBuildError; no result is returned.
Completed build with validation errors	Return AOIBuildResult with has_errors=True and is_valid=False.
Completed build with warnings but no errors	Return AOIBuildResult with has_warnings=True and is_valid=True.
Completed build with no validation errors	Return AOIBuildResult with is_valid=True.

Malformed untyped calls outside the declared API are not guaranteed to become AOIBuildError: request access and final result assembly occur outside _run_stage().

Exceptions and logging

AOI domain exceptions now inherit from AOIError. The hierarchy distinguishes request configuration, spatial data/CRS/geometry failures, stage failures, and top-level build failures.

AOIBuilder._run_stage() records stage and aoi_id on AOIBuildError and preserves the original exception chain. Stage names are normalization, part_building, inspection, and validation. root_cause() follows explicit __cause__ links.

Expected AOI failures receive concise error logs, with root tracebacks at debug level. Unexpected exceptions raised within a stage are logged with their traceback and wrapped. Build start/completion and validation summaries are logged by the builder; normalization, part, and inspection detail is available at debug level.

Constructor defaults now use explicit is not None checks. A supplied normalizer, inspector, validator, or part builder is retained even if it evaluates as false.

Public API and migration

This is a breaking caller-facing change.

Previous API or behavior	New API or migration action
builder.from_gdf(spec, raw_gdf)	builder.build_from_request(AOIBuildRequest(spec=spec, raw_gdf=raw_gdf))
Returns AreaOfInterest	Read the AOI from result.aoi.
raise_errors=True/False	Evaluate result.has_errors; explicitly choose whether to stop.
aoi.validation	result.validation
aoi.normalization_report	result.normalization_report
aoi.overlay_area_ha or properties.overlay_area_ha	aoi.parts_area_ha or properties.parts_area_ha
AOIValidationResult(is_valid=..., issues=...)	AOIValidationResult(issues=...); validity is computed.
Positional part_builder.build_parts(aoi_id, gdf)	Keyword-only part_builder.build_parts(aoi_id=aoi_id, gdf=gdf)
Part IDs ending in _part_1	IDs now end in _part_0001; update snapshots and references.
EPSG always available after inspection	Handle crs_epsg=None for custom metric CRSs.
Non-group attributes available after by_fields	Only grouping fields and geometry survive.
Null grouping values omitted by grouped dissolve	Null groups are retained.
Source geometry-column names/secondary geometry columns	Output has one active column named geometry.
Mutable report with defaulted fields	Report is frozen; direct construction requires all non-default fields. Use dataclasses.replace() for test variations.
AOIGeometryError / AOIGeometryTypeError	Removed; migrate relevant catches/imports to SpatialGeometryError and the builder boundary.
Minimal direct AOIPart / AOIProperties construction	Supply new required metadata fields or use the part builder/inspector in production. Update independent test fixtures.

To preserve the previous default behavior of stopping on validation errors, make that decision explicitly in the calling service:

import logging

import geopandas as gpd

from ast_engine.core.aoi import (
    AOIBuilder,
    AOIBuildRequest,
    AOIRequest,
    AreaOfInterest,
)
from ast_engine.core.aoi.exceptions import AOIValidationError

logger = logging.getLogger(__name__)


def prepare_analysis_aoi(raw_gdf: gpd.GeoDataFrame) -> AreaOfInterest:
    spec = AOIRequest(
        aoi_id="aoi_001",
        name="Example AOI",
        target_crs="EPSG:3005",
        dissolve_mode="full_union",
        allow_overlaps=False,
    )

    result = AOIBuilder().build_from_request(
        AOIBuildRequest(spec=spec, raw_gdf=raw_gdf)
    )

    for issue in result.warnings:
        logger.warning("%s: %s", issue.code, issue.message)

    if result.has_errors:
        messages = "; ".join(
            f"{issue.code}: {issue.message}" for issue in result.errors
        )
        # Application policy: stop before downstream spatial processing.
        raise AOIValidationError(messages)

    return result.aoi

AOIRequestError and fatal AOIBuildError propagate to the application's error handler. The explicit AOIValidationError above is raised by the calling service, not automatically by the new builder. A review/reporting workflow can instead retain the whole result and present its findings.

Files changed

All eight original Python files changed. Three Python files and a README were added; no source files were removed.

File	Change	Responsibility and usage notes
aoi_builder.py	Updated	Orchestrates stages, preserves injected dependencies, constructs AOIBuildResult, and provides stage error/logging boundaries.
models.py	Updated	Defines requests, AOI/part/properties models, immutable audit and validation models, and the build-result contract.
normalizer.py	Updated	Cleans and standardizes geometry, conforms CRS, applies policy, recursively extracts polygons, and checks output.
parts_builder.py	Updated	Creates one validated polygon per part and preserves retained attributes/CRS; public method arguments are keyword-only.
inspector.py	Updated	Computes footprint, parts-area ratio, counts, complexity, and optional EPSG metadata.
validator.py	Updated	Returns baseline quality findings. Stores an injected spatial_validator but does not use it yet.
exceptions.py	Updated	Adds a shared AOI hierarchy, stage metadata, and root-cause traversal; removes legacy geometry exception classes.
__init__.py	Updated	Removes duplicate imports and expands public exports to include build, part, property, report, and validation models.
normalization_reporter.py	Added	Accumulates normalization events and checks required values before creating a frozen audit report.
utils.py	Added	Shared GeoDataFrame/CRS checks, indexed overlap detection, and vertex/Z/M helpers.
constants.py	Added	Defines DEFAULT_CRS = "EPSG:3005" and DEFAULT_GEOM_FIELD = "geometry".
README.md	Added	Documents architecture, policies, usage, exceptions, logging, and planned validation. Some claims need the alignment noted below.

Testing

The supplied AOI suite contains eight test modules with 136 test functions before parameter expansion. It covers component behavior, mocked builder orchestration, real AOI-stage workflows using generated input, and the shared geometry factories used by those tests.

Behaviors exercised

• Request type validation, value normalization, dissolve-policy combinations, metric projected CRS requirements, and custom CRS metadata.
• Null/empty cleanup, invalid-geometry repair, recursive polygon extraction, geometry-column handling, and component-count reporting.
• All three dissolve policies, null grouping values, allowed/disallowed overlaps, and shared-edge/corner behavior.
• CRS transformation, footprint versus summed-part area, multipart splitting, retained attributes/dtypes, and ordered part IDs.
• Report completeness, immutable completed reports, and isolation of source data, sibling parts, and repeated runs.
• Stage ordering, forwarding of stage outputs, result assembly, and preservation of explicitly injected dependencies, including false-valued objects.
• Fatal versus returned validation outcomes, preserved exception causes, stage/AOI metadata, and prevention of later-stage execution after failure.
• Z-coordinate preservation and conditional M-coordinate preservation in part construction, plus metadata aggregation in inspection.

Verification before merge

• Confirm the expected AOI tests collect in the full repository environment.
• Run the suite against the PR commit and record the result above.
• Review any skips and identify functionality not exercised in that environment.
• Run affected downstream tests for the new AOIBuildResult contract and changed AOI fields.
• Reference or add targeted coverage for the remaining contract checks listed below.

Remaining coverage to confirm

The uploaded suite does not directly establish all items in the earlier checklist. Check for existing tests elsewhere in the repository before adding more:

• Duplicate dissolve-field rejection after whitespace normalization, including ("REGION", " REGION ").
• AOIBuildRequest rejection of missing spec or raw_gdf.
• ValidationIssue severity/code normalization, rejection of unsupported/blank values, and complete validation-result filtering behavior.
• Sliver-threshold behavior and controls immediately below/at/above the area thresholds. The real builder test already exercises LARGE_PART at the configured large-area limit; it does not cover the sliver rule or all boundary controls.
• Explicit removal of non-group attributes from by_fields output. Existing tests assert grouped geometry and keys, but do not directly assert that an extra non-group column is absent.
• Normalizer rejection of a conflicting non-geometry column named geometry, and rejection of the active geometry column as a dissolve key.

The supplied archive has no dedicated validator test module. Builder tests exercise actual large-part findings, and orchestration tests inject controlled findings to check the result contract. Those tests do not demonstrate complete coverage of every validator rule. If a separate validator suite exists, include it in the run and reference it here.

Reviewer focus

Pay particular attention to these compatibility and documentation details:

1. Caller validation gate: a returned result is not evidence that validation passed. Existing services (execution) must explicitly preserve their stop/continue policy.
2. Attribute and geometry-column changes: confirm consumers can handle grouped outputs containing only dissolve fields and geometry, retained null groups, and standardized geometry names.
3. Unused runtime dependency: validator.py imports SpatialValidator from ast_engine.core.validation.spatial_validation at runtime but only stores the supplied object. Standalone use of this archive requires that external module.
4. CRS and low-level entry points: the normal request/build path enforces metre units. The current metric check examines the first axis's unit name; broader custom-CRS support should validate both horizontal axes and their unit conversion factors.

Known limitations and follow-up work

The validator provides baseline findings only. MAX_VERTICES = 10_000 is declared but not enforced. The injected SpatialValidator is not invoked. Provincial/project footprint checks, expected-region or management-unit checks, boundary crossing, hole/donut checks, and configurable thresholds/severities remain future work.

Frozen dataclasses do not make contained GeoDataFrames immutable. AreaOfInterest is mutable, and part/build-result objects still contain mutable data. Downstream geometry edits can invalidate cached areas, bounds, counts, or validation findings; rebuild or recompute metadata when geometry changes.

Release-note summary

AOI preparation now validates configuration before processing, requires a projected target CRS using metres, and provides clearer build diagnostics. Geometry normalization handles nested collections, standardizes the active geometry column, and records its changes in an immutable audit report. Grouped dissolves retain null groups and return only their grouping fields and geometry. AOI parts preserve CRS and retained attributes, and inspection adds area ratios, geometry-complexity metadata, and support for custom metric CRSs without EPSG codes.

Migration required: replace from_gdf(...) with build_from_request(AOIBuildRequest(...)), read the AOI from result.aoi, and check result.has_errors before processing. Validation and normalization reports now belong to the build result; overlay_area_ha is renamed to parts_area_ha. Review grouped attributes, geometry-column assumptions, and part-ID formatting in downstream consumers.

Spatial-context and maximum-vertex validation remain future work.

Check List

• [Y] Code runs locally
• [N] Tests pass**
• [Y] New behavior has tests
• [Y] Documentation updated if behavior changed
• [Y] No secrets, credentials, or local-only paths committed
• [Y] Logging uses module-level loggers
• [Y] Exceptions and validation results follow engine convention

**Breaking change to existing tests outside of tests/unit/aoi. PR to resolve those issues and align with the updated aoi builder boundary.