EXTENDS validation with feasibility-pruned partition search - #105
Open
ericprud wants to merge 4 commits into
Open
EXTENDS validation with feasibility-pruned partition search#105ericprud wants to merge 4 commits into
ericprud wants to merge 4 commits into
Conversation
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
ericprud
force-pushed
the
extends-validation
branch
3 times, most recently
from
July 28, 2026 00:13
97beb1a to
e6dae0d
Compare
Implements ShEx 2.1 EXTENDS/ABSTRACT/RESTRICTS evaluation: - satisfiesExtendedShape: neighbourhood triples are allocated to the triple constraints of the extension targets (through references, AND/OR branches and extends chains); a constraint reached through several chains (diamond inheritance) allocates its triples to every one of them, so a triple cannot evade one chain's constraints by hiding in another. Extension targets are evaluated through ordinary satisfies() against their allocated part, so ShapeOr / ShapeAnd / ShapeNot parents work uniformly. - Node-scoped restriction stack: extension evaluation sees only its allocated part while value-expression recursion on other nodes sees the whole graph; the (node, shape) evaluation cache is bypassed under a restriction (the same pair can genuinely differ between parts). - satisfiesShapeDecl: restricts are conjuncts; an abstract declaration is satisfied only through a non-abstract descendant (except as an EXTENDS target); labeled shapes with registered extensions get the same descendant fallback. Handles both ShExJ 2.1 ShapeDecl and legacy inline-id flavours. - pyshex/utils/feasibility.py: a sound refutation test over partial bags of TripleConstraint matches, used to prune hopeless neighbourhoods before partition enumeration and to pre-filter EXTRA permutations. Ported from the same analysis implemented for Apache Jena, rudof and shex.js. shexSpec/shexTest validation manifest: ShExJ: 1182 tests = 1093 passed, 0 failed, 89 trait-skips ShExC: 1184 tests = 1091 passed, 0 failed, 93 trait-skips (baseline could not run: NotImplementedError on the first ShapeDecl) pyshexc is temporarily sourced from the EXTENDS-grammar PR branch (linkml/grammar-python-antlr-linkml#4); repoint to PyPI when released. Requires tests/data/shexTest (git submodule) for the manifest tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tive IRIs - SchemaLoader: IMPORT resolution (recursive merge, cycle-guarded, extensionless imports completed with .shex before redirect); relative IRIs parse against the schema's canonical location; ShExJ (JSON) input gets base absolutization (ObjectLiteral lexical values excluded). - ShapeAssociation accepts blank node selectors. - Test harness: preserve told bnode labels in rdflib parsing; sht:map JSON shape maps; canonical-space shape/focus/data-base alignment; empty skip_traits. - shexTest data fix pushed to shexSpec/shexTest#69: stale start2RefS2.json. shexTest validation manifest, zero trait-skips: ShExJ: 1182 = 1182 passed, 0 failed, 0 skips ShExC: 1184 = 1180 passed, 0 failed, 4 skips (schemas pyshexc cannot parse) was: 1093/0/89 and 1091/0/93. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Assertions were anonymous BNode()s with random UUID labels, and the turtle serializer orders bnode stanzas by label, so every regeneration shuffled the whole report. Zero-padded sequential labels preserve manifest order through serialization, and a single per-report dc:date (instead of per-result utcnow) keeps regeneration diffs line-for-line comparable. Regenerated against the pinned shexTest (0f45c51). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Windows / cross-version portability: - the schema loader handed raw filesystem paths to the ShExC parser as the document base; Windows paths (drive letter, backslashes) fail ShExJ IRIREF validation -- convert to file:/// URIs (POSIX untouched) - golden .results files read/written with the platform default encoding garble U+2260 on Windows (cp1252); pin encoding='utf-8' - "2016-07"^^xsd:date renders differently across Python versions (isodate on py<3.11 normalizes reduced-precision dates; fromisoformat on 3.11+ rejects them, preserving the lexical form) -- use a fully-specified date Also, to get the matrix running at all (red since the pytest migration): - workflow: checkout the shexTest submodule (https URL in .gitmodules), push trigger main -> master, SKIP_EXTERNAL_URLS (coerced to bool -- a string in pytest.mark.skipif is eval'ed as Python), guard the unmarked external-URL tests, never collect from tests/data - rdflib 7: serialize() returns str (drop .decode()), parse into bind_namespaces="core" graphs in PrefixLibrary.add_rdf, replace removed Graph.load(), regenerate goldens (dc1 renaming, bnode labels) - latent bugs: missing @ on shape refs in test_schemaorg DS_SHEX, CFGraph.open() loads nothing (validated an empty graph; parse() instead), manifest_tester failed the suite on unloadable-schema skips Full suite with SKIP_EXTERNAL_URLS: 129 passed, 21 skipped, 1 xfailed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ericprud
force-pushed
the
extends-validation
branch
from
July 28, 2026 01:08
8755309 to
816c2fa
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Implements ShEx 2.1
EXTENDS/ABSTRACT/RESTRICTSevaluation, plus a feasibility layer that prunes the partition search.satisfiesExtendedShape(p5_5): neighbourhood triples are allocated to the triple constraints of the extension targets (reached through references, AND/OR branches and extends chains). A constraint reached through several chains — diamond inheritance — allocates its triples to every one of them, so a triple cannot evade one chain'sNOTby hiding in another. Extension targets are evaluated through ordinarysatisfies()against their allocated part, so ShapeOr / ShapeAnd / ShapeNot parents work uniformly.subGraph).satisfiesShapeDecl(p5_3):restrictsare conjuncts; an abstract declaration is satisfied only through a non-abstract descendant (except when evaluated as an EXTENDS target); labeled shapes with registered extensions get the same descendant fallback. Handles both ShExJ 2.1ShapeDecland legacy inline-id flavours, and threads the declaration's id toShapeExternal(2.1 moved labels onto the decl).pyshex/utils/feasibility.py: a sound refutation test over partial bags of TripleConstraint matches — refutes structurally hopeless neighbourhoods (with Missing property reasons) before any partitioning, and pre-filters the 2^n EXTRA permutation loop. Ported from the same analysis implemented and proved for Apache Jena (jena-shex/docs/matching-search-optimization.md), rudof and shex.js.shexSpec/shexTest validation manifest
Baseline could not run the current manifest at all (
NotImplementedErroron the firstShapeDecl).Notes for review
pyshexcis temporarily sourced from the EXTENDS-grammar PR branch (Parse EXTENDS/RESTRICTS with shape references; fix chardet decode crash grammar-python-antlr-linkml#4); repoint[tool.uv.sources]to a PyPI release once published.tests/test_extends/(labeled/OR/AND+NOT parents, diamond inheritance, 7 cases).🤖 Generated with Claude Code