Skip to content

Gate search compatibility on result overlap - #1750

Merged
cjmyers merged 1 commit into
masterfrom
marpaia/search-result-overlap
Aug 2, 2026
Merged

Gate search compatibility on result overlap#1750
cjmyers merged 1 commit into
masterfrom
marpaia/search-result-overlap

Conversation

@marpaia

@marpaia marpaia commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace exact first-page result equality with semantic identity overlap.
  • Require at least 50% Jaccard overlap for at least 90% of corpus probes.
  • Treat exact result sets, counts, metadata, and ranking order as diagnostic information.
  • Update the focused Explorer search test to compare result identities instead of the complete rendered HTML page.
  • Preserve the pinned corpus, submission parity, lifecycle, indexing, ACL, publication, removal, and expected-member requirements.

Motivation

SBOLExplorer and sbol-db use different search and ranking implementations. Exact result counts, ordering, and page contents are therefore not compatibility requirements.

The previous policy failed even though both backends passed their independent full-corpus conformance gates. The new policy tests the intended contract: both implementations must return a significantly overlapping set of relevant results without requiring identical behavior.

Jaccard overlap penalizes first pages dominated by unrelated results while allowing reasonable implementation-specific differences.

Validation

  • python3 -m unittest tests/search-backends/test_compare_reports.py tests/search-backends/test_run_conformance.py
  • python3 -m py_compile tests/search-backends/compare-reports.py tests/search-backends/test_compare_reports.py tests/test_functions.py tests/test_explorer_search.py tests/test_root.py
  • git diff --check

All seven focused unit tests pass.

@cjmyers
cjmyers merged commit fc61760 into master Aug 2, 2026
10 checks passed
@cjmyers
cjmyers deleted the marpaia/search-result-overlap branch August 2, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants