Skip to content

fix(runner): reject unknown API inspection selections before probes - #220

Merged
n-papaioannou merged 1 commit into
ifixai-ai:mainfrom
rudycelekli:fix/ifix-api-selection-validation-20261007
Oct 7, 2026
Merged

n-papaioannou merged 1 commit into
ifixai-ai:mainfrom
rudycelekli:fix/ifix-api-selection-validation-20261007

Conversation

@rudycelekli

@rudycelekli rudycelekli commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Reject unknown IDs in run_selected instead of silently omitting requested inspections. Move run_single's existing ID lookup ahead of capability probes and judge construction. Both APIs now reject a bad selection before querying the agent, matching the CLI's existing validation.

On current main, await run_selected({"B02", "B999"}, provider) returns a report containing only B02; { "B999" } returns an empty report. The single-test API raises for B999, but first probes the provider. The change reports the unknown IDs and available choices. Valid selections, including lists previously accepted at runtime, retain their results and provider teardown.

Verification

  • Native public API, real registry/default fixture and owned in-memory ChatProvider; no monkeypatched runner/registry and no live or paid provider calls.
  • Regression tests on unchanged current-main runner: 4 failed, 3 passed. After fix plus existing parallel-abort/teardown tests: 11 passed.
  • Invalid all-unknown, mixed valid/unknown, and single-ID calls now raise before the provider's capability hook; valid selected/single B02 controls still execute. All five native scenarios close their provider.
  • ruff check ifixai with existing Ruff 0.15.21: passed.
  • bandit -r ifixai -ll: passed.
  • Advisory targeted mypy: two existing object.execute errors, reproduced on the unchanged baseline; no new errors.
  • ifixai validate: 60 inspections valid; all 11 example fixtures valid.

Local checks reused an existing Python 3.11 dependency overlay. Actual fork CI passed all Python 3.10/3.11/3.12 jobs on 83952ad474918566454761881f6bc19d92410f1d, including layout/examples, Ruff and Bandit. Upstream PR checks are not yet reported (MERGEABLE / BLOCKED); this is not a claim of an upstream green check. This is a runner validation fix, with no inspection, fixture, or provider-registration change.

Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
@n-papaioannou
n-papaioannou merged commit 3c5dd9f into ifixai-ai:main Oct 7, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants