Skip to content

fix: make CI example fail partial and unscored runs - #136

Merged
stefyi-4355 merged 4 commits into
ifixai-ai:mainfrom
rudycelekli:fix/ci-example-reject-partial-unscored
Oct 2, 2026
Merged

stefyi-4355 merged 4 commits into
ifixai-ai:mainfrom
rudycelekli:fix/ci-example-reject-partial-unscored

Conversation

@rudycelekli

Copy link
Copy Markdown
Contributor

Problem

The shipped ifixai/examples/ci_check.py formats overall_score as a percentage before checking whether a run is gradeable. When the score is None (the normal result for an unscored run), the example crashes with TypeError instead of explaining the failure. A partial run can also return a high provisional score and satisfy all thresholds, causing this CI example to exit 0 despite the run being incomplete. The main CLI explicitly rejects partial runs.

Change

  • Print n/a for unavailable overall scores and grades, and fail with an actionable message.
  • Fail partial runs regardless of provisional numeric scores, including the abort reason when available.
  • Keep complete scored runs on their existing threshold path.

Verification

  • Before the change, both new failure cases failed: an exception for null overall, and exit 0 for partial. Afterward, all three focused tests pass, including a complete passing control.
  • uvx ruff check on changed files, uvx bandit -r ifixai -ll -q, ifixai validate, and git diff --check pass.

No provider, fixture, or inspection behavior changes.

@mraalegend

Copy link
Copy Markdown

Please U need a guide

@mraalegend

Copy link
Copy Markdown

Please I need video guide

@rudycelekli

Copy link
Copy Markdown
Contributor Author

Here is a short written walkthrough for this PR:

  1. Check out this PR and install the development dependencies as described in CONTRIBUTING.md (pip install -e ".[dev]").
  2. Run python -m pytest ifixai/tests/test_ci_check_unscored.py -q. The tests use a local fixture rather than a live provider, so they require no API key or paid calls.
  3. The two failure cases are an unavailable overall score and a partial run despite a high score. Both must return exit code 1. A complete, scored run that meets the thresholds still returns 0.

The user-facing change is that an incomplete evaluation can no longer let CI pass, and an unavailable score prints n/a rather than crashing during formatting. This is a written reproduction, not a video guide.

@stefyi-4355 stefyi-4355 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@stefyi-4355
stefyi-4355 merged commit ffda13c into ifixai-ai:main Oct 2, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants