Skip to content

Commit e46f40b

Browse files
n-papaioannouclaude
andcommitted
ci: drop pytest pipeline, keep ruff/bandit/docs
The clean release ships without a tests/ folder, so the original CI's pytest+coverage steps had nothing to run. Strip them from ci.yml, drop pytest extras and ini_options from pyproject, and remove dead test references from CONTRIBUTING.md and README.md. CI now runs ruff, bandit, and a docs-presence check across Python 3.10/3.11/3.12; inspect-smoke on PRs is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 15d997e commit e46f40b

4 files changed

Lines changed: 17 additions & 58 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ on:
66
branches: [main, "fix/**"]
77

88
jobs:
9-
test:
9+
checks:
1010
runs-on: ubuntu-latest
1111
strategy:
1212
matrix:
@@ -24,13 +24,7 @@ jobs:
2424
run: python -m pip install -e ".[dev]"
2525

2626
- name: Lint
27-
run: ruff check ifixai tests
28-
29-
- name: Test with coverage
30-
run: pytest tests/ ifixai/tests/ --cov=ifixai --cov-report=term-missing --cov-fail-under=60 -q --tb=short -m "not integration"
31-
32-
- name: Integration tests (disk/fixture-dependent)
33-
run: pytest tests/ -q --tb=short -m integration
27+
run: ruff check ifixai
3428

3529
- name: Security scan
3630
run: bandit -r ifixai -ll
@@ -44,7 +38,7 @@ jobs:
4438
4539
inspect-smoke:
4640
runs-on: ubuntu-latest
47-
needs: test
41+
needs: checks
4842
if: ${{ github.event_name == 'pull_request' }}
4943
steps:
5044
- uses: actions/checkout@v4

‎CONTRIBUTING.md‎

Lines changed: 9 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Contributing to ifixai
22

3-
Thanks for considering a contribution. This guide covers the mechanics of adding inspections, fixtures, providers, and running the test suite. For the behavioural contract (what the project expects from the code you write), see the root `CLAUDE.md` if it exists plus `.claude/rules/common/*.md` in the repository.
3+
Thanks for considering a contribution. This guide covers the mechanics of adding inspections, fixtures, and providers. For the behavioural contract (what the project expects from the code you write), see the root `CLAUDE.md` if it exists plus `.claude/rules/common/*.md` in the repository.
44

55
## Environment setup
66

@@ -18,13 +18,10 @@ The `pre-commit install` step wires up local hooks (`gitleaks`, `ruff`, a `.env`
1818
Verify:
1919

2020
```bash
21-
ruff check ifixai tests
22-
python -m pytest --cov=ifixai --cov-report=term
21+
ruff check ifixai
2322
```
2423

25-
The coverage floor is declared in `.github/workflows/ci.yml`. The policy is simple: the floor is pinned at `floor(current_baseline) - 2` percentage points so a flaky test cannot accidentally erode coverage, and is ratcheted upward when a sustained improvement lands. Drop-below fails CI.
26-
27-
## Adding a test (inspection)
24+
## Adding an inspection
2825

2926
Each inspection is one file under `ifixai/tests/bNN_short_name.py`. The minimum contract:
3027

@@ -35,14 +32,6 @@ Each inspection is one file under `ifixai/tests/bNN_short_name.py`. The minimum
3532
5. Register the inspection in `ifixai/tests/registry.py` (import + add to `ALL_SPECS` + `create_inspection` switch).
3633
6. Update `ifixai/scoring/category_weights.py` only if the inspection belongs to the strategic set.
3734

38-
### Inspection testing
39-
40-
Every inspection should have a companion test under `tests/test_bNN_*.py` covering at least:
41-
42-
- `test_{inspection}_spec_invariants_unchanged` — asserts immutable fields (id, category, threshold, weight, strategic-ness).
43-
- Happy-path evidence generation against a fixture.
44-
- Failure path (provider error, empty response, malformed output).
45-
4635
## Authoring a fixture
4736

4837
Fixtures are YAML files under `ifixai/fixtures/`. Validate against `ifixai/fixtures/schema.json`. A fixture MUST supply every key listed in the union of every registered inspection's `required_fixture_keys`.
@@ -59,22 +48,12 @@ Providers implement the `ChatProvider` protocol from `ifixai/providers/base.py`.
5948
2. Register the provider string in `ifixai/providers/resolver.py`.
6049
3. Add an optional dependency extra in `pyproject.toml` under `[project.optional-dependencies]` so users install only what they need.
6150
4. Do not swallow exceptions silently. If the provider has idiomatic error types, translate them into `ProviderError` (or a subclass).
62-
5. Add `tests/test_<your_provider>.py` with stub / mock coverage.
6351

64-
## Running tests locally
52+
## Local checks
6553

6654
```bash
67-
# full suite
68-
python -m pytest
69-
70-
# one module
71-
python -m pytest tests/test_your_thing.py
72-
73-
# with coverage
74-
python -m pytest --cov=ifixai --cov-report=term-missing
75-
76-
# ruff
77-
ruff check ifixai tests
55+
# lint
56+
ruff check ifixai
7857

7958
# type check
8059
mypy ifixai
@@ -91,19 +70,17 @@ Follow the Conventional Commits-style prefixes used by this project:
9170
- `fix:` bug fix
9271
- `refactor:` behaviour-preserving change
9372
- `docs:` documentation only
94-
- `test:` test-only changes
9573
- `chore:` tooling / housekeeping
9674
- `perf:` performance improvement
9775
- `ci:` CI configuration
9876

99-
Keep commits small and atomic. Include a test with every `fix:` or `feat:` that changes observable behaviour.
77+
Keep commits small and atomic.
10078

10179
## Pull requests
10280

10381
- Target branch: `main`.
104-
- Include a test plan in the PR body.
105-
- Confirm `ruff`, `pytest`, and `bandit` all pass locally before requesting review. `mypy` is advisory (run it locally if you touched typed surfaces, but it is not a CI gate).
106-
- For any inspection / fixture / provider change, paste one worked example scorecard snippet (JSON or Markdown) into the PR body. The `ifixai-results/` directory is gitignored and cannot be updated as part of a PR.
82+
- Confirm `ruff` and `bandit` pass locally before requesting review. `mypy` is advisory (run it locally if you touched typed surfaces, but it is not a CI gate).
83+
- For any inspection / fixture / provider change, paste one worked example scorecard snippet (JSON or Markdown) into the PR body.
10784

10885
## Where to ask
10986

‎README.md‎

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -184,13 +184,10 @@ healthcare, software engineering, customer support, finance, legal,
184184
government, anything. Industry knowledge lives **only in user authored
185185
fixture YAML**, never in test code.
186186

187-
A guardrail test ([`tests/test_industry_agnosticism.py`](tests/test_industry_agnosticism.py))
188-
scans every inspection, rule, judge reference, and analytic rubric against a
189-
word and phrase level industry blocklist on every CI run. New contributors
190-
who try to hardcode healthcare, legal, or finance prompts are blocked at PR
191-
time. The blocklist is literal substring matching; a deliberate contributor
192-
can still route domain content through fixture YAML (the intended path)
193-
rather than hardcoded strings.
187+
Inspection prompts, rules, judge references, and analytic rubrics are kept
188+
free of domain specific content by convention. Industry knowledge belongs in
189+
user authored fixture YAML — keeping it out of the inspection code is what
190+
lets the same suite run meaningfully against agents in any domain.
194191

195192
Three illustrative example fixtures live under
196193
[`ifixai/fixtures/examples/`](ifixai/fixtures/examples/) demonstrating the
@@ -371,8 +368,7 @@ Custom providers can be passed directly by implementing `ChatProvider` from [ifi
371368

372369
```bash
373370
pip install -e ".[dev]"
374-
pytest tests/
375-
ruff check ifixai tests
371+
ruff check ifixai
376372
```
377373

378374
## License

‎pyproject.toml‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,6 @@ all = [
5050
"huggingface-hub>=0.20",
5151
]
5252
dev = [
53-
"pytest>=7.0",
54-
"pytest-asyncio>=0.23",
55-
"pytest-cov>=4.0",
5653
"ruff>=0.4",
5754
"mypy>=1.0",
5855
"bandit>=1.7",
@@ -69,11 +66,6 @@ Documentation = "https://github.com/ifixai-ai/diagnostic#readme"
6966
Repository = "https://github.com/ifixai-ai/diagnostic"
7067
Issues = "https://github.com/ifixai-ai/diagnostic/issues"
7168

72-
[tool.pytest.ini_options]
73-
markers = [
74-
"integration: tests that depend on on-disk fixtures, shipped YAML, or recorded cassettes",
75-
]
76-
7769
[tool.setuptools.packages.find]
7870
include = ["ifixai*"]
7971

0 commit comments

Comments
 (0)