|
| 1 | +# Contributing to ifixai |
| 2 | + |
| 3 | +Thanks for considering a contribution. This guide covers the mechanics of adding inspections, fixtures, providers, and running the test suite. For the behavioural contract (what the project expects from the code you write), see the root `CLAUDE.md` if it exists plus `.claude/rules/common/*.md` in the repository. |
| 4 | + |
| 5 | +## Environment setup |
| 6 | + |
| 7 | +```bash |
| 8 | +git clone <your-fork-url> ifixai |
| 9 | +cd ifixai |
| 10 | +python -m venv .venv |
| 11 | +source .venv/bin/activate |
| 12 | +pip install -e ".[dev]" |
| 13 | +pre-commit install |
| 14 | +``` |
| 15 | + |
| 16 | +The `pre-commit install` step wires up local hooks (`gitleaks`, `ruff`, a `.env` guard, and a sanitizer regex for known-internal identifiers). Run them on demand with `pre-commit run --all-files`. |
| 17 | + |
| 18 | +Verify: |
| 19 | + |
| 20 | +```bash |
| 21 | +ruff check ifixai tests |
| 22 | +python -m pytest --cov=ifixai --cov-report=term |
| 23 | +``` |
| 24 | + |
| 25 | +The coverage floor is declared in `.github/workflows/ci.yml`. The policy is simple: the floor is pinned at `floor(current_baseline) - 2` percentage points so a flaky test cannot accidentally erode coverage, and is ratcheted upward when a sustained improvement lands. Drop-below fails CI. |
| 26 | + |
| 27 | +## Adding a test (inspection) |
| 28 | + |
| 29 | +Each inspection is one file under `ifixai/tests/bNN_short_name.py`. The minimum contract: |
| 30 | + |
| 31 | +1. Declare the `SPEC` — a `InspectionSpec` instance with `test_id`, `name`, `category` (one of the five `InspectionCategory` values), `description`, `threshold`, `weight`, `scoring_method`, and optional `is_strategic` / `is_mandatory_minimum` flags. |
| 32 | +2. Implement a subclass of `BaseTest` (from `ifixai.tests.base`). Override `run()` to produce a list of `EvidenceItem`s. Use `self.pipeline.evaluate(...)` to get a pass/fail from the configured judge. |
| 33 | +3. Declare `required_fixture_keys: frozenset[str]` on the subclass listing every fixture key the inspection's templates reference. The fixture loader validates this at load time; inspections that reference keys the fixture doesn't provide fail fast with an actionable error. |
| 34 | +4. Render every prompt through `ifixai.utils.template_renderer.render(template, context)`. Direct `str.format(...)` or f-string interpolation on fixture values is forbidden — it silently leaks `{placeholder}` literals to the model when a key is missing. |
| 35 | +5. Register the inspection in `ifixai/tests/registry.py` (import + add to `ALL_SPECS` + `create_inspection` switch). |
| 36 | +6. Update `ifixai/scoring/category_weights.py` only if the inspection belongs to the strategic set. |
| 37 | + |
| 38 | +### Inspection testing |
| 39 | + |
| 40 | +Every inspection should have a companion test under `tests/test_bNN_*.py` covering at least: |
| 41 | + |
| 42 | +- `test_{inspection}_spec_invariants_unchanged` — asserts immutable fields (id, category, threshold, weight, strategic-ness). |
| 43 | +- Happy-path evidence generation against a fixture. |
| 44 | +- Failure path (provider error, empty response, malformed output). |
| 45 | + |
| 46 | +## Authoring a fixture |
| 47 | + |
| 48 | +Fixtures are YAML files under `ifixai/fixtures/`. Validate against `ifixai/fixtures/schema.json`. A fixture MUST supply every key listed in the union of every registered inspection's `required_fixture_keys`. |
| 49 | + |
| 50 | +The `x-placeholders` section of the schema (lint-only) enumerates the placeholder keys any inspection may reference. Keep it in sync when you add a inspection that references a new key. |
| 51 | + |
| 52 | +Three example fixtures live under `ifixai/fixtures/examples/`. Copy one as a starting point. |
| 53 | + |
| 54 | +## Registering a new provider |
| 55 | + |
| 56 | +Providers implement the `ChatProvider` protocol from `ifixai/providers/base.py`. Steps: |
| 57 | + |
| 58 | +1. Create `ifixai/providers/<your_provider>.py` implementing at minimum `send_message` — other capability methods may raise `NotImplementedError` if the provider does not expose them. |
| 59 | +2. Register the provider string in `ifixai/providers/resolver.py`. |
| 60 | +3. Add an optional dependency extra in `pyproject.toml` under `[project.optional-dependencies]` so users install only what they need. |
| 61 | +4. Do not swallow exceptions silently. If the provider has idiomatic error types, translate them into `ProviderError` (or a subclass). |
| 62 | +5. Add `tests/test_<your_provider>.py` with stub / mock coverage. |
| 63 | + |
| 64 | +## Running tests locally |
| 65 | + |
| 66 | +```bash |
| 67 | +# full suite |
| 68 | +python -m pytest |
| 69 | + |
| 70 | +# one module |
| 71 | +python -m pytest tests/test_your_thing.py |
| 72 | + |
| 73 | +# with coverage |
| 74 | +python -m pytest --cov=ifixai --cov-report=term-missing |
| 75 | + |
| 76 | +# ruff |
| 77 | +ruff check ifixai tests |
| 78 | + |
| 79 | +# type check |
| 80 | +mypy ifixai |
| 81 | + |
| 82 | +# security scan |
| 83 | +bandit -r ifixai -ll |
| 84 | +``` |
| 85 | + |
| 86 | +## Commit conventions |
| 87 | + |
| 88 | +Follow the Conventional Commits-style prefixes used by this project: |
| 89 | + |
| 90 | +- `feat:` new user-visible feature |
| 91 | +- `fix:` bug fix |
| 92 | +- `refactor:` behaviour-preserving change |
| 93 | +- `docs:` documentation only |
| 94 | +- `test:` test-only changes |
| 95 | +- `chore:` tooling / housekeeping |
| 96 | +- `perf:` performance improvement |
| 97 | +- `ci:` CI configuration |
| 98 | + |
| 99 | +Keep commits small and atomic. Include a test with every `fix:` or `feat:` that changes observable behaviour. |
| 100 | + |
| 101 | +## Pull requests |
| 102 | + |
| 103 | +- Target branch: `main`. |
| 104 | +- Include a test plan in the PR body. |
| 105 | +- Confirm `ruff`, `pytest`, and `bandit` all pass locally before requesting review. `mypy` is advisory (run it locally if you touched typed surfaces, but it is not a CI gate). |
| 106 | +- For any inspection / fixture / provider change, paste one worked example scorecard snippet (JSON or Markdown) into the PR body. The `ifixai-results/` directory is gitignored and cannot be updated as part of a PR. |
| 107 | + |
| 108 | +## Where to ask |
| 109 | + |
| 110 | +Open a GitHub issue on the repository for questions, bug reports, or feature proposals. For security-sensitive reports see `SECURITY.md`. |
0 commit comments