sync: per-test layout restructure and public sanitization pass

.github/workflows/ci.yml

@@ -35,31 +35,3 @@ jobs:
           test -f CONTRIBUTING.md || { echo "CONTRIBUTING.md missing"; exit 1; }
           test -f SECURITY.md || { echo "SECURITY.md missing"; exit 1; }
           echo "governance docs present"
-
-  inspect-smoke:
-    runs-on: ubuntu-latest
-    needs: checks
-    if: ${{ github.event_name == 'pull_request' }}
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v5
-        with:
-          python-version: "3.12"
-      - name: Check OpenAI key presence
-        id: check_key
-        env:
-          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
-        run: |
-          if [ -n "$OPENAI_API_KEY" ]; then
-            echo "present=true" >> "$GITHUB_OUTPUT"
-          else
-            echo "present=false" >> "$GITHUB_OUTPUT"
-          fi
-      - name: Install with inspect extras
-        if: steps.check_key.outputs.present == 'true'
-        run: python -m pip install -e ".[dev,inspect]"
-      - name: Inspect B01 smoke
-        if: steps.check_key.outputs.present == 'true'
-        env:
-          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
-        run: inspect eval ifixai/inspect_integration/tasks.py@ifixai_b01

.gitignore

@@ -0,0 +1,103 @@
+# ─── Python ────────────────────────────────────────────────────────────────
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg
+*.egg-info/
+*.so
+*.dylib
+*.pyd
+build/
+dist/
+wheels/
+.python-version
+
+# ─── Virtual environments ──────────────────────────────────────────────────
+.venv/
+venv/
+env/
+ENV/
+
+# ─── Lock files (resolve from pyproject.toml) ──────────────────────────────
+uv.lock
+poetry.lock
+Pipfile.lock
+
+# ─── IDE / editors ─────────────────────────────────────────────────────────
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+*.bak
+*.orig
+*.tmp
+
+# ─── Test, lint, type-check caches ─────────────────────────────────────────
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.tox/
+.nox/
+.hypothesis/
+.cache/
+.coverage
+.coverage.*
+coverage.xml
+htmlcov/
+
+# ─── Notebooks ─────────────────────────────────────────────────────────────
+.ipynb_checkpoints/
+
+# ─── Logs ──────────────────────────────────────────────────────────────────
+*.log
+logs/
+
+# ─── OS ────────────────────────────────────────────────────────────────────
+.DS_Store
+Thumbs.db
+
+# ─── Test / test outputs (regenerated locally) ────────────────────────
+ifixai-results/
+runs/
+test_data/
+ifixai/reliability/.kappa_cache/
+gold_sets/*/labels-v*.json
+tests.html
+
+# ─── Library pytest suite (local-only; public surface is ifixai/tests/) ─────
+/tests/
+
+# ─── Spec-kit & local AI tooling ───────────────────────────────────────────
+specs/
+.specify/
+.cursor/
+.cursor/commands/speckit*
+.claude/
+
+# ─── Local-only docs (contributor notes, agent guidance, internal artifacts)
+CLAUDE.md
+TESTING.md
+OPEN_SOURCE_READINESS.md
+TESTS.md
+/examples/
+scripts/
+docs-content/
+website-misalignments-*.md
+
+# ─── Loose desktop dumps at repo root ──────────────────────────────────────
+/Screenshot*.png
+/screencapture-*.pdf
+/screencapture-*.png
+
+# ─── Secrets / local config (NEVER commit) ─────────────────────────────────
+.env
+.env.*
+!.env.example
+.envrc
+secrets/
+credentials.json
+*.pem
+*.key
+*.crt
+*.p12

CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing to ifixai
 
-Thanks for considering a contribution. This guide covers the mechanics of adding inspections, fixtures, and providers. For the behavioural contract (what the project expects from the code you write), see the root `CLAUDE.md` if it exists plus `.claude/rules/common/*.md` in the repository.
+Thanks for considering a contribution. This guide covers the mechanics of adding inspections, fixtures, and providers.
 
 ## Environment setup
 
@@ -13,30 +13,51 @@ pip install -e ".[dev]"
 pre-commit install
 ```
 
-The `pre-commit install` step wires up local hooks (`gitleaks`, `ruff`, a `.env` guard, and a sanitizer regex for known-internal identifiers). Run them on demand with `pre-commit run --all-files`.
+The `pre-commit install` step wires up local hooks (`gitleaks`, `ruff`, and a `.env` guard). Run them on demand with `pre-commit run --all-files`.
 
 Verify:
 
 ```bash
 ruff check ifixai
+bandit -r ifixai -ll
 ```
 
-## Adding an inspection
+The package ships `ifixai/` plus the per-inspection bundles under `ifixai/inspections/b<NN>_<slug>/`.
+
+## Adding a test (inspection)
+
+Each inspection lives in its own folder under `ifixai/inspections/bNN_short_name/`. Required contents:
+
+- `runner.py` — declares `SPEC` and the `BaseTest` subclass
+- `definition.yaml` — the conversation plan (steps, prompts, evaluation hints)
+- `rubric.yaml` — analytic-judge dimensions and weights
+- `references.yaml` — reference responses used by atomic-claims grounding
+- `corpus.yaml` — adversarial seeds (only B12, B14, B30)
 
-Each inspection is one file under `ifixai/tests/bNN_short_name.py`. The minimum contract:
+The minimum contract:
 
-1. Declare the `SPEC` — a `InspectionSpec` instance with `test_id`, `name`, `category` (one of the five `InspectionCategory` values), `description`, `threshold`, `weight`, `scoring_method`, and optional `is_strategic` / `is_mandatory_minimum` flags.
-2. Implement a subclass of `BaseTest` (from `ifixai.tests.base`). Override `run()` to produce a list of `EvidenceItem`s. Use `self.pipeline.evaluate(...)` to get a pass/fail from the configured judge.
+1. Declare the `SPEC` — an `InspectionSpec` instance with `test_id`, `name`, `category` (one of the five `InspectionCategory` values), `description`, `threshold`, `weight`, `scoring_method`, and optional `is_strategic` / `is_mandatory_minimum` flags.
+2. Implement a subclass of `BaseTest` (from `ifixai.harness.base`). Override `run()` to produce a list of `EvidenceItem`s. Use `self.pipeline.evaluate(...)` to get a pass/fail from the configured judge.
 3. Declare `required_fixture_keys: frozenset[str]` on the subclass listing every fixture key the inspection's templates reference. The fixture loader validates this at load time; inspections that reference keys the fixture doesn't provide fail fast with an actionable error.
 4. Render every prompt through `ifixai.utils.template_renderer.render(template, context)`. Direct `str.format(...)` or f-string interpolation on fixture values is forbidden — it silently leaks `{placeholder}` literals to the model when a key is missing.
-5. Register the inspection in `ifixai/tests/registry.py` (import + add to `ALL_SPECS` + `create_inspection` switch).
+5. Register the inspection in `ifixai/harness/registry.py` (import the class and `SPEC` from `ifixai.inspections.bNN_short_name.runner` + add to `ALL_SPECS` + `create_inspection` switch).
 6. Update `ifixai/scoring/category_weights.py` only if the inspection belongs to the strategic set.
+7. Run `ifixai validate` (no args) — the layout validator will fail loudly if any required artifact is missing or the folder name disagrees with the YAML `test_id`.
+
+### Inspection self-validation
+
+Run `ifixai validate` (no args) after authoring or editing an inspection. The layout validator checks:
+
+- the per-test folder contains every required artifact (`runner.py`, `definition.yaml`, `rubric.yaml`, `references.yaml`, plus `corpus.yaml` for B12/B14/B30);
+- the folder name agrees with the YAML `test_id`;
+- `SPEC` invariants (id, category, threshold, weight, strategic flags) match the registry;
+- the runner registers cleanly with `harness/registry.py`.
 
 ## Authoring a fixture
 
 Fixtures are YAML files under `ifixai/fixtures/`. Validate against `ifixai/fixtures/schema.json`. A fixture MUST supply every key listed in the union of every registered inspection's `required_fixture_keys`.
 
-The `x-placeholders` section of the schema (lint-only) enumerates the placeholder keys any inspection may reference. Keep it in sync when you add a inspection that references a new key.
+The `x-placeholders` section of the schema (lint-only) enumerates the placeholder keys any inspection may reference. Keep it in sync when you add an inspection that references a new key.
 
 Three example fixtures live under `ifixai/fixtures/examples/`. Copy one as a starting point.
 
@@ -48,8 +69,9 @@ Providers implement the `ChatProvider` protocol from `ifixai/providers/base.py`.
 2. Register the provider string in `ifixai/providers/resolver.py`.
 3. Add an optional dependency extra in `pyproject.toml` under `[project.optional-dependencies]` so users install only what they need.
 4. Do not swallow exceptions silently. If the provider has idiomatic error types, translate them into `ProviderError` (or a subclass).
+5. Exercise the provider end-to-end against a fixture before opening a PR (`ifixai run --provider <your_provider> ...`); attach the resulting scorecard snippet to the PR body.
 
-## Local checks
+## Running checks locally
 
 ```bash
 # lint
@@ -60,6 +82,9 @@ mypy ifixai
 
 # security scan
 bandit -r ifixai -ll
+
+# layout / inspection validation
+ifixai validate
 ```
 
 ## Commit conventions
@@ -79,7 +104,8 @@ Keep commits small and atomic.
 ## Pull requests
 
 - Target branch: `main`.
-- Confirm `ruff` and `bandit` pass locally before requesting review. `mypy` is advisory (run it locally if you touched typed surfaces, but it is not a CI gate).
+- Include a test plan in the PR body.
+- Confirm `ruff`, `bandit`, and `ifixai validate` all pass locally before requesting review. `mypy` is advisory (run it locally if you touched typed surfaces, but it is not a CI gate).
 - For any inspection / fixture / provider change, paste one worked example scorecard snippet (JSON or Markdown) into the PR body.
 
 ## Where to ask

LICENSE

@@ -175,7 +175,7 @@
 
    END OF TERMS AND CONDITIONS
 
-   Copyright 2026 iMe Life Ltd
+   Copyright 2026 The ifixai Authors
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.