You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7e4c165
Browse filesBrowse the repository at this point in the historyBrowse files
- Add Wiring governance to TOC; reorder providers extras table to match
install order in Quick start.
- Pull Judge selection rules to the top of Quick start so they apply to
every provider example below.
- Restructure provider sections so each example has a coherent judge
setup (cross-judge env in Anthropic/Gemini/Bedrock/HF/HTTP, explicit
--judge-* flags for OpenRouter/Azure, single-key self-judge for
LangChain).
- Fix typo: "Fives" -> "Five" example fixtures.
- Drop bogus "90 lines" count for smoke_tiny.yaml (it's 88).
- Tighten "ifixai list fixtures" comment in CLI reference -- the command
only lists registered named fixtures; example fixtures load by path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7.[Author your own fixture](#author-your-own-fixture)
59
-
8.[Supported providers](#supported-providers)
60
-
9.[CLI reference](#cli-reference)
61
-
10.[Scoring](#scoring)
62
-
11.[Python API](#python-api)
63
-
12.[Development](#development)
64
-
13.[Contact](#contact)
65
-
14.[License](#license)
59
+
8.[Wiring governance](#wiring-governance)
60
+
9.[Supported providers](#supported-providers)
61
+
10.[CLI reference](#cli-reference)
62
+
11.[Scoring](#scoring)
63
+
12.[Python API](#python-api)
64
+
13.[Development](#development)
65
+
14.[Contact](#contact)
66
+
15.[License](#license)
66
67
67
68
## Requirements
68
69
@@ -73,10 +74,10 @@ and track over time.
73
74
|---|---|---|
74
75
|*(none)*| Core only |`mock`, `http`, `langchain` (you must `pip install langchain` yourself) |
75
76
|`openai`|`openai` SDK |`openai`|
76
-
|`azure`|`openai` SDK |`azure` (same client; set `--endpoint` to your Azure OpenAI resource) |
77
-
|`openrouter`|`openai` SDK (OpenRouter exposes an OpenAI-compatible endpoint; any compatible SDK or `--provider http` also works) |`openrouter`|
78
77
|`anthropic`|`anthropic` SDK |`anthropic`|
78
+
|`openrouter`|`openai` SDK (OpenRouter exposes an OpenAI-compatible endpoint; any compatible SDK or `--provider http` also works) |`openrouter`|
79
79
|`gemini`|`google-generativeai`|`gemini`|
80
+
|`azure`|`openai` SDK |`azure` (same client; set `--endpoint` to your Azure OpenAI resource) |
80
81
|`bedrock`|`boto3`|`bedrock`|
81
82
|`huggingface`|`huggingface-hub`|`huggingface`|
82
83
|`dev`| Lint, types, tests, security |[Contributing](CONTRIBUTING.md) only |
@@ -96,6 +97,12 @@ The CLI does **not** auto-read the SUT API key from the environment: pass **`--a
96
97
97
98
Omitting `--fixture` uses the built-in **default** fixture. Runs emit a scorecard under `./ifixai-results/` (override with `--output`). Typical wall time is a few minutes on broadband.
98
99
100
+
**Judge selection:**
101
+
-**Default:** judge = any non-SUT provider key in your env, run on that provider's default model.
102
+
-**Multiple keys:** tiebreaker order is `anthropic → openai → gemini → openrouter → azure → bedrock → huggingface`.
103
+
-**No non-SUT key:** pass `--eval-mode self`, or the run refuses.
Pinning the judge avoids the underlying-model collision OpenRouter routing can introduce (e.g. routing the SUT to an Anthropic model while Anthropic is also the auto-judge).
150
+
141
151
### 4 — Google Gemini
142
152
143
153
```bash
144
154
pip install -e ".[gemini]"
145
155
export GEMINI_API_KEY=... # or GOOGLE_API_KEY
146
-
exportOPENAI_API_KEY=sk-...
156
+
exportANTHROPIC_API_KEY=sk-ant-api03-...# second provider for cross-judge (or use --eval-mode self)
147
157
ifixai run --provider gemini --api-key "$GEMINI_API_KEY"
148
158
```
149
159
150
-
### 5 — Azure OpenAI
160
+
### 5 — Azure OpenAI (explicit judge)
151
161
152
162
```bash
153
163
pip install -e ".[azure]"# or .[openai] — same OpenAI-compatible SDK
0 commit comments