1.17.5 (2026-08-21)
1.17.4 (2026-08-14)
- judge verdict extraction survives braces and quoted examples in prose (fed9818)
- judge verdict extraction survives braces and quoted examples in prose (69991cf)
- review follow-ups — verdict value-type validation, hard parse budget, O(1) span eviction (7f815b9)
1.17.3 (2026-08-12)
- judging and general failures contributing to lower scores (81a3008)
1.17.2 (2026-07-17)
- docs homepage link (6c16831)
1.17.1 (2026-07-17)
- claude-code: guard non-object JSON in copyClaudeConfigWithoutMcp (8f1bcdd)
- claude-code: stop personal MCP servers leaking into scenario runs (f3839c7)
- claude-code: strip operator MCP config from copied .claude.json (00738d1)
- support slah names in models (3306ea5)
- support slah names in models (2d386db)
1.17.0 (2026-06-24)
- loosen up some language for contributors (7fe6fe0)
1.16.2 (2026-06-24)
- tests (d030b8c)
1.16.1 (2026-06-23)
- improving docs styling (c032e24)
1.16.0 (2026-06-23)
- docs-site: complete landing page and docs redesign (87981de)
- ui consistency work (948136c)
1.15.4 (2026-06-23)
- Gemini ACP runs recorded as false 15m timeouts (3eacdba)
1.15.3 (2026-06-23)
- don't zero-score ACP runs killed by signal after completing (e8b4615)
- preserve completed signal-exit classification (fd4b26d)
1.15.2 (2026-06-23)
- runner: always promote jobs to running before completion (10c2dd2)
1.15.1 (2026-06-20)
- cli duration outputs (9234ebd)
- kiro keychain symlinking (15a8f39)
- kiro-cli needed updating to pull in the right settings (1ec2f73)
- setting up skills docs (9fe8653)
- show all at end of cli run (2b7e790)
1.15.0 (2026-06-15)
- add AXIS skills (1d32968)
- only show active runs (e934b74)
1.14.0 (2026-06-03)
- support remote scenarios (44c44f1)
- adjust the evals for env and agents (2fee690)
- improve the judge system prompt to stay on track (2583d1d)
- reduce visual noise on cli output with judge clarifications (1ab6e5d)
1.13.0 (2026-05-21)
- add fallback support for agent auth using local sessions (b1b5fec)
- add favicon (d3b241c)
1.12.0 (2026-05-20)
- support scoring via different agents (2c76fec)
1.11.0 (2026-05-16)
- isolate the home dir from working dir (5aaf474)
1.10.0 (2026-05-15)
- add --failed support for targeting only tests that fail (2cdbc7a)
- allow canceling in-flight evals and retain data (9b79de6)
1.9.0 (2026-05-15)
- rubric -> judge (b15377c)
- make it clear that things are in teardown mode (a54de01)
- show scenario key on report (6d4a5bf)
- update default concurrency to 15 (38526fb)
1.8.1 (2026-05-14)
- add execution status to scoring context (e9898ea)
- gemini token counting and acp error reporting (8f1d7c2)
1.8.0 (2026-05-13)
- support before/after all hooks (a3cab68)
1.7.4 (2026-05-13)
- improve fuzzy detection of configs (5de722c)
1.7.3 (2026-05-13)
- improve the exports for scenarioinput (f4742d7)
1.7.2 (2026-05-13)
1.7.1 (2026-05-12)
1.7.0 (2026-05-12)
- add globs support for scnearios,variants, and agents (c124bfb)
1.6.5 (2026-05-12)
- make the default timout for lifecycle methods to be larger (fed03a1)
1.6.4 (2026-05-12)
- improve acp end detection (0b559cc)
1.6.3 (2026-05-12)
1.6.2 (2026-05-12)
- support debug streaming (bd2b0ef)
1.6.1 (2026-05-11)
- type on config (c6df111)
1.6.0 (2026-05-11)
- add docs for expanded agent support (19d0799)
1.5.0 (2026-05-08)
1.4.0 (2026-05-07)
1.3.1 (2026-05-06)
- tests (04b28d2)
1.3.0 (2026-05-06)
- improving web search details (6275565)
1.2.2 (2026-05-05)
1.2.1 (2026-05-05)
- report adjustments (3108a5a)
1.2.0 (2026-05-04)
- axis for axis (a3eab12)
- improve table layouts on report (141e2f8)
- improve variant rendering on cli (76ab416)
- render the variants properly (37cd2a2)
1.1.7 (2026-05-04)
1.1.6 (2026-05-01)
1.1.5 (2026-04-30)
- improved skip support (f08e89c)
1.1.4 (2026-04-30)
1.1.3 (2026-04-29)
- improve scoring concerns (b8f2a8d)
1.1.2 (2026-04-29)
- reuse common workspace (17f13a4)
1.1.1 (2026-04-29)
- release-please is stuck (cf0e7f5)
1.1.0 (2026-04-29)
- monolithic scoring to multi-category scoring (d191417)
- drop the triage phase for now (11abff5)