diff --git a/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/feature_spec.md b/docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/feature_spec.md similarity index 100% rename from docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/feature_spec.md rename to docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/feature_spec.md diff --git a/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/idea.md b/docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/idea.md similarity index 100% rename from docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/idea.md rename to docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/idea.md diff --git a/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/implementation_plan.md b/docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/implementation_plan.md similarity index 99% rename from docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/implementation_plan.md rename to docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/implementation_plan.md index ebc8ba5f..43d6541a 100644 --- a/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/implementation_plan.md +++ b/docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/implementation_plan.md @@ -1,7 +1,7 @@ # Implementation Plan — Baseline-phase unit tests depend on suite ordering **Date:** 2026-06-02 -**Status:** Ready for Execution +**Status:** Complete (PR #466, squash-merged `6298e77`, 2026-06-05) **Primary spec:** [feature_spec.md](feature_spec.md) **Policy source(s):** [CLAUDE.md §"Bug Fix Protocol"](../../../../../CLAUDE.md), [docs/05_quality/testing.md](../../../../05_quality/testing.md) diff --git a/docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/pipeline_status.md b/docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/pipeline_status.md new file mode 100644 index 00000000..22a620dc --- /dev/null +++ b/docs/00_overview/implemented_features/2026_06_05_bug_baseline_phase_test_isolation/pipeline_status.md @@ -0,0 +1,25 @@ +# Pipeline Status — Baseline-phase unit tests depend on suite ordering + +## Idea +- Status: Complete +- File: idea.md + +## Spec +- Status: Approved +- Date: 2026-06-02 +- File: feature_spec.md +- Cross-model review: GPT-5.5 passed (2 cycles, 2 findings — 2 accepted, both Low) +- Phases: 1 total, 1 covered by spec (single-phase) + +## Plan +- Status: Approved +- Date: 2026-06-02 +- File: implementation_plan.md +- Cross-model review: GPT-5.5 passed (2 cycles, 1 finding — rejected-with-counter-evidence, Low) +- Stories: 2 total across 1 epic +- Phases covered: 1 of 1 (single-phase) + +## Implementation +- Status: Complete (PR #466, squash-merged `6298e77`, 2026-06-05) +- Release: mvp2 +- Note: Backend test-only, no migration. Story 1.1 deferred the `get_settings()` call in `_compute_baseline_wait_s` (`backend/workers/orchestrator.py`) into the falsy-`trial_timeout_s` branch so explicit-timeout callers never construct `Settings` (return values unchanged). Story 1.2 added an autouse `_settings_env_and_restore` fixture + a `test_explicit_timeout_does_not_read_settings` regression to `test_orchestrator_baseline_phase.py`. Standalone run with secrets unset: 14 passed (pre-fix `3 failed, 1 passed`); full unit suite 2400 passed. No Gemini findings; final GPT-5.5 skipped (≤40 LOC, test-only, below threshold). All 19 CI checks green. diff --git a/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/pipeline_status.md b/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/pipeline_status.md deleted file mode 100644 index 5a59a488..00000000 --- a/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/pipeline_status.md +++ /dev/null @@ -1,23 +0,0 @@ -# Pipeline Status — Baseline-phase unit tests depend on suite ordering - -## Idea -- Status: Complete -- File: idea.md - -## Spec -- Status: Approved -- Date: 2026-06-02 -- File: feature_spec.md -- Cross-model review: GPT-5.5 passed (2 cycles, 2 findings — 2 accepted, both Low) -- Phases: 1 total, 1 covered by spec (single-phase) - -## Plan -- Status: Approved -- Date: 2026-06-02 -- File: implementation_plan.md -- Cross-model review: GPT-5.5 passed (2 cycles, 1 finding — rejected-with-counter-evidence, Low) -- Stories: 2 total across 1 epic -- Phases covered: 1 of 1 (single-phase) - -## Implementation -- Status: Not started diff --git a/state.md b/state.md index 01b76edf..bf428ae6 100644 --- a/state.md +++ b/state.md @@ -2,7 +2,7 @@ > Read this first. A one-page snapshot: current focus, the last few merges, what's in flight, what's queued, and where the project sits in the MVP1 → MVP2 → MVP3 → GA roadmap. **Historical feature-merge narrative + chained execution context lives in [`state_history.md`](state_history.md)** — new merge entries land there, not here (per `chore_state_md_size_compression`, 2026-05-29). Keep this file loadable in a single `Read` call. -**Last updated:** 2026-06-05 (`chore_cluster_detail_rung_badge` merged — PR #464, squash `3e03ce7`. **Frontend-only, NO migration** (Alembic head stays `0023`). New `ClusterDetailUbiReadinessCard` on `/clusters/[id]` — query-set picker + debounced target input + auto-seed (single query set + `target_filter`) drive `useUbiReadiness` → ``; the synthetic-UBI `` relocated out of `ClusterDetailSummary` into the card, adjacent to the rung it qualifies. Shared `useUbiReadiness` hook gained `placeholderData: keepPreviousData` (Story 8). Dual leak gate + 404/503 fallback caption + inline cache-invalidate retry. 13-case vitest (MSW + real QueryClient for the placeholderData assertion; AC-9 no-inline-rung-literal static guard), summary regression, one gated real-backend Playwright spec, demo-ubi surface #3 re-anchored. Gemini 2 medium accepted+fixed (instant clear via `targetRaw.trim()`; queryFn null-param guard). All 19 `pr.yml` checks green. Full narrative in [`state_history.md`](state_history.md).) +**Last updated:** 2026-06-05 (`bug_baseline_phase_test_isolation` merged — PR #466, squash `6298e77`. **Backend test-only, NO migration** (Alembic head stays `0023`). `_compute_baseline_wait_s` (`backend/workers/orchestrator.py`) read `get_settings()` unconditionally, so the three explicit-timeout `TestComputeBaselineWaitS` cases only passed when an earlier test module had seeded the secret env vars — they failed standalone. Story 1.1 defers the `get_settings()` read into the falsy-`trial_timeout_s` branch (return values unchanged); Story 1.2 adds an autouse `_settings_env_and_restore` fixture + a `test_explicit_timeout_does_not_read_settings` regression. Standalone with secrets unset: 14 passed (was `3 failed, 1 passed`); full unit suite 2400 passed. No Gemini findings; final GPT-5.5 skipped (≤40 LOC, test-only). All 19 `pr.yml` checks green. Full narrative in [`state_history.md`](state_history.md).) ## Where the roadmap sits @@ -16,9 +16,9 @@ MVP1 (v0.1) **shipped** — all six differentiators live (Bayesian/TPE optimizer ## Current branch / execution context -- **Branch:** `main` (PR #464 `chore_cluster_detail_rung_badge` just merged `3e03ce7`, 2026-06-05). All 19 `pr.yml` checks green (smoke skipped — opt-in/off). -- **Active feature:** _None in flight._ `chore_cluster_detail_rung_badge` shipped 2026-06-05 (PR #464). Next: pull from the MVP2 Idea/Plan backlog (run `/pipeline status`). -- **Alembic head:** `0023_proposals_superseded_status` (unchanged — `chore_cluster_detail_rung_badge` is frontend-only, no migration; head last moved by `feat_overnight_final_solution_phase3` PR #457). +- **Branch:** `main` (PR #466 `bug_baseline_phase_test_isolation` just merged `6298e77`, 2026-06-05). All 19 `pr.yml` checks green (smoke skipped — opt-in/off). +- **Active feature:** _None in flight._ `bug_baseline_phase_test_isolation` shipped 2026-06-05 (PR #466). Next: pull from the MVP2 Idea/Plan backlog (run `/pipeline status`). +- **Alembic head:** `0023_proposals_superseded_status` (unchanged — `bug_baseline_phase_test_isolation` is backend test-only, no migration; head last moved by `feat_overnight_final_solution_phase3` PR #457). - **Python:** 3.13. **Frontend stack:** Next 16 (App Router + Turbopack), React 19, Tailwind 4 (CSS-first), Vitest 4, ESLint 9 (flat), TypeScript 6, Playwright (chromium, single worker) for E2E. - **Coverage gates:** backend 80% (`fail_under` in pyproject), UI vitest + tsc + ESLint + Next build, plus a full-stack smoke E2E job. Live pass counts: see the latest `pr.yml` run (the historical per-feature counts moved to `state_history.md`). @@ -26,17 +26,17 @@ MVP1 (v0.1) **shipped** — all six differentiators live (Bayesian/TPE optimizer Detail + reasoning for each is in [`state_history.md`](state_history.md). +- **2026-06-05** — `bug_baseline_phase_test_isolation` (PR #466, squash-merged `6298e77`). **Hermetic baseline-wait unit tests via lazy settings read.** `_compute_baseline_wait_s` (`backend/workers/orchestrator.py`) called `get_settings()` unconditionally, so the three explicit-timeout `TestComputeBaselineWaitS` cases only passed when an earlier test module had already seeded `DATABASE_URL_FILE`/`POSTGRES_PASSWORD_FILE` — they failed standalone (`3 failed, 1 passed`). **Backend test-only — no migration** (head stays `0023`). 2 stories / 1 epic: (1.1) defer the `get_settings()` read into the missing/falsy-`trial_timeout_s` branch so explicit-timeout callers never construct `Settings` (return values unchanged, falsy-fallback semantics preserved); (1.2) add an autouse `_settings_env_and_restore` fixture (seeds the secret env vars at `/dev/null` + clears the canonical `get_settings` lru_cache — not `orch.get_settings`, which `test_missing_trial_timeout_uses_settings_default` monkeypatches) so the module is hermetic regardless of collection order, plus a `test_explicit_timeout_does_not_read_settings` regression that fails on pre-fix code and passes post-fix. Standalone with secrets unset: 14 passed; full unit suite 2400 passed. No Gemini findings; final GPT-5.5 skipped (≤40 LOC, test-only). All 19 `pr.yml` checks green. - **2026-06-05** — `chore_cluster_detail_rung_badge` (PR #464, squash-merged `3e03ce7`). **Cluster-detail UBI readiness card with rung badge.** A new `ClusterDetailUbiReadinessCard` on `/clusters/[id]` (between the action bar and indices card) lets the operator see a cluster's UBI readiness rung without opening the generate-judgments dialog. **Frontend-only — no backend, no migration** (head stays `0023`). 8 stories / 1 epic, executed 8→1→…→7: (S8) shared `useUbiReadiness` gains `placeholderData: keepPreviousData` so the rung persists across `(query_set_id, target)` edits without a skeleton flash (no-op for the dialog consumer); (S1) card scaffold + `` on the title (reachable in every state); (S2) query-set picker (`limit=50`, `has_more` "Browse all" footer, empty-state Link, explicit **Clear** button since Radix `` + 200ms `useDebouncedValue`; (S4) separate `limit=2` auto-seed probe + once-locked `useEffect` (seeds when `rows.length===1 && !has_more && target_filter` set; locks on first success **or** error so a later refetch can't overwrite operator input — same `set-state-in-effect` disable precedent as `studies/page.tsx:56`); (S5) `useUbiReadiness` gated on `pickerReady` + dual leak gate (`targetRaw.trim()` for instant hide + debounced `target`) → `` + relocated `` (deleted from `ClusterDetailSummary`); (S6) first-fetch skeleton (gated on `data==null` so it never replaces a placeholderData-preserved badge), unified 404/503 fallback caption, inline error + `invalidateQueries` retry; (S7) 13-case vitest (MSW network mocking + real `QueryClientProvider` so AC-8 genuinely exercises `placeholderData`; AC-9 reads the card source and asserts zero inline `rung_[0-3]` literals) + summary regression (chip absent) + one gated real-backend Playwright spec + demo-ubi surface #3 re-anchored to the new placement. CI: prettier flagged the test file on run 1 (local `pnpm lint` doesn't run `prettier --check`) — fixed inline (`6b88e72`). Gemini 2 medium findings, both accepted+fixed (`b0063dd`): instant-clear via `targetRaw.trim().length>0` in `pickerStateValid` (spec "hides immediately"); `useUbiReadiness` queryFn null-param guard against a manual-refetch `?query_set_id=null`. Cross-model final GPT-5.5 skipped (frontend-only, ≤8 files, no studies/judgments/adapter/migration surface). 1233 vitest green. All 19 `pr.yml` checks green. - **2026-06-05** — `feat_ubi_llm_study_comparison` (PR #461, squash-merged `60ba1417`). **Side-by-side LLM-vs-UBI study comparison view.** A new `/studies/compare?a&b` route renders two completed studies (one LLM-graded, one UBI-graded, same query set) side by side with four diff panels. **Read-only — no migration, no write path, no adapter call** (head stays `0023`). 16 stories / 5 epics: (E1) `services/study_comparison.py` — pure `classify_judgment_kind` (`generation_kind=='ubi'`) + `validate_compare_pair` (404 `STUDY_NOT_FOUND`; 422 `COMPARE_STUDY_NOT_COMPLETED`/`COMPARE_QUERY_SET_MISMATCH`/`COMPARE_NOT_LLM_UBI_PAIR`; non-fatal `CROSS_CLUSTER`/`TARGET_MISMATCH`/`OBJECTIVE_MISMATCH` warnings) + repo `find_paired_ubi_llm_study` (opposite-kind completed counterpart on same query_set+cluster via JSONB `generation_kind`; None on 0/>1) + `get_completed_study_for_judgment_list`; (E2) `GET /studies/compare` (declared ABOVE `/studies/{study_id}` so `compare` isn't a path param — AC-8) + `GET /studies/{id}/pair` + `GET /judgment-lists/{id}/study`, all added to the OpenAPI orphan-endpoint allowlist; (E3) `/studies/compare` page + `` (column normalization LLM-left/UBI-right regardless of URL order — AC-18, keyed error state, warning banner, synthetic chip) + four panels (best-metric kind-normalized `ubi−llm` delta direction-aware + zero=neutral + OBJECTIVE_MISMATCH caption; digest-narrative sentence diff via jsdiff `diffSentences`; best-trial param table `=`/`Δ`/em-dash; two-series convergence overlay — consume `convergence.best_so_far_curve` when present else derive from `/trials`) + pure libs `narrative-diff`/`param-diff`/`best-so-far-curve`; (E4) `` on `/studies/{id}` (self-gates on `useStudyPair`, canonical `?a={llm}&b={ubi}`) + "View matched study comparison" on the UBI value-delta card (two-step `useJudgmentListStudy`→`useStudyPair`) + gated real-backend E2E; (E5) tutorial Step 11 + ui-architecture docs. Added `diff` (jsdiff, BSD-3-Clause; license inventory 0 violations). Cross-model: spec 3 GPT-5.5 cycles (26 findings, 0 rejected) pre-merge; plan Opus-only per operator decision; Gemini 8 findings — 5 accepted+fixed (drop redundant `useJudgmentList`; zero-delta neutral; empty-narrative placeholder; `primary_metric != null`; objective-direction `or "maximize"`), 3 rejected (dead None-guards on the non-nullable `judgment_list_id`); final GPT-5.5 skipped per the operator decision. 16 unit + 6 integration + ~15 contract + ~26 vitest + 1 gated E2E; 2399 backend unit + 1219 vitest green. All 18 `pr.yml` checks green. **The two normalizer siblings `feat_apply_path_normalizer_declaration` + `feat_query_normalizer_typed_pipeline` remain product-gated (G-2 operator-friction evidence / Q-1/Q-2 locks) — not auto-executable.** - **2026-06-05** — `feat_query_normalization_tuning` (PR #459, squash-merged `7436bf92`). **Query normalization as a tunable, opt-in query-time parameter (MVP2 Phase 1).** A template declares a reserved Categorical `query_normalizer` param (subset of `none`/`lowercase`/`lowercase+trim`/`lowercase+trim+expand_contractions`, English 30-entry contraction dict) and the Optuna loop tunes it. **Backend + frontend, NO migration** (head stays `0023`). 11 stories / 6 epics: (E1) pure-domain `normalizers.py` (`normalize`, 30-entry frozen `_CONTRACTIONS` + import-time word-boundary regex, `_PR_BODY_NORMALIZER_SNIPPETS`, `validate_normalizer_reservation` + 2 exc) + `compute_default_params` reserves `query_normalizer → "none"` (cycle-2 load-bearing fix) + `template_validator` `_RESERVED_NONRENDER_PARAMS`/`ReservedParamReferenced`; (E2) pre-render hook in BOTH `ElasticAdapter.render` + `SolrAdapter.render` (pop off a local copy, never mutate caller's dict; cross-engine parity test) + router wires `NORMALIZER_CHOICE_INVALID`/`NORMALIZER_PARAM_SHAPE` (POST /studies) + `RESERVED_PARAM_REFERENCED` (POST /query-templates); (E3) PR-body "Operator-side requirement" section (verbatim snippet for non-`none`, explanatory line for `none`; `_render_pr_body_manual` untouched per I-3); (E4) digest analyzer-redundancy advisory (FR-6, `shouldShowNormalizerAdvisory`, whitespace-analyzer excluded) + constrained `query_normalizer` `