diff --git a/docs/00_overview/DASHBOARD.md b/docs/00_overview/DASHBOARD.md index 3cab9e18..1c30f854 100644 --- a/docs/00_overview/DASHBOARD.md +++ b/docs/00_overview/DASHBOARD.md @@ -7,7 +7,7 @@ _Top-level index across MVP1 → GA v1+ as of **2026-05-31**. Click a release na | Release | Theme | Progress | Status | |---|---|---|---| | [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 94 / 94 scoped done | **Complete** | -| [MVP2 / v0.2](MVP2_DASHBOARD.md) | Three-Engine + Real Signals | 6 / 10 scoped done · 13 remaining | **In progress** | +| [MVP2 / v0.2](MVP2_DASHBOARD.md) | Three-Engine + Real Signals | 6 / 10 scoped done · 22 remaining | **In progress** | | MVP3 / v0.3 | Observable | — | **Not yet scoped** | | GA v1 / v1.0 | Production-ready | — | **Not yet scoped** | diff --git a/docs/00_overview/MVP2_DASHBOARD.md b/docs/00_overview/MVP2_DASHBOARD.md index 68c8ccb1..3f5fc678 100644 --- a/docs/00_overview/MVP2_DASHBOARD.md +++ b/docs/00_overview/MVP2_DASHBOARD.md @@ -20,16 +20,16 @@ Plan approved; run /impl-execute to ship | Metric | Value | |---|---| -| Filed under MVP2 | **24** folders total (done + specced not-done + idea backlog + bugs) | +| Filed under MVP2 | **34** folders total (done + specced not-done + idea backlog + bugs) | | Specced features done | **6 / 10** (60%) — of features *past the idea stage* (those with a spec); the idea backlog below is NOT in this denominator, so 100% ≠ release complete | -| Pending work | **18** items (every not-done feat/infra/chore/bug across all priorities) | +| Pending work | **28** items (every not-done feat/infra/chore/bug across all priorities) | | → P0 — do next | **0** unblocking / paying daily cost | | → P1 | **0** high-value, ready when P0 clears | -| → P2 (default) | 14 important to file, not blocking | +| → P2 (default) | 24 important to file, not blocking | | → Backlog | 4 captured for record, not planned | -| Open bugs | 3 | -| Legacy "Path to MVP2" | 13 items — scoped-not-done + bugs + chore-ideas only (excludes feat/infra ideas) | -| Backlog ideas | 5 idea-only feat/infra (not yet scoped into MVP2) | +| Open bugs | 9 | +| Legacy "Path to MVP2" | 22 items — scoped-not-done + bugs + chore-ideas only (excludes feat/infra ideas) | +| Backlog ideas | 6 idea-only feat/infra (not yet scoped into MVP2) | | In flight | 0 feature(s) actively shipping | ## Pipeline @@ -62,24 +62,34 @@ _None._ _None._ -### Idea (14) +### Idea (24) | # | Priority | Feature | Type | One-liner | Depends on | Status | |---|---|---|---|---|---|---| | 1 | P2 | [feat_apply_path_normalizer_declaration](planned_features/02_mvp2/feat_apply_path_normalizer_declaration/idea.md) | Feature | Phase 1 ships option (b) of the prod-reproducibility fork — the PR body documents the chosen normalizer and embeds a Python snippet, and the operator's merge contract is "you must replicate this norma | — | Idea — deferred Phase 3 of [`feat_query_normalization_tuning`](../feat_query_normalization_tuning/feature_spec.md) (§3 Phase boundaries, §19 D-1). Split into its own planned-features folder 2026-05-31 (was `feat_query_normalization_tuning/phase3_idea.md`). | | 2 | P2 | [feat_overnight_studies_summary_card](planned_features/02_mvp2/feat_overnight_studies_summary_card/idea.md) | Feature | Phase 1 makes the overnight chain reviewable from the study detail page — but the operator has to know to go to *some* study in the chain. If they wake up and load `/studies`, the list looks identical | — | Idea — deferred Phase 2 work from [`feat_overnight_autopilot/feature_spec.md`](../feat_overnight_autopilot/feature_spec.md) §3 Phase boundaries. Split into its own planned-features folder 2026-05-31 (was `feat_overnight_autopilot/phase2_idea.md`). | | 3 | P2 | [feat_query_normalizer_typed_pipeline](planned_features/02_mvp2/feat_query_normalizer_typed_pipeline/idea.md) | Feature | Phase 1 ships four built-in Categorical bundles, an English-only ASCII-apostrophe contraction dictionary, and a Python-only PR-body snippet. Three follow-on capabilities are deferred until operator si | — | Idea — deferred Phase 2 of [`feat_query_normalization_tuning`](../feat_query_normalization_tuning/feature_spec.md) (§3 Phase boundaries, §19 D-4 / D-6 / D-7). Split into its own planned-features folder 2026-05-31 (was `feat_query_normalization_tuning/phase2_idea.md`). | -| 4 | P2 | [chore_cluster_detail_rung_badge](planned_features/02_mvp2/chore_cluster_detail_rung_badge/idea.md) | Chore | The `` component ([`ui/src/components/clusters/ubi-rung-badge.tsx`](../ui/src/components/clusters/ubi-rung-badge.tsx)) renders the UBI-readiness rung (rung_0 → rung_3) for a cluster, but | — | Idea — split out from `feat_ubi_llm_study_comparison` §3 (out-of-scope decision) at spec generation | -| 5 | P2 | [chore_studies_post_arq_spy_fixture](planned_features/02_mvp2/chore_studies_post_arq_spy_fixture/idea.md) | Chore | The studies POST handler at [`backend/app/api/v1/studies.py:307`](../../backend/app/api/v1/studies.py#L307) calls `await _enqueue_start_study(request, study_id)` after a successful create. The helper | — | Idea — surfaced during `feat_study_preflight_overlap_probe` (PR ___) phase-gate review | -| 6 | P2 | [chore_template_library_expansion](planned_features/02_mvp2/chore_template_library_expansion/idea.md) | Chore | Three connected gaps: | — | Idea — surfaced during a UX review of parameter-tuning ergonomics on 2026-05-19. | -| 7 | P2 | [chore_ubi_hybrid_template_render](planned_features/02_mvp2/chore_ubi_hybrid_template_render/idea.md) | Chore | Idea — contract decision deferred (NOT a worker bug) | — | Idea — contract decision deferred (NOT a worker bug) | -| 8 | P2 | [chore_ubi_reader_search_after_pagination](planned_features/02_mvp2/chore_ubi_reader_search_after_pagination/idea.md) | Chore | `UbiReader._scan_ubi_events` / `_scan_ubi_queries` each issue ONE `search_batch` (a `size`-limited query). To stay under the engine result-window they now cap at 10000 rows per (target, window).… | — | Idea — deferred from `feat_ubi_judgments` (found during the rung-3 E2E) | -| 9 | P2 | [bug_seed_meaningful_demos_silent_bulk_errors](planned_features/02_mvp2/bug_seed_meaningful_demos_silent_bulk_errors/idea.md) | Bug | [`scripts/seed_meaningful_demos.py:917-935`](../../scripts/seed_meaningful_demos.py#L917-L935) bulk-indexes 1000 Amazon ESCI products into a dedicated index per demo scenario: | — | Idea — captured during `bug_smoke_seed_es_unavailable_shards_race` Phase 2.5 tangential sweep | -| 10 | P2 | [bug_webhook_concurrent_merge_race_timing_sensitive](planned_features/02_mvp2/bug_webhook_concurrent_merge_race_timing_sensitive/idea.md) | Bug | Idea — surfaced during `bug_demo_clusters_unreachable_in_healthz` PR #236 CI. | — | Idea — surfaced during `bug_demo_clusters_unreachable_in_healthz` PR #236 CI. | -| 11 | Backlog | [feat_fts_rank_ordering](planned_features/02_mvp2/feat_fts_rank_ordering/idea.md) | Feature | `feat_data_table_primitive` shipped filter-only FTS — `?q=foo` matches rows where `search_vector @@ plainto_tsquery('english', 'foo')` is true but orders results by `created_at DESC, id DESC` (the def | — | Idea — deferred from `feat_data_table_primitive` (MVP1) per spec §16. | -| 12 | Backlog | [infra_arq_subprocess_test](planned_features/02_mvp2/infra_arq_subprocess_test/idea.md) | Infra | Idea (deferred from `feat_study_lifecycle` Phase 2 / PR #25 final GPT-5.5 review). Still applicable as of 2026-05-14: the three in-process tests cited below still cover the resume contract correctly; | — | Idea (deferred from `feat_study_lifecycle` Phase 2 / PR #25 final GPT-5.5 review). Still applicable as of 2026-05-14: the three in-process tests cited below still cover the resume contract correctly; a subprocess test would add a narrow Arq-version-regression guard. | -| 13 | Backlog | [chore_auto_followup_parent_advisory_lock](planned_features/02_mvp2/chore_auto_followup_parent_advisory_lock/idea.md) | Chore | The shipped `feat_auto_followup_studies` worker uses a two-layer idempotency scheme: | — | Idea — captured as a standalone file to resolve broken cross-references in `feat_auto_followup_studies` D-11 + plan F2 + `bug_auto_followup_completed_parent_stop_chain_race/idea.md`. The slug was coined 2026-05-24 in D-11 but only existed as descriptive prose across other documents until now. | -| 14 | Backlog | [bug_chat_long_conversation_truncation](planned_features/02_mvp2/bug_chat_long_conversation_truncation/idea.md) | Bug | [`backend/app/services/agent_chat.send_user_message`](../../backend/app/services/agent_chat.py) defensively caps the OpenAI history at the most recent `HISTORY_MAX_MESSAGES = 100` messages… | — | Held for MVP2 (decided 2026-05-13). Folder renamed with `_mvp2` suffix to make the deferral visible at-a-glance in `ls docs/00_overview/planned_features/`. Resume work when MVP2 starts — no technical dependency on MVP2 infra (audit_log is N/A; Langfuse is convenience only); the deferral is scope discipline + zero current impact (latent bug, no operator has hit the 100-message cap). | +| 4 | P2 | [infra_generated_artifact_freshness_gate](planned_features/02_mvp2/infra_generated_artifact_freshness_gate/idea.md) | Infra | - `ui/src/lib/types.ts` — regenerated from `http://localhost:8000/openapi.json` via `pnpm types:gen` (`ui/scripts/gen-types.mjs`). Nothing fails CI when a backend schema change lands without a matchin | — | Idea — tangential discovery during `feat_overnight_autopilot` (Story 2.1 + 4.1, PR forthcoming) | +| 5 | P2 | [chore_arq_pool_aclose_deprecation](planned_features/02_mvp2/chore_arq_pool_aclose_deprecation/idea.md) | Chore | [`backend/app/main.py`](../backend/app/main.py) (~line 144, the FastAPI lifespan/shutdown that closes the Arq Redis pool) calls `arq_pool.close()`. arq deprecated the sync-named `close()` in favor of | — | Idea — tangential discovery during `feat_overnight_autopilot` (Epic 1 integration tests, PR forthcoming) | +| 6 | P2 | [chore_cluster_detail_rung_badge](planned_features/02_mvp2/chore_cluster_detail_rung_badge/idea.md) | Chore | The `` component ([`ui/src/components/clusters/ubi-rung-badge.tsx`](../ui/src/components/clusters/ubi-rung-badge.tsx)) renders the UBI-readiness rung (rung_0 → rung_3) for a cluster, but | — | Idea — split out from `feat_ubi_llm_study_comparison` §3 (out-of-scope decision) at spec generation | +| 7 | P2 | [chore_solr_cred_backfill_needs_api_restart](planned_features/02_mvp2/chore_solr_cred_backfill_needs_api_restart/idea.md) | Chore | `scripts/install.sh` step 5a backfills a `local-solr:` entry into a **pre-existing** `./secrets/cluster_credentials.yaml`. That write is correct and idempotent. But the `api` (and `worker`) process re | — | — | +| 8 | P2 | [chore_solr_post_pipeline_followups](planned_features/02_mvp2/chore_solr_post_pipeline_followups/idea.md) | Chore | The 13-story `infra_adapter_solr` execution surfaced several follow-on items that fit neither the original spec nor any sister feature folder. None block the MVP2 Solr release — they're operator-exper | — | Idea — tangential observations from `infra_adapter_solr` end-to-end | +| 9 | P2 | [chore_studies_post_arq_spy_fixture](planned_features/02_mvp2/chore_studies_post_arq_spy_fixture/idea.md) | Chore | The studies POST handler at [`backend/app/api/v1/studies.py:307`](../../backend/app/api/v1/studies.py#L307) calls `await _enqueue_start_study(request, study_id)` after a successful create. The helper | — | Idea — surfaced during `feat_study_preflight_overlap_probe` (PR ___) phase-gate review | +| 10 | P2 | [chore_template_library_expansion](planned_features/02_mvp2/chore_template_library_expansion/idea.md) | Chore | Three connected gaps: | — | Idea — surfaced during a UX review of parameter-tuning ergonomics on 2026-05-19. | +| 11 | P2 | [chore_ubi_hybrid_template_render](planned_features/02_mvp2/chore_ubi_hybrid_template_render/idea.md) | Chore | Idea — contract decision deferred (NOT a worker bug) | — | Idea — contract decision deferred (NOT a worker bug) | +| 12 | P2 | [chore_ubi_reader_search_after_pagination](planned_features/02_mvp2/chore_ubi_reader_search_after_pagination/idea.md) | Chore | `UbiReader._scan_ubi_events` / `_scan_ubi_queries` each issue ONE `search_batch` (a `size`-limited query). To stay under the engine result-window they now cap at 10000 rows per (target, window).… | — | Idea — deferred from `feat_ubi_judgments` (found during the rung-3 E2E) | +| 13 | P2 | [bug_baseline_phase_test_isolation](planned_features/02_mvp2/bug_baseline_phase_test_isolation/idea.md) | Bug | `backend/tests/unit/workers/test_orchestrator_baseline_phase.py::TestComputeBaselineWaitS::*` fail when run in isolation with: | — | Idea — pre-existing bug surfaced during `feat_ubi_judgments` PR #317 | +| 14 | P2 | [bug_demo_seed_ubi_insufficient_data](planned_features/02_mvp2/bug_demo_seed_ubi_insufficient_data/idea.md) | Bug | | — | — | +| 15 | P2 | [bug_e2e_teardown_chain_node_delete_500](planned_features/02_mvp2/bug_e2e_teardown_chain_node_delete_500/idea.md) | Bug | The E2E global-teardown deletes seeded rows in a fixed order (per `chore_e2e_test_rows_isolation` Story 1.2 cleanup registration). For auto-followup **chains**, the seeded nodes are `queued` studies c | — | Idea — tangential discovery during `feat_overnight_autopilot` (Story 4.2 E2E, PR forthcoming) | +| 16 | P2 | [bug_judgment_header_omits_click_bucket](planned_features/02_mvp2/bug_judgment_header_omits_click_bucket/idea.md) | Bug | A judgment list generated from UBI (or hybrid UBI+LLM) carries non-zero `source_breakdown.click`, but the header's source-breakdown card silently drops it. Operators reviewing a UBI list see only the | — | Idea — tangential discovery during `feat_overnight_autopilot` (Story 2.1, PR forthcoming) | +| 17 | P2 | [bug_relyloop_spec_ubi_section_drift](planned_features/02_mvp2/bug_relyloop_spec_ubi_section_drift/idea.md) | Bug | [`docs/00_overview/relyloop-spec.md`](relyloop-spec.md) §"Click-derived judgments — OpenSearch UBI as the engine-neutral primary path" (line ~706) carries two staleness bugs from the 2026-05-27 releas | — | Idea — captured during `feat_ubi_judgments` preflight (2026-05-29) | +| 18 | P2 | [bug_reseed_failure_blocks_retry_arq_singleton_dedup](planned_features/02_mvp2/bug_reseed_failure_blocks_retry_arq_singleton_dedup/idea.md) | Bug | `run_demo_reseed` is enqueued with a fixed Arq job id `demo_reseed:singleton` (the singleton concurrency guard). When a run reaches a terminal state, Arq stores its **result** under `arq:result:demo_r | — | Idea — tangential discovery while verifying `fix(demo): add Solr (8983) to the reseed engine host-URL mapping` (branch `feat_demo_reseed_solr_and_steplog`) | +| 19 | P2 | [bug_seed_meaningful_demos_silent_bulk_errors](planned_features/02_mvp2/bug_seed_meaningful_demos_silent_bulk_errors/idea.md) | Bug | [`scripts/seed_meaningful_demos.py:917-935`](../../scripts/seed_meaningful_demos.py#L917-L935) bulk-indexes 1000 Amazon ESCI products into a dedicated index per demo scenario: | — | Idea — captured during `bug_smoke_seed_es_unavailable_shards_race` Phase 2.5 tangential sweep | +| 20 | P2 | [bug_webhook_concurrent_merge_race_timing_sensitive](planned_features/02_mvp2/bug_webhook_concurrent_merge_race_timing_sensitive/idea.md) | Bug | Idea — surfaced during `bug_demo_clusters_unreachable_in_healthz` PR #236 CI. | — | Idea — surfaced during `bug_demo_clusters_unreachable_in_healthz` PR #236 CI. | +| 21 | Backlog | [feat_fts_rank_ordering](planned_features/02_mvp2/feat_fts_rank_ordering/idea.md) | Feature | `feat_data_table_primitive` shipped filter-only FTS — `?q=foo` matches rows where `search_vector @@ plainto_tsquery('english', 'foo')` is true but orders results by `created_at DESC, id DESC` (the def | — | Idea — deferred from `feat_data_table_primitive` (MVP1) per spec §16. | +| 22 | Backlog | [infra_arq_subprocess_test](planned_features/02_mvp2/infra_arq_subprocess_test/idea.md) | Infra | Idea (deferred from `feat_study_lifecycle` Phase 2 / PR #25 final GPT-5.5 review). Still applicable as of 2026-05-14: the three in-process tests cited below still cover the resume contract correctly; | — | Idea (deferred from `feat_study_lifecycle` Phase 2 / PR #25 final GPT-5.5 review). Still applicable as of 2026-05-14: the three in-process tests cited below still cover the resume contract correctly; a subprocess test would add a narrow Arq-version-regression guard. | +| 23 | Backlog | [chore_auto_followup_parent_advisory_lock](planned_features/02_mvp2/chore_auto_followup_parent_advisory_lock/idea.md) | Chore | The shipped `feat_auto_followup_studies` worker uses a two-layer idempotency scheme: | — | Idea — captured as a standalone file to resolve broken cross-references in `feat_auto_followup_studies` D-11 + plan F2 + `bug_auto_followup_completed_parent_stop_chain_race/idea.md`. The slug was coined 2026-05-24 in D-11 but only existed as descriptive prose across other documents until now. | +| 24 | Backlog | [bug_chat_long_conversation_truncation](planned_features/02_mvp2/bug_chat_long_conversation_truncation/idea.md) | Bug | [`backend/app/services/agent_chat.send_user_message`](../../backend/app/services/agent_chat.py) defensively caps the OpenAI history at the most recent `HISTORY_MAX_MESSAGES = 100` messages… | — | Held for MVP2 (decided 2026-05-13). Folder renamed with `_mvp2` suffix to make the deferral visible at-a-glance in `ls docs/00_overview/planned_features/`. Resume work when MVP2 starts — no technical dependency on MVP2 infra (audit_log is N/A; Langfuse is convenience only); the deferral is scope discipline + zero current impact (latent bug, no operator has hit the 100-message cap). | ## Dependency graph diff --git a/docs/00_overview/dashboard.html b/docs/00_overview/dashboard.html index 2d1ddf41..74404c9c 100644 --- a/docs/00_overview/dashboard.html +++ b/docs/00_overview/dashboard.html @@ -392,7 +392,7 @@

Releases

MVP2 / v0.2
Three-Engine + Real Signals
-
6 / 10 scoped done · 13 remaining
+
6 / 10 scoped done · 22 remaining
In progress
diff --git a/docs/00_overview/mvp2_dashboard.html b/docs/00_overview/mvp2_dashboard.html index cf402953..224581c9 100644 --- a/docs/00_overview/mvp2_dashboard.html +++ b/docs/00_overview/mvp2_dashboard.html @@ -398,17 +398,17 @@

MVP2 Progress

Specced features done
6 / 10
-
60% specced · 24 filed under MVP2
+
60% specced · 34 filed under MVP2
Pending work
-
18
+
28
every not-done feat/infra/chore/bug across all priorities
Open bugs
-
3
+
9
tracked bug_* idea files
@@ -425,7 +425,7 @@

MVP2 Progress

P2 (default)
-
14
+
24
important to file, not blocking
@@ -435,14 +435,14 @@

MVP2 Progress

Legacy "Path to MVP2"
-
13
+
22
scoped not-done + bugs + chore-ideas only (excludes feat/infra ideas)
Backlog ideas: - 5 idea-only feat/infra folders (not yet scoped into MVP2) + 6 idea-only feat/infra folders (not yet scoped into MVP2) In flight: @@ -463,7 +463,7 @@

Pipeline

-

Idea 14

+

Idea 24

Apply Path Normalizer Declaration
@@ -504,6 +504,32 @@

Idea 14

+
+
Generated Artifact Freshness Gate
+
+ Infra + P2 + +
+
- `ui/src/lib/types.ts` — regenerated from `http://localhost:8000/openapi.json` via `pnpm types:gen` (`ui/scripts/gen-types.mjs`). Nothing fails CI when a backend schema change lands without a matchin
+ + +
+ + +
+
Arq Pool Aclose Deprecation
+
+ Chore + P2 + +
+
[`backend/app/main.py`](../backend/app/main.py) (~line 144, the FastAPI lifespan/shutdown that closes the Arq Redis pool) calls `arq_pool.close()`. arq deprecated the sync-named `close()` in favor of
+ + +
+ +
Cluster Detail Rung Badge
@@ -517,6 +543,32 @@

Idea 14

+
+
Solr Cred Backfill Needs Api Restart
+
+ Chore + P2 + +
+
`scripts/install.sh` step 5a backfills a `local-solr:` entry into a **pre-existing** `./secrets/cluster_credentials.yaml`. That write is correct and idempotent. But the `api` (and `worker`) process re
+ + +
+ + +
+
Solr Post Pipeline Followups
+
+ Chore + P2 + +
+
The 13-story `infra_adapter_solr` execution surfaced several follow-on items that fit neither the original spec nor any sister feature folder. None block the MVP2 Solr release — they're operator-exper
+ + +
+ +
Studies Post Arq Spy Fixture
@@ -569,6 +621,84 @@

Idea 14

+
+
Baseline Phase Test Isolation
+
+ Bug + P2 + +
+
`backend/tests/unit/workers/test_orchestrator_baseline_phase.py::TestComputeBaselineWaitS::*` fail when run in isolation with:
+ + +
+ + +
+
Demo Seed Ubi Insufficient Data
+
+ Bug + P2 + +
+
+ + +
+ + +
+
E2E Teardown Chain Node Delete 500
+
+ Bug + P2 + +
+
The E2E global-teardown deletes seeded rows in a fixed order (per `chore_e2e_test_rows_isolation` Story 1.2 cleanup registration). For auto-followup **chains**, the seeded nodes are `queued` studies c
+ + +
+ + +
+
Judgment Header Omits Click Bucket
+
+ Bug + P2 + +
+
A judgment list generated from UBI (or hybrid UBI+LLM) carries non-zero `source_breakdown.click`, but the header's source-breakdown card silently drops it. Operators reviewing a UBI list see only the
+ + +
+ + +
+
Relyloop Spec Ubi Section Drift
+
+ Bug + P2 + +
+
[`docs/00_overview/relyloop-spec.md`](relyloop-spec.md) §"Click-derived judgments — OpenSearch UBI as the engine-neutral primary path" (line ~706) carries two staleness bugs from the 2026-05-27 releas
+ + +
+ + +
+
Reseed Failure Blocks Retry Arq Singleton Dedup
+
+ Bug + P2 + +
+
`run_demo_reseed` is enqueued with a fixed Arq job id `demo_reseed:singleton` (the singleton concurrency guard). When a run reaches a terminal state, Arq stores its **result** under `arq:result:demo_r
+ + +
+ +
Seed Meaningful Demos Silent Bulk Errors
diff --git a/docs/00_overview/planned_features/00_unsure/bug_baseline_phase_test_isolation/idea.md b/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/bug_baseline_phase_test_isolation/idea.md rename to docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/idea.md diff --git a/docs/00_overview/planned_features/00_unsure/bug_demo_seed_ubi_insufficient_data/idea.md b/docs/00_overview/planned_features/02_mvp2/bug_demo_seed_ubi_insufficient_data/idea.md similarity index 90% rename from docs/00_overview/planned_features/00_unsure/bug_demo_seed_ubi_insufficient_data/idea.md rename to docs/00_overview/planned_features/02_mvp2/bug_demo_seed_ubi_insufficient_data/idea.md index 83c541f7..3ac0e105 100644 --- a/docs/00_overview/planned_features/00_unsure/bug_demo_seed_ubi_insufficient_data/idea.md +++ b/docs/00_overview/planned_features/02_mvp2/bug_demo_seed_ubi_insufficient_data/idea.md @@ -2,6 +2,7 @@ **Type:** bug **Priority:** P2 (demo-only; does not affect the product/adapter code paths) +**Re-verify (2026-05-31):** the **async** home-button reseed (`demo_seeding.py`, the UI path) now completes **6/6 incl. UBI judgment generation on all scenarios** after `feat_demo_reseed_solr_and_steplog` (PR #348). This bug is about the **sync CLI** path (`make seed-demo FORCE=1` → `seed_meaningful_demos.py`) + the synthetic-event-count vs readiness-threshold, which #348 did not directly touch — but re-run the sync CLI to confirm the symptom persists before scoping; it may now be stale/resolved. Moved from `00_unsure/` to `02_mvp2/` 2026-05-31. **Origin:** Surfaced during the infra_adapter_solr rework (2026-05-30), Phase 8, while running `make seed-demo FORCE=1` to investigate pre-existing UBI demo failures. **NOT caused by the Solr adapter** — the failing scenarios are all diff --git a/docs/00_overview/planned_features/00_unsure/bug_e2e_teardown_chain_node_delete_500/idea.md b/docs/00_overview/planned_features/02_mvp2/bug_e2e_teardown_chain_node_delete_500/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/bug_e2e_teardown_chain_node_delete_500/idea.md rename to docs/00_overview/planned_features/02_mvp2/bug_e2e_teardown_chain_node_delete_500/idea.md diff --git a/docs/00_overview/planned_features/00_unsure/bug_judgment_header_omits_click_bucket/idea.md b/docs/00_overview/planned_features/02_mvp2/bug_judgment_header_omits_click_bucket/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/bug_judgment_header_omits_click_bucket/idea.md rename to docs/00_overview/planned_features/02_mvp2/bug_judgment_header_omits_click_bucket/idea.md diff --git a/docs/00_overview/planned_features/00_unsure/bug_relyloop_spec_ubi_section_drift/idea.md b/docs/00_overview/planned_features/02_mvp2/bug_relyloop_spec_ubi_section_drift/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/bug_relyloop_spec_ubi_section_drift/idea.md rename to docs/00_overview/planned_features/02_mvp2/bug_relyloop_spec_ubi_section_drift/idea.md diff --git a/docs/00_overview/planned_features/00_unsure/bug_reseed_failure_blocks_retry_arq_singleton_dedup/idea.md b/docs/00_overview/planned_features/02_mvp2/bug_reseed_failure_blocks_retry_arq_singleton_dedup/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/bug_reseed_failure_blocks_retry_arq_singleton_dedup/idea.md rename to docs/00_overview/planned_features/02_mvp2/bug_reseed_failure_blocks_retry_arq_singleton_dedup/idea.md diff --git a/docs/00_overview/planned_features/00_unsure/chore_arq_pool_aclose_deprecation/idea.md b/docs/00_overview/planned_features/02_mvp2/chore_arq_pool_aclose_deprecation/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/chore_arq_pool_aclose_deprecation/idea.md rename to docs/00_overview/planned_features/02_mvp2/chore_arq_pool_aclose_deprecation/idea.md diff --git a/docs/00_overview/planned_features/00_unsure/chore_solr_cred_backfill_needs_api_restart/idea.md b/docs/00_overview/planned_features/02_mvp2/chore_solr_cred_backfill_needs_api_restart/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/chore_solr_cred_backfill_needs_api_restart/idea.md rename to docs/00_overview/planned_features/02_mvp2/chore_solr_cred_backfill_needs_api_restart/idea.md diff --git a/docs/00_overview/planned_features/00_unsure/chore_solr_post_pipeline_followups/idea.md b/docs/00_overview/planned_features/02_mvp2/chore_solr_post_pipeline_followups/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/chore_solr_post_pipeline_followups/idea.md rename to docs/00_overview/planned_features/02_mvp2/chore_solr_post_pipeline_followups/idea.md diff --git a/docs/00_overview/planned_features/00_unsure/infra_generated_artifact_freshness_gate/idea.md b/docs/00_overview/planned_features/02_mvp2/infra_generated_artifact_freshness_gate/idea.md similarity index 100% rename from docs/00_overview/planned_features/00_unsure/infra_generated_artifact_freshness_gate/idea.md rename to docs/00_overview/planned_features/02_mvp2/infra_generated_artifact_freshness_gate/idea.md diff --git a/state.md b/state.md index 0cf8caef..a9f8708e 100644 --- a/state.md +++ b/state.md @@ -2,7 +2,7 @@ > Read this first. A one-page snapshot: current focus, the last few merges, what's in flight, what's queued, and where the project sits in the MVP1 → MVP2 → MVP3 → GA roadmap. **Historical feature-merge narrative + chained execution context lives in [`state_history.md`](state_history.md)** — new merge entries land there, not here (per `chore_state_md_size_compression`, 2026-05-29). Keep this file loadable in a single `Read` call. -**Last updated:** 2026-05-31 (`feat_overnight_autopilot` finalized — surfaces the auto-followup chain as a first-class "run overnight" path; PR #343 squash-merged `fe146950`, folder moved to `implemented_features/`. MVP2 now 6/6 scoped done.) +**Last updated:** 2026-05-31 (`feat_demo_reseed_solr_and_steplog` merged — completes the Solr demo-seed in the reseed (deferred A13) + fixes a UBI-on-Solr product bug + adds a reseed step-log; PR #348 squash-merged `66323aba`. `make seed-demo` now completes 6/6 incl. the Solr scenario.) ## Where the roadmap sits @@ -14,8 +14,8 @@ MVP1 (v0.1) **shipped** — all six differentiators live (Bayesian/TPE optimizer ## Current branch / execution context -- **Branch:** `main` (clean). `feat_overnight_autopilot` merged via PR #343 (squash, merge commit `fe146950`); finalization on `docs/finalize-overnight-autopilot` (this branch). -- **Active feature:** _None in flight._ `feat_overnight_autopilot` shipped — read-only `GET /api/v1/studies/{id}/chain` chain-summary endpoint + the `AutoFollowupChainPanel` rolled-up summary + the "🌙 Run overnight (compound automatically)" wizard relabel. The chaining engine itself was untouched (read-only). Next: pull from the MVP2 Idea backlog (run `/pipeline status`). +- **Branch:** `main` (clean). `feat_demo_reseed_solr_and_steplog` merged via PR #348 (squash, merge commit `66323aba`); finalization on `docs/finalize-demo-reseed-solr` (this branch). +- **Active feature:** _None in flight._ `feat_demo_reseed_solr_and_steplog` shipped — the async demo reseed now seeds the Solr scenario end-to-end (configset+collection create, engine-aware synthetic-UBI write **and** read, search-space cardinality fix), fixing a real UBI-on-Solr product bug (UBI judgment generation was ES-DSL-only → broke on Solr for all operators); plus a reseed step-history log surface. Verified by a live `make seed-demo` → 6/6. Next: pull from the MVP2 Idea backlog (run `/pipeline status`). - **Alembic head:** `0022_solr_engine_auth_check` (added by `infra_adapter_solr` Story A6 — extends `clusters.engine_type` + `clusters.auth_kind` CHECK constraints for Solr). - **Python:** 3.13. **Frontend stack:** Next 16 (App Router + Turbopack), React 19, Tailwind 4 (CSS-first), Vitest 4, ESLint 9 (flat), TypeScript 6, Playwright (chromium, single worker) for E2E. - **Coverage gates:** backend 80% (`fail_under` in pyproject), UI vitest + tsc + ESLint + Next build, plus a full-stack smoke E2E job. Live pass counts: see the latest `pr.yml` run (the historical per-feature counts moved to `state_history.md`). @@ -24,15 +24,14 @@ MVP1 (v0.1) **shipped** — all six differentiators live (Bayesian/TPE optimizer Detail + reasoning for each is in [`state_history.md`](state_history.md). +- **2026-05-31** — `feat_demo_reseed_solr_and_steplog` (PR #348, squash-merged `66323aba`). Completes the deferred `infra_adapter_solr` Story A13 — the async home-button demo reseed (`make seed-demo`) now seeds the **Solr** scenario (`acme-kb-docs-solr`) end-to-end, where before it crashed. Fixed layer by layer (each verified by the live reseed getting further): Solr host-URL→Compose-DNS mapping (`8983`); the dispatcher now creates the Solr collection from its configset + bulk-indexes (reusing `seed_solr_products`) instead of the ES `index_mapping` PUT; engine-aware **synthetic-UBI write** (`ubi_queries`/`ubi_events` Solr collections + Solr `/update`, per-engine ensure gate); study **search-space cardinality** (Solr scenario tunes 2 boosts, `estimate_cardinality` floats=100 → ≤3-float cap). **Surfaced + fixed a real product bug:** the UBI **read** path (`ubi_readiness`, `UbiReader`) built Elasticsearch query DSL → `UBI judgment generation was broken on Solr for every operator` (contradicting the "works everywhere from day one" claim); now builds Solr `q`/`fq`/`{!terms}`/`rows`/`fl` when `adapter.engine_type=="solr"`. Plus a **reseed step-history log** (worker accumulates `steps[]` → status endpoint → scrolling UI panel). **Verified by a live `make seed-demo` completing 6/6** (10 studies, 10 proposals). 8 commits; backend 2012 unit + frontend 983 vitest. Gemini: 3 findings accepted+fixed (`{!terms}` query_id vs maxBooleanClauses — verified live; collision-safe synthetic ids; scroll-on-reopen dep). Final GPT-5.5 review clean. Captured `bug_reseed_failure_blocks_retry_arq_singleton_dedup` (a failed reseed deduped retries for ~1h via the Arq singleton result). - **2026-05-31** — `feat_overnight_autopilot` (PR #343, squash-merged `fe146950`). MVP2 ergonomics feature surfacing the shipped auto-followup chaining engine as a first-class "set it and wake up to results" path — **read-side + UI only, the engine stayed untouched**. New read-only `GET /api/v1/studies/{id}/chain` (pure-domain `chain_summary.py`: stop-reason matrix, universal cumulative-lift, best-link selection — reuses `compute_first_decile_max`/`_direction_normalized_lift` from `auto_followup.py`; + a bounded `parent_study_id` traversal repo helper with a 10-hop upward cap + cycle guard, `LIMIT 1` downward walk, `DISTINCT ON` newest-non-rejected proposal lookup). Frontend: create-study wizard relabel to "🌙 Run overnight (compound automatically)" + Deep-preset discoverability hint + `overnight_autopilot` glossary key; `AutoFollowupChainPanel` rolled-up summary (ordered links + deltas, cumulative lift, 3-branch best-config, stop-reason phrases) via a new `useStudyChain` TanStack hook (focus/cancel/transition refetch + 120s grace poll). Tutorial Step 12 + arch-doc notes. No migration (Alembic head stays `0022`). 7 stories / 4 epics. Tests: 30 unit + 31 chain integration/contract + frontend vitest (panel 13/wizard/glossary) + 1 real-backend E2E. Cross-model: GPT-5.5 Epic 1 (1 Low accepted) + Epics 2+3 clean + final (1 Medium rejected — SQLite-portability moot, Postgres-only); Gemini (1 High accepted — zero-row hydration guard `9b1d894f`). Phase 2 ("ran while away" card) deferred to `feat_overnight_studies_summary_card`. 4 tangential idea files captured (`infra_generated_artifact_freshness_gate`, `bug_judgment_header_omits_click_bucket`, `chore_arq_pool_aclose_deprecation`, `bug_e2e_teardown_chain_node_delete_500`). - **2026-05-31** — `infra_adapter_solr` (PR #336 squash-merged `60aec9af` + demo-seed fix #337). MVP2 three-engine headliner: a single `SolrAdapter` implements the full `SearchAdapter` Protocol against Apache Solr 9.x/10.x (SolrCloud + standalone), pivoting on a construction-time capability probe. All 13 stories (A1–A13): adapter skeleton + probe, `edismax`/`dismax`/`lucene` render with unified-param pivots, parallel `/select` search_batch, get_schema/list_targets, explain (Lucene-escaped), get_document/list_documents (RealTime Get + cursorMark), LTR rescore (`rq={!ltr}`) + `LTR_MODEL_NOT_FOUND`, migration 0022 (engine_type + auth_kind CHECK extension) + `registry.py` allowlist relocation, `POST /clusters/{id}/reprobe` + `POST /clusters/test-connection`, Compose `solr:10.0` service + configsets + `/healthz` subsystems.solr, frontend wire literals + per-engine auth filtering + 3-engine `` + test-connection button, Guide 01 + runbook + tutorial Path C, and a 5th `acme-kb-docs-solr` demo scenario. **Live-Solr rework correction** (folded into the squash): local Solr runs security-DISABLED (parity with ES/OpenSearch), `bootstrap-security.sh` deleted, LTR via `SOLR_MODULES=ltr`; stock Solr ships **no** `solr.UBIComponent`, so UBI on Solr is read-path-only (demo synthesizes events, probe reports `ubi_component_present=false`). Cross-model review: GPT-5.5 F1/F2/F3 + 6 Gemini findings adjudicated. Post-pipeline followups tracked in `00_unsure/chore_solr_post_pipeline_followups` + `chore_solr_cred_backfill_needs_api_restart`. - **2026-05-30** — `chore_oss_public_launch_punchlist` (PRs #322, #330, + history-audit PR). Closed the 3-capability OSS public-launch gate. (1) SPDX headers via FSFE REUSE on every source file + `REUSE.toml` + `reuse-lint` pre-commit/CI gates (1477/1477 compliant). (2) Dependency license inventory (`scripts/gen_license_inventory.py` → `docs/04_security/license-inventory.md`, 786 deps / 0 violations, deterministic from locked closure) + `license-inventory` CI gate; 9 non-permissive licenses all adjudicated Accept (no GPL/AGPL ships). (3) Full-history `gitleaks` + manual sweep — 1 gitleaks finding + a few pickaxe hits, all confirmed false positives — captured in a repeatable runbook (`docs/03_runbooks/oss-history-audit.md`). Repo cleared for the visibility flip (operator action). NOTE: `SKIP_HEAVY_CI=true` during this work, so `license-headers`/`license-inventory` jobs were verified locally, not in CI. - **2026-05-29** — `feat_ubi_judgments` (PR #317, squash-merged). MVP2 second feature: engine-neutral User Behavior Insights judgment generation, shipped end-to-end (all 13 stories incl. E2E + DB-backed integration tests + operator docs — none deferred). Migration 0021 (judgment_lists.generation_params JSONB) → domain/ubi/ pure-domain library (FeatureVec, async SignalsConverter Protocol + 3 impls, position-bias prior) → UbiReader (engine-neutral two-index scan + client-side join, no new adapter method) → ubi_readiness classifier (rung_0..rung_3, 60s Redis cache) → start_ubi_judgment_generation dispatcher (refactor extracts 5 shared helpers; LLM dispatcher parity preserved by all 12 existing tests) → 5 new wire Literals + _SourceBreakdown three-term evolution (FR-10) → GET /clusters/{id}/ubi-readiness + POST /judgments/generate-from-ubi endpoints → generate_judgments_from_ubi Arq worker with mapping_strategy + hybrid LLM-fill callback → 21st agent tool + orchestrator prompt update → frontend method picker dialog + on-ramp nudge + sparse-data card + value-delta + ambiguous-skip recovery cards → operator runbook + 3 FAQ entries + data-model patches → 4 Playwright E2E specs (rung_0/rung_3/hybrid/source-filter) green against the live ES-backed stack. **Real-engine E2E caught a production bug**: UbiReader requested `size=50000 > ES index.max_result_window (10000)` → "all shards failed" swallowed by the adapter → spurious `UBI_INSUFFICIENT_DATA` on dense clusters; fixed by clamping both index scans to `ES_MAX_RESULT_WINDOW=10000` + regression guard. Cross-model review: 6 Gemini findings + 6 GPT-5.5 findings all adjudicated (fixed or documented as working-as-designed). Remaining follow-ups are pure deferrals, not gaps: `chore_ubi_reader_search_after_pagination` (P2, >10k-event clusters), `chore_ubi_hybrid_template_render` (P3, vestigial-template contract cleanup — current behavior correct per FR-2), `feat_demo_ubi_study_comparison` (P1, side-by-side UBI-vs-LLM demo study). -- **2026-05-30** — `feat_demo_ubi_study_comparison` Phase 1 (PR #320, squash-merged `853a5053`). MVP2 third feature: the demo reseed now seeds **synthetic UBI clickstream** on 3 of 4 demo clusters so the on-ramp ladder (rung_0→rung_3) is browser-visible without operator setup, plus a "Synthetic demo data" disclosure chip on 5 UI surfaces. 14 stories across 4 epics; pure-domain generator + allowlist-guarded writer + canonical `samples/ubi_index_mappings.json` + dual (LLM)/(UBI) demo studies. Phase 2 (side-by-side LLM-vs-UBI comparison view) + the deferred cluster-detail rung badge split out to `feat_ubi_llm_study_comparison`. - ## In flight -- _None._ `feat_overnight_autopilot` (PR #343) merged 2026-05-31; finalization on `docs/finalize-overnight-autopilot` (this branch). +- _None._ `feat_demo_reseed_solr_and_steplog` (PR #348) merged 2026-05-31; finalization on `docs/finalize-demo-reseed-solr` (this branch). ## Queued (priority-ordered by dashboard / dep graph) diff --git a/website/docs/roadmap.md b/website/docs/roadmap.md index 2b1640e3..63d26d6b 100644 --- a/website/docs/roadmap.md +++ b/website/docs/roadmap.md @@ -180,18 +180,28 @@ - 🟡 [Study Convergence Indicator](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/feat_study_convergence_indicator) - 🟡 [UBI LLM Study Comparison](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/feat_ubi_llm_study_comparison) -??? note "Infrastructure & tooling (2)" +??? note "Infrastructure & tooling (3)" - ✅ [Adapter Solr](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/implemented_features/2026_05_31_infra_adapter_solr) · [#336](https://github.com/SoundMindsAI/relyloop/pull/336) - 🟡 [Arq Subprocess Test](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/infra_arq_subprocess_test) + - 🟡 [Generated Artifact Freshness Gate](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/infra_generated_artifact_freshness_gate) -??? note "Maintenance & fixes (10)" +??? note "Maintenance & fixes (19)" + - 🟡 [Arq Pool Aclose Deprecation](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_arq_pool_aclose_deprecation) - 🟡 [Auto Followup Parent Advisory Lock](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_auto_followup_parent_advisory_lock) + - 🟡 [Baseline Phase Test Isolation](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation) - 🟡 [Chat Long Conversation Truncation](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/bug_chat_long_conversation_truncation) - 🟡 [Cluster Detail Rung Badge](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_cluster_detail_rung_badge) + - 🟡 [Demo Seed UBI Insufficient Data](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/bug_demo_seed_ubi_insufficient_data) - 🟡 [Demo Seeding Integration Tests Rewrite](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_demo_seeding_integration_tests_rewrite) + - 🟡 [E2E Teardown Chain Node Delete 500](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/bug_e2e_teardown_chain_node_delete_500) + - 🟡 [Judgment Header Omits Click Bucket](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/bug_judgment_header_omits_click_bucket) + - 🟡 [Relyloop Spec UBI Section Drift](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/bug_relyloop_spec_ubi_section_drift) + - 🟡 [Reseed Failure Blocks Retry Arq Singleton Dedup](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/bug_reseed_failure_blocks_retry_arq_singleton_dedup) - 🟡 [Seed Meaningful Demos Silent Bulk Errors](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/bug_seed_meaningful_demos_silent_bulk_errors) + - 🟡 [Solr Cred Backfill Needs API Restart](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_solr_cred_backfill_needs_api_restart) + - 🟡 [Solr Post Pipeline Followups](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_solr_post_pipeline_followups) - 🟡 [Studies Post Arq Spy Fixture](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_studies_post_arq_spy_fixture) - 🟡 [Template Library Expansion](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_template_library_expansion) - 🟡 [UBI Hybrid Template Render](https://github.com/SoundMindsAI/relyloop/tree/main/docs/00_overview/planned_features/02_mvp2/chore_ubi_hybrid_template_render)