Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/00_overview/DASHBOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ _Top-level index across MVP1 → GA v1+ as of **2026-05-22**. Click a release na

| Release | Theme | Progress | Status |
|---|---|---|---|
| [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 61 / 62 scoped done · 10 remaining | **In progress** |
| [MVP1 / v0.1](MVP1_DASHBOARD.md) | The Loop | 62 / 62 scoped done · 9 remaining | **In progress** |
| [MVP2 / v0.2](MVP2_DASHBOARD.md) | Observable | 1 / 1 scoped done · 1 remaining | **In progress** |
| MVP3 / v0.3 | Production Stacks | — | **Not yet scoped** |
| MVP4 / v0.4 | Multi-tenant, Multi-LLM | — | **Not yet scoped** |
Expand Down
31 changes: 12 additions & 19 deletions docs/00_overview/MVP1_DASHBOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,34 +6,28 @@ _Reflects feature-folder state as of **2026-05-22** (latest mtime of any planned

## Next up

**[feat_study_preflight_overlap_probe](../02_product/planned_features/feat_study_preflight_overlap_probe/feature_spec.md)** — Feature, currently in **Plan**
All scoped MVP1 features shipped 🎉

> `POST /api/v1/studies` issues a single bounded `ids`-existence query against the study's target asking "for the *first* query in the query set that has any judgments (chosen deterministically by `id ASC`), do its judged doc IDs still exist

Plan approved; run /impl-execute to ship

```bash
/impl-execute docs/02_product/planned_features/feat_study_preflight_overlap_probe/implementation_plan.md --all
```
Pull from the Idea backlog or capture a new feature spec.

## MVP1 Progress

| Metric | Value |
|---|---|
| Scoped items done | **61 / 62** (98%) — feat_/infra_/chore_/epic_ past idea stage |
| Pending work | **18** items (every not-done feat/infra/chore/bug across all priorities) |
| Scoped items done | **62 / 62** (100%) — feat_/infra_/chore_/epic_ past idea stage |
| Pending work | **17** items (every not-done feat/infra/chore/bug across all priorities) |
| → P0 — do next | **0** unblocking / paying daily cost |
| → P1 | **4** high-value, ready when P0 clears |
| → P1 | **3** high-value, ready when P0 clears |
| → P2 (default) | 13 important to file, not blocking |
| → Backlog | 1 captured for record, not planned |
| Open bugs | 2 |
| Legacy "Path to MVP1" | 10 items — scoped-not-done + bugs + chore-ideas only (excludes feat/infra ideas) |
| Legacy "Path to MVP1" | 9 items — scoped-not-done + bugs + chore-ideas only (excludes feat/infra ideas) |
| Backlog ideas | 8 idea-only feat/infra (not yet scoped into MVP1) |
| In flight | 0 feature(s) actively shipping |

## Pipeline

### Done (73)
### Done (74)

| Feature | Type | One-liner | Depends on | Status |
|---|---|---|---|---|
Expand All @@ -56,6 +50,7 @@ Plan approved; run /impl-execute to ship
| [feat_query_inline_crud](implemented_features/2026_05_14_feat_query_inline_crud/feature_spec.md) | Feature | A relevance engineer on the `/query-sets/[id]` page sees a paginated table of every query in the set with `query_text`, `reference_answer`, `query_metadata`, and a `judgment_count` derived field. | `infra_foundation` `infra_adapter_elastic` `feat_study_lifecycle` `feat_llm_judgments` `feat_studies_ui` | [PR #101](https://github.com/SoundMindsAI/relyloop/pull/101) merged 2026-05-14 |
| [feat_studies_ui](implemented_features/2026_05_12_feat_studies_ui/feature_spec.md) | Feature | A Next.js app provides 9 of the 11 MVP1 routes from [`ui-architecture.md` §"Routes (MVP1)"](../01_architecture/ui-architecture.md): dashboard, clusters list/detail, query sets list/detail, judgment re | `infra_foundation` `feat_study_lifecycle` `feat_digest_proposal` `feat_llm_judgments` `infra_adapter_elastic` | [PR #50](https://github.com/SoundMindsAI/relyloop/pull/50) merged 2026-05-12 |
| [feat_study_lifecycle](implemented_features/2026_05_10_feat_study_lifecycle/feature_spec.md) | Feature | A relevance engineer creates a study via API or chat, the orchestrator enqueues N parallel `run_trial` jobs, trials accumulate in real time on the study detail page, the orchestrator detects stop-cond | — | [PR #18](https://github.com/SoundMindsAI/relyloop/pull/18) merged 2026-05-10 |
| [feat_study_preflight_overlap_probe](implemented_features/2026_05_22_feat_study_preflight_overlap_probe/feature_spec.md) | Feature | `POST /api/v1/studies` issues a single bounded `ids`-existence query against the study's target asking "for the *first* query in the query set that has any judgments (chosen deterministically by `id A | — | [PR #193](https://github.com/SoundMindsAI/relyloop/pull/193) merged 2026-05-21 |
| [feat_study_target_judgment_mismatch_guard](implemented_features/2026_05_21_feat_study_target_judgment_mismatch_guard/feature_spec.md) | Feature | `POST /api/v1/studies` rejects the mismatch at create time with a specific machine-readable error code (`JUDGMENT_TARGET_MISMATCH`, 422). | — | [PR #184](https://github.com/SoundMindsAI/relyloop/pull/184) merged 2026-05-21 |
| [infra_adapter_elastic](implemented_features/2026_05_10_infra_adapter_elastic/feature_spec.md) | Infra | A single `ElasticAdapter` implements the `SearchAdapter` Protocol and serves both Elasticsearch (8.11+ / 9.x) and OpenSearch (2.x / 3.x), distinguished by a `engine_type` column. | — | [PR #16](https://github.com/SoundMindsAI/relyloop/pull/16) merged 2026-05-10 |
| [infra_ci_smoke_makeup](implemented_features/2026_05_13_infra_ci_smoke_makeup/idea.md) | Infra | Complete | — | Complete |
Expand Down Expand Up @@ -115,11 +110,9 @@ Plan approved; run /impl-execute to ship

_None._

### Plan (1)
### Plan (0)

| Priority | Feature | Type | One-liner | Depends on | Status |
|---|---|---|---|---|---|
| P1 | [feat_study_preflight_overlap_probe](../02_product/planned_features/feat_study_preflight_overlap_probe/feature_spec.md) | Feature | `POST /api/v1/studies` issues a single bounded `ids`-existence query against the study's target asking "for the *first* query in the query set that has any judgments (chosen deterministically by `id A | — | [PR #184](https://github.com/SoundMindsAI/relyloop/pull/184) merged 2026-05-21 |
_None._

### Spec (0)

Expand Down Expand Up @@ -158,8 +151,6 @@ graph LR
classDef plan fill:#fef9c3,stroke:#854d0e,color:#854d0e;
classDef spec fill:#dbeafe,stroke:#1e40af,color:#1e40af;
classDef idea fill:#f1f5f9,stroke:#334155,color:#334155;
feat_study_preflight_overlap_probe["study preflight overlap probe"]
class feat_study_preflight_overlap_probe plan;
infra_foundation["foundation"]
class infra_foundation done;
feat_study_lifecycle["study lifecycle"]
Expand Down Expand Up @@ -282,6 +273,8 @@ graph LR
class feat_home_first_run_demo_nudge done;
feat_orchestrator_zero_streak_abort["orchestrator zero streak abort"]
class feat_orchestrator_zero_streak_abort done;
feat_study_preflight_overlap_probe["study preflight overlap probe"]
class feat_study_preflight_overlap_probe done;
feat_study_lifecycle --> feat_digest_proposal
feat_llm_judgments --> feat_digest_proposal
infra_foundation --> feat_llm_judgments
Expand Down
2 changes: 1 addition & 1 deletion docs/00_overview/dashboard.html
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,7 @@ <h2>Releases</h2>
<div class="roadmap-row">
<div class="release-name"><a href="mvp1_dashboard.html">MVP1 / v0.1</a></div>
<div class="theme">The Loop</div>
<div class="progress">61 / 62 scoped done · 10 remaining</div>
<div class="progress">62 / 62 scoped done · 9 remaining</div>
<span class="state-pill in_progress">In progress</span>
</div>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Implementation Plan — feat_study_preflight_overlap_probe

**Date:** 2026-05-22
**Status:** Draft
**Status:** Complete (PR #193, squash-merged `ca835e0` on 2026-05-22)
**Primary spec:** [feature_spec.md](feature_spec.md)
**Policy source(s):** [api-conventions.md](../../../01_architecture/api-conventions.md), [CLAUDE.md](../../../../CLAUDE.md), [adapters.md](../../../01_architecture/adapters.md)

Expand Down Expand Up @@ -715,17 +715,20 @@ None. The feature is purely additive — 1 new service module + 2 new repo funct

### Current sprint

- [ ] Story 1.1 — Repo functions
- [ ] Story 1.2 — Service helper
- [ ] Story 1.3 — Handler integration + api-conventions.md row + contract tests
- [x] Story 1.1 — Repo functions
- [x] Story 1.2 — Service helper
- [x] Story 1.3 — Handler integration + api-conventions.md row + contract tests

### Blocked items

- None.

### Done this sprint

- (none yet)
- [x] Story 1.1 (commit `2d1727e`) — `find_first_judged_query` + `list_doc_ids_for_list_and_query` + `__init__.py` exports
- [x] Story 1.2 (commit `2d1727e`) — `backend/app/services/study_preflight.py` (`OverlapProbeResult` + `probe_judgment_overlap` + module-level constants)
- [x] Story 1.3 (commit `2d1727e`) — `studies.py` handler integration + `api-conventions.md` row + `study-lifecycle-debugging.md` paragraph + source-presence contract test
- [x] Tests (commits `2d1727e`, `0602927`) — 4 unit + 14 integration test functions (18 parametrized cases) + 2 contract

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,16 @@
- Test counts: 4 unit + 14 integration test functions (18 parametrized cases — AC-13 contributes 5) + 2 contract

## Implementation
- Status: Not started
- Status: Complete
- Date: 2026-05-22
- PR: #193 (squash-merged as `ca835e0` on 2026-05-22)
- CI: 5/5 green on the final push
- Stories: 3 stories in 1 epic, all `[x]`
- Tests: 4 unit + 14 integration test functions (18 parametrized cases) + 2 contract + 1 source-presence ordering lock
- Cross-model reviews: phase-gate GPT-5.5 (5 findings, 2 applied + 2 deferred-as-ideas + 1 rejected), Gemini Code Assist (1 finding rejected with cited counter-evidence), final GPT-5.5 (2 findings — 1 rejected + 1 accepted-as-documented)
- Follow-up ideas captured: `infra_study_preflight_real_engine_integration`, `chore_studies_post_arq_spy_fixture`, `bug_dashboard_banner_dismiss_persistence_flake`

## Done
- Status: Deployed to operator stack (MVP1 — no remote staging)
- Date: 2026-05-22
- Alembic head unchanged at `0015_trials_per_query_metrics` (feature is purely additive at the application layer)
63 changes: 32 additions & 31 deletions docs/00_overview/mvp1_dashboard.html
Original file line number Diff line number Diff line change
Expand Up @@ -382,28 +382,28 @@ <h1>RelyLoop MVP1 Dashboard</h1>
<main>

<section>
<div class="next-up">
<div class="eyebrow">Next up — Feature, currently in <strong>Plan</strong></div>
<div class="title"><a href="../../docs/02_product/planned_features/feat_study_preflight_overlap_probe/feature_spec.md">Study Preflight Overlap Probe</a></div>
<div class="one-liner">`POST /api/v1/studies` issues a single bounded `ids`-existence query against the study&#x27;s target asking &quot;for the *first* query in the query set that has any judgments (chosen deterministically by `id ASC`), do its judged doc IDs still exist </div>
<div class="stage-hint">Plan approved; run /impl-execute to ship</div>
<code class="cmd">/impl-execute docs/02_product/planned_features/feat_study_preflight_overlap_probe/implementation_plan.md --all</code>
<div class="next-up done">
<div class="eyebrow">Next up</div>
<div class="title">All scoped MVP1 features shipped 🎉</div>
<div class="one-liner">
Pull from the Idea backlog or capture a new feature spec.
</div>
</div>
</section>


<section>
<h2>MVP1 Progress</h2>
<div class="kpi-row">
<div class="kpi ">
<div class="kpi complete">
<div class="label">Scoped items done</div>
<div class="value">61 / 62</div>
<div class="sub">98% of feat_/infra_/chore_/epic_ items past idea stage</div>
<div class="bar"><span style="width:98%"></span></div>
<div class="value">62 / 62</div>
<div class="sub">100% of feat_/infra_/chore_/epic_ items past idea stage</div>
<div class="bar"><span style="width:100%"></span></div>
</div>
<div class="kpi warn">
<div class="label">Pending work</div>
<div class="value">18</div>
<div class="value">17</div>
<div class="sub">every not-done feat/infra/chore/bug across all priorities</div>
</div>
<div class="kpi bug">
Expand All @@ -420,7 +420,7 @@ <h2>MVP1 Progress</h2>
<div class="kpi-row">
<div class="kpi">
<div class="label">P1</div>
<div class="value">4</div>
<div class="value">3</div>
<div class="sub">high-value, ready when P0 clears</div>
</div>
<div class="kpi">
Expand All @@ -435,7 +435,7 @@ <h2>MVP1 Progress</h2>
</div>
<div class="kpi">
<div class="label">Legacy "Path to MVP1"</div>
<div class="value">10</div>
<div class="value">9</div>
<div class="sub">scoped not-done + bugs + chore-ideas only (excludes feat/infra ideas)</div>
</div>
</div>
Expand Down Expand Up @@ -693,19 +693,7 @@ <h3>Spec <span class="count">0</span></h3>
</div>

<div class="col plan">
<h3>Plan <span class="count">1</span></h3>

<div class="card feat" data-prefix="feat" data-priority="P1">
<div class="name"><a href="../../docs/02_product/planned_features/feat_study_preflight_overlap_probe/feature_spec.md">Study Preflight Overlap Probe</a></div>
<div class="meta">
<span class="badge feat">Feature</span>
<span class="badge priority" data-priority="P1">P1</span>
<a class="pr" href="https://github.com/SoundMindsAI/relyloop/pull/184">PR #184</a><span>merged 2026-05-21</span>
</div>
<div class="one-liner">`POST /api/v1/studies` issues a single bounded `ids`-existence query against the study&#x27;s target asking &quot;for the *first* query in the query set that has any judgments (chosen deterministically by `id A</div>


</div>
<h3>Plan <span class="count">0</span></h3>

</div>

Expand All @@ -715,7 +703,7 @@ <h3>Implementing <span class="count">0</span></h3>
</div>

<div class="col done">
<h3>Done <span class="count">73</span></h3>
<h3>Done <span class="count">74</span></h3>

<div class="card feat" data-prefix="feat" data-priority="P2">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_21_feat_agent_propose_search_space/feature_spec.md">Agent Propose Search Space</a></div>
Expand Down Expand Up @@ -964,6 +952,19 @@ <h3>Done <span class="count">73</span></h3>
</div>


<div class="card feat" data-prefix="feat" data-priority="P2">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_22_feat_study_preflight_overlap_probe/feature_spec.md">Study Preflight Overlap Probe</a></div>
<div class="meta">
<span class="badge feat">Feature</span>

<a class="pr" href="https://github.com/SoundMindsAI/relyloop/pull/193">PR #193</a><span>merged 2026-05-21</span>
</div>
<div class="one-liner">`POST /api/v1/studies` issues a single bounded `ids`-existence query against the study&#x27;s target asking &quot;for the *first* query in the query set that has any judgments (chosen deterministically by `id A</div>


</div>


<div class="card feat" data-prefix="feat" data-priority="P2">
<div class="name"><a href="../../docs/00_overview/implemented_features/2026_05_21_feat_study_target_judgment_mismatch_guard/feature_spec.md">Study Target Judgment Mismatch Guard</a></div>
<div class="meta">
Expand Down Expand Up @@ -1678,8 +1679,6 @@ <h2>Dependency graph (feat_ + infra_)</h2>
classDef plan fill:#fef9c3,stroke:#854d0e,color:#854d0e;
classDef spec fill:#dbeafe,stroke:#1e40af,color:#1e40af;
classDef idea fill:#f1f5f9,stroke:#334155,color:#334155;
feat_study_preflight_overlap_probe[&quot;study preflight overlap probe&quot;]
class feat_study_preflight_overlap_probe plan;
infra_foundation[&quot;foundation&quot;]
class infra_foundation done;
feat_study_lifecycle[&quot;study lifecycle&quot;]
Expand Down Expand Up @@ -1802,6 +1801,8 @@ <h2>Dependency graph (feat_ + infra_)</h2>
class feat_home_first_run_demo_nudge done;
feat_orchestrator_zero_streak_abort[&quot;orchestrator zero streak abort&quot;]
class feat_orchestrator_zero_streak_abort done;
feat_study_preflight_overlap_probe[&quot;study preflight overlap probe&quot;]
class feat_study_preflight_overlap_probe done;
feat_study_lifecycle --&gt; feat_digest_proposal
feat_llm_judgments --&gt; feat_digest_proposal
infra_foundation --&gt; feat_llm_judgments
Expand Down Expand Up @@ -1901,8 +1902,6 @@ <h2>Dependency graph (feat_ + infra_)</h2>
classDef plan fill:#fef9c3,stroke:#854d0e,color:#854d0e;
classDef spec fill:#dbeafe,stroke:#1e40af,color:#1e40af;
classDef idea fill:#f1f5f9,stroke:#334155,color:#334155;
feat_study_preflight_overlap_probe[&quot;study preflight overlap probe&quot;]
class feat_study_preflight_overlap_probe plan;
infra_foundation[&quot;foundation&quot;]
class infra_foundation done;
feat_study_lifecycle[&quot;study lifecycle&quot;]
Expand Down Expand Up @@ -2025,6 +2024,8 @@ <h2>Dependency graph (feat_ + infra_)</h2>
class feat_home_first_run_demo_nudge done;
feat_orchestrator_zero_streak_abort[&quot;orchestrator zero streak abort&quot;]
class feat_orchestrator_zero_streak_abort done;
feat_study_preflight_overlap_probe[&quot;study preflight overlap probe&quot;]
class feat_study_preflight_overlap_probe done;
feat_study_lifecycle --&gt; feat_digest_proposal
feat_llm_judgments --&gt; feat_digest_proposal
infra_foundation --&gt; feat_llm_judgments
Expand Down
6 changes: 3 additions & 3 deletions state.md

Large diffs are not rendered by default.