Commit 933a86f
test(core): add the 1.U end-to-end harness suite — the M2 proof
1.U is the proof the engine works end-to-end before any surface exists. It composes
the already-landed pieces behind the @relavium/llm seam (1.K FallbackChain, 1.N run
loop + RunEventBus, 1.O AgentRunner, 1.P node handlers, 1.Q human gate, 1.R
checkpoint/resume, 1.S node retry, 1.T ToolRegistry) on the in-memory ExecutionHost
reference — zero platform imports, no live network/keys, deterministic. No engine
code and no new ADR: the harness uses only already-exported @relavium/core symbols.
A scenario suite (the seed the Phase-2 CLI regression harness 2.K grows from):
- happy-path — the literal 3-node input -> agent(+tool call) -> output: live token
streaming, per-attempt cost (attributed to the model), a gap-free sequenceNumber
stream that validates against the canonical RunEventSchema.
- flagship — input -> agent -> human_gate -> output: in ONE run across a process
boundary, the agent's forced provider error -> node retry (ADR-0040) -> failover to
the second chain entry (1.K), per-attempt cost attributed to the fallback model;
then a pause at the gate (the durable mid-run checkpoint persisted to the
SQLite-shaped store); then a FRESH engine resumes via resumeFromCheckpoint, runs
output to run:completed, and reproduces the final output with sequenceNumber
continuing gap-free. The agent is NOT re-run on resume (its output is restored).
- determinism — a re-run yields an identical event signature + final output (the
no-wall-clock/no-RNG ban the risk table binds to the harness).
Design decision (maintainer-approved): the resume demonstration uses a human gate as
the durable suspend point, because the Phase-1 engine resumes ONLY from a gate/budget
pause — a gate-less interrupted run is reconciled to run:failed (ADR-0036; verified by
the resume-model review + an adversarial refutation). The literal 3-node topology is
preserved by the happy-path member; the flagship adds the gate. Recorded as the §1.U
"Harness shape" clarification in the phase plan (no new ADR — composes decided
mechanisms). Exit criterion #1 annotated to point resume at the gated flagship.
Per-attempt cost is asserted as distinct cost:updated events (not only the terminal
cumulative); all LLM cost is incurred pre-gate, so the plain-human-gate cost-restore
deferral (deferred-tasks.md) is off this path.
1.U / M2 will be marked Done only after this PR merges (roadmap-done-after-merge).
Full gate (format/lint/typecheck/test/build) green — core 683 tests; Leakwatch clean.
Refs: §1.U, M2, ADR-0036, ADR-0040, ADR-0003
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent 8087d85 commit 933a86f
2 files changed
Lines changed: 425 additions & 1 deletion
File tree
- docs/roadmap/phases
- packages/core/src/engine
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
842 | 842 | | |
843 | 843 | | |
844 | 844 | | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
845 | 865 | | |
846 | 866 | | |
847 | 867 | | |
| |||
1212 | 1232 | | |
1213 | 1233 | | |
1214 | 1234 | | |
1215 | | - | |
| 1235 | + | |
| 1236 | + | |
1216 | 1237 | | |
1217 | 1238 | | |
1218 | 1239 | | |
| |||
0 commit comments