diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5b7938e5..20362b7a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -104,6 +104,12 @@ jobs: - name: Seam fence is enforced run: pnpm lint:fence-check + # The size half of the fence: engine packages' runtime dependency graphs must stay + # within their reviewed allowlists (a new engine dep = an ADR + a deliberate edit to + # tools/engine-deps/check.mjs in the same change). + - name: Engine dependency allowlist + run: pnpm lint:engine-deps + # CI-only strict peer-dependency gate (.npmrc keeps this OFF for local installs so fresh # checkouts resolve while the surface packages' peers are not all in the tree yet). Catch # peer drift here without breaking local dev. diff --git a/.npmrc b/.npmrc index 391d289e..aff5d877 100644 --- a/.npmrc +++ b/.npmrc @@ -14,3 +14,10 @@ strict-peer-dependencies=false # of letting a wrong runtime surface as confusing errors later. The pinned versions live in # `.nvmrc` (Node 22) and `package.json` `engines` (Node >=20.11.0, pnpm >=9). engine-strict=true + +# Supply-chain: install scripts run ONLY for packages on the explicit allowlist in the root +# `package.json` `pnpm.onlyBuiltDependencies` (better-sqlite3 builds/fetches its native binding; +# esbuild fetches its platform binary). Every other package's install/postinstall script is +# skipped, so a compromised transitive dependency cannot execute code at install time. Growing +# that allowlist is a deliberate, reviewed change — the install-time twin of "no new runtime +# dependency without an ADR" (architectural-principles.md §9). diff --git a/CLAUDE.md b/CLAUDE.md index db95ace9..40b83ada 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -43,7 +43,10 @@ Phase 0 (M0, 2026-06-04) landed the monorepo, strict toolchain + CI, `@relavium/ full Zod contract set), the no-vendor-type seam fence, and `@relavium/db`. Phase 1 has since landed `@relavium/llm` — the `LLMProvider` seam + all three adapters (Anthropic, OpenAI/DeepSeek, Gemini), green on one shared conformance suite with no vendor type crossing the seam, plus the -ADR-0030 seam-shape amendment. Active work continues on the `FallbackChain` (1.K) and the +ADR-0030 seam-shape amendment — and the **ADR-0031 multimodal seam-shape amendment (1.AD, PR #11, +2026-06-10)**: the media content/stream union members, the per-modality capability matrix, and the +reserved generator methods, shape-only, landed before the seam's exhaustive consumers. Active work +continues on the `FallbackChain` (1.K) and the [`@relavium/core` engine](docs/roadmap/phases/phase-1-engine-and-llm.md); see [docs/roadmap/current.md](docs/roadmap/current.md). See [README.md](README.md) for the public overview. diff --git a/README.md b/README.md index 1767ecb9..a7098c13 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,10 @@ set), the no-vendor-type seam fence, and `@relavium/db`. Phase 1 then landed [`@relavium/llm`](docs/roadmap/phases/phase-1-engine-and-llm.md): the provider-agnostic `LLMProvider` seam and **all three adapters** (Anthropic; the OpenAI-compatible adapter serving OpenAI + DeepSeek; Gemini), passing one shared conformance suite behind the frozen seam with no -vendor type crossing it (PR #7–#9). Next on the critical path: the `FallbackChain` runner (1.K) +vendor type crossing it (PR #7–#9), followed by the **ADR-0031 multimodal seam-shape amendment +(1.AD)** — the media content/stream union members, the per-modality capability matrix, and the +reserved generator methods, landed **shape-only** before the seam's exhaustive consumers exist +(PR #11, 2026-06-10). Next on the critical path: the `FallbackChain` runner (1.K) and the `@relavium/core` engine — parser, run loop, checkpoint/resume, retry, plus the **AgentSession** runtime + export-to-workflow sub-spine. See [docs/roadmap/current.md](docs/roadmap/current.md) for live status. diff --git a/docs/analysis/README.md b/docs/analysis/README.md index 3647916b..d7931154 100644 --- a/docs/analysis/README.md +++ b/docs/analysis/README.md @@ -16,14 +16,12 @@ It has two parts: | Document | What it is | |----------|-----------| -| [competitive-landscape-2026-06-05.md](competitive-landscape-2026-06-05.md) | Competitive analysis carrying the two-entry-point thesis (agent chat + workflow): four surfaces, one runtime, two entry points, and the chat → workflow continuum. Supersedes the 2026-06-03 read after the agent-first pivot. | -| [competitive-landscape-2026-06-03.md](competitive-landscape-2026-06-03.md) | Competitive analysis vs Claude Code, Cursor, Continue.dev, Cline/Roo-code, GitHub Copilot Agent Mode, n8n/Zapier, and CrewAI/AutoGen, with where Relavium wins. | | [managed-inference-business-model-2026-06-03.md](managed-inference-business-model-2026-06-03.md) | Decision analysis: BYOK vs managed inference (Relavium's own keys, metered by license). ToS/legality, competitive pricing, unit economics, gateway architecture, compliance. Led to the dual-mode decision ([ADR-0012](../decisions/0012-managed-inference-dual-mode.md)). | | [_archive/README.md](_archive/README.md) | Provenance map: which living docs were seeded by which frozen raw-analysis section. The archive itself is **frozen — never edited**. | ## Conventions -- **Dated research** uses an ISO-date suffix: `competitive-landscape-2026-06-03.md`. +- **Dated research** uses an ISO-date suffix: `managed-inference-business-model-2026-06-03.md`. When the analysis is redone, a new dated file is added rather than overwriting the old one — the dated trail is the point. - **Analysis is not a spec.** Concrete specs (schemas, contracts, DDL, node types) live diff --git a/docs/analysis/_archive/README.md b/docs/analysis/_archive/README.md index d614107b..f04d2253 100644 --- a/docs/analysis/_archive/README.md +++ b/docs/analysis/_archive/README.md @@ -73,7 +73,7 @@ This is the current-vision pivot and the **primary** source for most of the tree | `vscode` (7 capabilities, Model C hybrid connection, LSP design, extension API) | [reference/vscode/extension-api.md](../../reference/vscode/extension-api.md), [tutorials/vscode/trigger-from-vscode.md](../../tutorials/vscode/trigger-from-vscode.md) | | `cli` (null in the raw JSON — CLI facts reconstructed from `synthesis`) | [reference/cli/commands.md](../../reference/cli/commands.md), [tutorials/cli/run-a-workflow-in-ci.md](../../tutorials/cli/run-a-workflow-in-ci.md) | | `corePortal` (Workflow YAML Spec v1.0, complete example, engine API, plugin API, MCP, local-vs-cloud mode, portal pages) | [reference/contracts/workflow-yaml-spec.md](../../reference/contracts/workflow-yaml-spec.md), [reference/contracts/agent-yaml-spec.md](../../reference/contracts/agent-yaml-spec.md), [architecture/shared-core-engine.md](../../architecture/shared-core-engine.md), [reference/portal/api-reference.md](../../reference/portal/api-reference.md), [architecture/cloud-phase-2.md](../../architecture/cloud-phase-2.md) | -| `strategy` (7-competitor analysis, UVP, go-to-market, workflow-as-code, pricing) | [analysis/competitive-landscape-2026-06-03.md](../competitive-landscape-2026-06-03.md), [uvp.md](../../uvp.md), [vision.md](../../vision.md) | +| `strategy` (7-competitor analysis, UVP, go-to-market, workflow-as-code, pricing) | `competitive-landscape-2026-06-03.md` *(removed 2026-06-10; surviving framing: uvp.md)*, [uvp.md](../../uvp.md), [vision.md](../../vision.md) | | `synthesis` (product identity, locked tech choices, monorepo, phasing, day-one DX, killer features, build order, top risks) | [vision.md](../../vision.md), [tech-stack.md](../../tech-stack.md), [project-structure.md](../../project-structure.md), [roadmap/README.md](../../roadmap/README.md), [architecture/overview.md](../../architecture/overview.md), the [tutorials/](../../tutorials/README.md) | ## Rules diff --git a/docs/analysis/competitive-landscape-2026-06-03.md b/docs/analysis/competitive-landscape-2026-06-03.md deleted file mode 100644 index 2e7188c0..00000000 --- a/docs/analysis/competitive-landscape-2026-06-03.md +++ /dev/null @@ -1,183 +0,0 @@ -# Competitive Landscape - -- **Status**: Analysis -- **Date**: 2026-06-03 -- **Related**: [uvp.md](../uvp.md), [vision.md](../vision.md), [product-constraints.md](../product-constraints.md) - -This is a point-in-time competitive analysis of the AI-coding / agent-workflow space as -of 2026-06-03. It maps where Relavium sits against the tools developers already use, and -where Relavium's structural differences create an advantage. It is **analysis, not a -spec** — for the canonical value proposition see [uvp.md](../uvp.md); for the product -scope it assumes see [product-constraints.md](../product-constraints.md). - -> This document is dated. When the analysis is redone, add a new -> `competitive-landscape-YYYY-MM-DD.md` rather than overwriting this one — the dated -> trail is the record of how the market moved. - -## The structural thesis - -Every competitor owns **at most two surfaces** and treats AI interaction as a *session* -or a *completion*. Relavium's two structural bets are different: - -1. **Four surfaces, one runtime.** Desktop canvas, VS Code extension, CLI, and - (Phase 2) web portal all run the *same* `@relavium/core` engine, so a workflow - behaves identically whether triggered from an editor, a terminal, the canvas, or CI. -2. **The workflow is a git-committable file**, not a session or a vendor database row. - A `.relavium.yaml` is reviewable, diffable, revertable, and shareable through the same - git workflow as code. - -These two bets are the lens for the comparison below. - -## Where Relavium fits - -```mermaid -quadrantChart - title Surface reach vs. workflow persistence - x-axis "Single surface" --> "Multi-surface" - y-axis "Ephemeral session" --> "Git-native workflow file" - quadrant-1 "Multi-surface + persistent" - quadrant-2 "Single-surface + persistent" - quadrant-3 "Single-surface + ephemeral" - quadrant-4 "Multi-surface + ephemeral" - "Relavium": [0.85, 0.9] - "Claude Code": [0.55, 0.2] - "Cursor": [0.2, 0.15] - "Continue.dev": [0.4, 0.25] - "Cline/Roo": [0.25, 0.2] - "Copilot Agent": [0.45, 0.2] - "n8n/Zapier": [0.5, 0.6] - "CrewAI/AutoGen": [0.2, 0.55] -``` - -## Competitor-by-competitor - -### Claude Code (Anthropic) - -- **Strengths:** deep Claude integration with strong reasoning; excellent native CLI and - terminal UX; tight VS Code editor integration; agentic file editing with bash - execution; Anthropic's trust and safety credibility. -- **Gaps:** single-model only (no model switching or multi-model orchestration); no - visual workflow canvas; no workflow persistence or version control; one agent per - session, no multi-agent coordination; no team sharing or org governance; no CI/CD - integration path; CLI and extension feel like different products. -- **Relavium's advantage:** Relavium does what Claude Code does *across surfaces*, then - adds multi-model routing, multi-agent orchestration, visual design, and - version-controllable workflows. Claude Code power users graduate to Relavium when they - outgrow single-agent, single-model work. - -### Cursor - -- **Strengths:** best-in-class AI-native editor; fast inline completions and multi-file - edits; agent mode with strong codebase awareness; large loyal community; predictable - pricing. -- **Gaps:** editor-only (no CLI, no canvas, no portal); no visual workflow builder; - every session starts from scratch — no reuse; no multi-agent coordination; locked into - the Cursor fork (no standard VS Code or JetBrains); no CI/CD parity; no team workflow - library or governance. -- **Relavium's advantage:** Cursor wins on editor polish but creates vendor lock-in and - has no workflow abstraction layer. Relavium is an orchestration layer *above* any - editor — keep your preferred tools and gain reusable, shareable, versioned workflows. - -### Continue.dev - -- **Strengths:** fully open-source with strong community trust; genuine multi-model - support (including local models); VS Code and JetBrains; highly configurable via - `config.json`; bring-your-own-key, no lock-in. -- **Gaps:** no visual canvas or designer; single-agent sessions only; no - version-controllable workflow files; no CLI for CI/CD; no desktop app; no team/ - enterprise portal; community-paced enterprise roadmap. -- **Relavium's advantage:** Relavium shares the multi-model, bring-your-own-key - philosophy but adds the orchestration and workflow-abstraction layer above chat/ - autocomplete. Continue.dev users who want to *chain* agents and *persist* workflows are - a natural upgrade path. - -### Cline / Roo-code - -- **Strengths:** highly capable agentic coding with browser and terminal access; - open-source and extensible; strong power-user community; model-agnostic; aggressive - agentic capabilities (filesystem, shell, browser). -- **Gaps:** purely conversational agent paradigm, no canvas; no workflow version control - or reuse; single agent per session; no team collaboration; no CLI for CI/CD; no desktop - or portal; high token cost per session with no optimization layer; no governance. -- **Relavium's advantage:** Cline is the power user's agentic workhorse, but every run is - ephemeral and manual. Relavium adds the orchestration layer that makes Cline-style - execution reusable, composable, team-shareable, visually designed, and version- - controlled. - -### GitHub Copilot Agent Mode - -- **Strengths:** massive distribution through GitHub; deep GitHub integration (PRs, - issues, Actions, Codespaces); Microsoft enterprise trust; familiar UX for millions; - repo-wide context awareness. -- **Gaps:** GitHub/Microsoft ecosystem lock-in; no visual canvas or designer; no - multi-model routing (Microsoft/OpenAI models only); no local-first execution (runs go - through Microsoft infra); workflows are not portable, version-controllable artifacts; - VS Code only; limited multi-agent coordination; weak for teams not on GitHub. -- **Relavium's advantage:** Copilot Agent Mode is powerful *inside the GitHub bubble* but - deeply locked to it. Relavium is VCS-agnostic, model-agnostic, and surface-agnostic — - the same workflow runs from VS Code, CLI, canvas, or CI regardless of git host. - -### n8n / Zapier - -- **Strengths:** mature visual workflow builders with large node libraries; strong - non-technical adoption; n8n is self-hostable and open-core; hundreds of SaaS - connectors; proven enterprise business model. -- **Gaps:** not AI-native (LLM nodes are bolted on, not first-class); no developer-native - surfaces (no VS Code, no dev CLI); cannot natively run code agents or shell commands; - no local-first model (cloud or self-hosted server); no understanding of code, repos, or - dev workflows; no multi-agent AI primitives; no IDE integration. -- **Relavium's advantage:** n8n and Zapier own *general* automation but have no - credibility in the developer AI-agent space. Relavium is the developer-native, AI-first - version — with code context, IDE integration, and agent orchestration they can't - replicate. - -### CrewAI / AutoGen - -- **Strengths:** powerful multi-agent orchestration frameworks; flexible, programmable - agent topologies; active open-source communities; rich agent-to-agent communication; - research and enterprise adoption for complex pipelines. -- **Gaps:** Python framework only — no IDE integration, no visual canvas; no VS Code - extension or CLI for non-Python users; no desktop app or designer; requires Python - expertise; no human-readable, version-controllable workflow format; no native - local-first execution on dev machines; no portal for governance; high barrier to entry. -- **Relavium's advantage:** CrewAI and AutoGen prove multi-agent orchestration is - valuable but demand Python expertise and produce no visual artifacts. Relavium delivers - the same multi-agent power through a visual canvas and YAML files any developer can - read, edit, and share — no framework expertise required. (Relavium reuses the - *concepts* from this space, not the Python idiom; see - [ADR-0003](../decisions/0003-pure-ts-engine-not-langgraph-python.md).) - -## Summary matrix - -| Capability | Claude Code | Cursor | Continue | Cline/Roo | Copilot Agent | n8n/Zapier | CrewAI/AutoGen | **Relavium** | -|---|---|---|---|---|---|---|---|---| -| Multi-model routing | ✗ | ✗ | ✓ | ✓ | ✗ | partial | ✓ | **✓** | -| Multi-agent orchestration | ✗ | ✗ | ✗ | ✗ | partial | ✗ | ✓ | **✓** | -| Visual workflow canvas | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | **✓** | -| Git-native workflow files | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ (JSON blobs) | partial (code) | **✓** | -| Local-first execution | partial | partial | ✓ | ✓ | ✗ | ✗ | partial | **✓** | -| CLI for CI/CD | ✓ | ✗ | ✗ | ✗ | ✗ | partial | partial | **✓** | -| VS Code extension | ✓ | n/a (fork) | ✓ | ✓ | ✓ | ✗ | ✗ | **✓** | -| Desktop app | ✗ | ✓ (editor) | ✗ | ✗ | ✗ | ✗ (web) | ✗ | **✓** | -| No vendor lock-in | partial | ✗ | ✓ | ✓ | ✗ | partial | ✓ | **✓** | - -## Strategic takeaways - -- **The graduation path is the wedge.** The primary user is the senior developer or tech - lead who already uses Claude Code, Cline, or Cursor and has hit the ceiling of - single-agent, single-surface work. Relavium is where they go next, not a replacement - for the editor they love. -- **The workflow file is the viral mechanic.** A `.relavium.yaml` committed to a team - repo is the invite: every teammate who pulls the repo encounters it, installs the - extension to run it, and becomes a user. Value compounds with team size in a way no - session-based tool can match. -- **Local-first is a Phase-1 trust moat.** Until a competitor offers visual design + - local execution + multi-model routing + multi-agent orchestration in one product with - zero data leaving the machine, Relavium owns the security-conscious developer segment. -- **Phase-2 cloud is a business-model unlock, not a feature gap.** Cloud execution, - scheduling, and team governance are deliberately deferred (see - [product-constraints.md](../product-constraints.md)); they are gated on real Phase-1 - demand, not built speculatively. - -> Source: the 7-competitor analysis in the frozen `synthesis-raw.json` (`strategy` -> section). See the [archive provenance map](_archive/README.md). diff --git a/docs/analysis/competitive-landscape-2026-06-05.md b/docs/analysis/competitive-landscape-2026-06-05.md deleted file mode 100644 index ab1480b0..00000000 --- a/docs/analysis/competitive-landscape-2026-06-05.md +++ /dev/null @@ -1,239 +0,0 @@ -# Competitive Landscape - -- **Status**: Analysis -- **Date**: 2026-06-05 -- **Related**: [uvp.md](../uvp.md), [vision.md](../vision.md), [product-constraints.md](../product-constraints.md) - -This is a point-in-time competitive analysis of the AI-coding / agent-workflow space as -of 2026-06-05. It maps where Relavium sits against the tools developers already use, and -where Relavium's structural differences create an advantage. It is **analysis, not a -spec** — for the canonical value proposition see [uvp.md](../uvp.md); for the product -scope it assumes see [product-constraints.md](../product-constraints.md). - -It supersedes the [2026-06-03 analysis](competitive-landscape-2026-06-03.md), which -predates the agent-first pivot ([ADR-0024](../decisions/0024-agent-first-entry-point-agentsession.md), -[ADR-0026](../decisions/0026-session-export-to-workflow.md)). The earlier file is frozen -as the record of the pre-pivot market read; this one carries the **two-entry-point** -thesis. - -> This document is dated. When the analysis is redone, add a new -> `competitive-landscape-YYYY-MM-DD.md` rather than overwriting this one — the dated -> trail is the record of how the market moved. - -## The structural thesis - -Every competitor owns **at most two surfaces** and treats AI interaction as a *session* -or a *completion*. Relavium's structural bets are different: - -1. **Four surfaces, one runtime.** Desktop canvas, VS Code extension, CLI, and - (Phase 2) web portal all run the *same* `@relavium/core` engine, so behavior is - identical whether triggered from an editor, a terminal, the canvas, or CI. -2. **Two entry points, one engine.** The same runtime is reachable two ways: an - **agent chat** (a multi-turn conversation — the `AgentSession` entry point) *and* a - **workflow** (a git-committable `.relavium.yaml` DAG). Both ride one substrate - (`ToolRegistry`, the `@relavium/llm` seam, the event bus); see - [ADR-0024](../decisions/0024-agent-first-entry-point-agentsession.md). -3. **The workflow is a git-committable file**, not a session or a vendor database row. - A `.relavium.yaml` is reviewable, diffable, reversible, and shareable through the same - git workflow as code. -4. **A one-way chat → workflow continuum.** A chat session is auto-persisted and - resumable, and can be **exported to a workflow scaffold** that is reviewed before - commit — so Relavium sits at *both* the start (conversation) and the destination - (committed workflow). See - [ADR-0026](../decisions/0026-session-export-to-workflow.md) and the contract in - [agent-session-spec.md](../reference/contracts/agent-session-spec.md). - -These bets are the lens for the comparison below. - -## Where Relavium fits - -```mermaid -quadrantChart - title Surface reach vs. workflow persistence - x-axis "Single surface" --> "Multi-surface" - y-axis "Ephemeral session" --> "Git-native workflow file" - quadrant-1 "Multi-surface + persistent" - quadrant-2 "Single-surface + persistent" - quadrant-3 "Single-surface + ephemeral" - quadrant-4 "Multi-surface + ephemeral" - "Relavium": [0.85, 0.9] - "Claude Code": [0.55, 0.2] - "Cursor": [0.2, 0.15] - "Continue.dev": [0.4, 0.25] - "Cline/Roo": [0.25, 0.2] - "Copilot Agent": [0.45, 0.2] - "n8n/Zapier": [0.5, 0.6] - "CrewAI/AutoGen": [0.2, 0.55] -``` - -Relavium still sits in the top-right (multi-surface + git-native), but the agent-first -pivot moves it *along* the path most competitors live on: it now meets users where they -already are — a conversation — and then carries them up to the persistent, committable -artifact. The competitors clustered low-left have no path *off* the ephemeral session; -Relavium's chat is the same low-left starting point with a route out. - -## Competitor-by-competitor - -### Claude Code (Anthropic) - -- **Strengths:** deep Claude integration with strong reasoning; excellent native CLI and - terminal UX; tight VS Code editor integration; agentic file editing with bash - execution; Anthropic's trust and safety credibility. -- **Gaps:** single-model only (no model switching or multi-model orchestration); no - visual workflow canvas; no workflow persistence or version control; one agent per - session, no multi-agent coordination; sessions are ephemeral with no resumable, - committable artifact; no team sharing or org governance; no CI/CD integration path; - CLI and extension feel like different products. -- **Relavium's advantage:** Relavium now offers **conversational parity** with Claude - Code — an agent chat as a first-class entry point on every surface (`relavium chat`, a - desktop Chat tab, a VS Code coding assistant) — and then adds what Claude Code lacks: - the same conversation is **auto-persisted and resumable**, and **exports to a workflow - scaffold** ([ADR-0026](../decisions/0026-session-export-to-workflow.md)) that becomes a - reviewable, version-controllable `.relavium.yaml`. On top of that come multi-model - routing, multi-agent orchestration, and visual design. A Claude Code user is no longer - asked to leave chat behind to get value — they start in the same place and graduate - when single-agent, single-model, ephemeral work hits its ceiling. - -### Cursor - -- **Strengths:** best-in-class AI-native editor; fast inline completions and multi-file - edits; agent mode with strong codebase awareness; large loyal community; predictable - pricing. -- **Gaps:** editor-only (no CLI, no canvas, no portal); no visual workflow builder; - every session starts from scratch — no reuse; no multi-agent coordination; locked into - the Cursor fork (no standard VS Code or JetBrains); no CI/CD parity; no team workflow - library or governance. -- **Relavium's advantage:** Cursor wins on editor polish but creates vendor lock-in and - has no workflow abstraction layer. Relavium is an orchestration layer *above* any - editor — keep your preferred tools and gain reusable, shareable, versioned workflows, - with a conversational entry point that, unlike Cursor's, survives the session and - exports to a committable artifact. - -### Continue.dev - -- **Strengths:** fully open-source with strong community trust; genuine multi-model - support (including local models); VS Code and JetBrains; highly configurable via - `config.json`; bring-your-own-key, no lock-in. -- **Gaps:** no visual canvas or designer; single-agent sessions only; no - version-controllable workflow files; no CLI for CI/CD; no desktop app; no team/ - enterprise portal; community-paced enterprise roadmap. -- **Relavium's advantage:** Relavium shares the multi-model, bring-your-own-key - philosophy but adds the orchestration and workflow-abstraction layer above chat/ - autocomplete. Continue.dev users who want to *chain* agents and *persist* workflows are - a natural upgrade path — and Relavium's chat carries them there directly, since the - conversation itself exports to the workflow. - -### Cline / Roo-code - -- **Strengths:** highly capable agentic coding with browser and terminal access; - open-source and extensible; strong power-user community; model-agnostic; aggressive - agentic capabilities (filesystem, shell, browser). -- **Gaps:** purely conversational agent paradigm, no canvas; no workflow version control - or reuse; single agent per session; no team collaboration; no CLI for CI/CD; no desktop - or portal; high token cost per session with no optimization layer; no governance. -- **Relavium's advantage:** Cline is the power user's agentic workhorse, but every run is - ephemeral and manual. Relavium matches the conversational paradigm Cline users live in, - then adds the orchestration layer that makes that same execution **resumable**, - reusable, composable, team-shareable, visually designed, and version-controlled — the - chat is where work starts, the exported workflow is where it stops being ephemeral. - -### GitHub Copilot Agent Mode - -- **Strengths:** massive distribution through GitHub; deep GitHub integration (PRs, - issues, Actions, Codespaces); Microsoft enterprise trust; familiar UX for millions; - repo-wide context awareness. -- **Gaps:** GitHub/Microsoft ecosystem lock-in; no visual canvas or designer; no - multi-model routing (Microsoft/OpenAI models only); no local-first execution (runs go - through Microsoft infra); workflows are not portable, version-controllable artifacts; - VS Code only; limited multi-agent coordination; weak for teams not on GitHub. -- **Relavium's advantage:** Copilot Agent Mode is powerful *inside the GitHub bubble* but - deeply locked to it. Relavium is VCS-agnostic, model-agnostic, and surface-agnostic — - the same workflow runs from VS Code, CLI, canvas, or CI regardless of git host, and the - agent chat that produced it is owned locally rather than living in a vendor's infra. - -### n8n / Zapier - -- **Strengths:** mature visual workflow builders with large node libraries; strong - non-technical adoption; n8n is self-hostable and open-core; hundreds of SaaS - connectors; proven enterprise business model. -- **Gaps:** not AI-native (LLM nodes are bolted on, not first-class); no developer-native - surfaces (no VS Code, no dev CLI); cannot natively run code agents or shell commands; - no local-first model (cloud or self-hosted server); no understanding of code, repos, or - dev workflows; no multi-agent AI primitives; no IDE integration; no conversational - entry point at all. -- **Relavium's advantage:** n8n and Zapier own *general* automation but have no - credibility in the developer AI-agent space. Relavium is the developer-native, AI-first - version — with code context, IDE integration, agent orchestration, and a conversational - on-ramp they can't replicate. - -### CrewAI / AutoGen - -- **Strengths:** powerful multi-agent orchestration frameworks; flexible, programmable - agent topologies; active open-source communities; rich agent-to-agent communication; - research and enterprise adoption for complex pipelines. -- **Gaps:** Python framework only — no IDE integration, no visual canvas; no VS Code - extension or CLI for non-Python users; no desktop app or designer; requires Python - expertise; no human-readable, version-controllable workflow format; no native - local-first execution on dev machines; no portal for governance; high barrier to entry. -- **Relavium's advantage:** CrewAI and AutoGen prove multi-agent orchestration is - valuable but demand Python expertise and produce no visual artifacts. Relavium delivers - the same multi-agent power through a conversational entry point, a visual canvas, and - YAML files any developer can read, edit, and share — no framework expertise required. - (Relavium reuses the *concepts* from this space, not the Python idiom; see - [ADR-0003](../decisions/0003-pure-ts-engine-not-langgraph-python.md).) - -## Summary matrix - -| Capability | Claude Code | Cursor | Continue | Cline/Roo | Copilot Agent | n8n/Zapier | CrewAI/AutoGen | **Relavium** | -|---|---|---|---|---|---|---|---|---| -| Multi-surface agent chat | partial (CLI+IDE) | ✗ (editor) | partial (IDE) | ✗ | ✗ | ✗ | ✗ | **✓** | -| Chat → workflow export | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | **✓** | -| Multi-model routing | ✗ | ✗ | ✓ | ✓ | ✗ | partial | ✓ | **✓** | -| Multi-agent orchestration | ✗ | ✗ | ✗ | ✗ | partial | ✗ | ✓ | **✓** | -| Visual workflow canvas | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | **✓** | -| Git-native workflow files | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ (JSON blobs) | partial (code) | **✓** | -| Local-first execution | partial | partial | ✓ | ✓ | ✗ | ✗ | partial | **✓** | -| CLI for CI/CD | ✓ | ✗ | ✗ | ✗ | ✗ | partial | partial | **✓** | -| VS Code extension | ✓ | n/a (fork) | ✓ | ✓ | ✓ | ✗ | ✗ | **✓** | -| Desktop app | ✗ | ✓ (editor) | ✗ | ✗ | ✗ | ✗ (web) | ✗ | **✓** | -| No vendor lock-in | partial | ✗ | ✓ | ✓ | ✗ | partial | ✓ | **✓** | - -## Strategic takeaways - -- **Chat is the on-ramp.** The conversational coding assistant is the entry point users - already reach for, and every competitor in the low-left of the quadrant lives there. - Relavium now meets the senior developer or tech lead *in that conversation* — across - CLI, desktop, and VS Code — instead of asking them to start with a canvas; nobody has to - learn the workflow model to get value on day one. (The committed workflow file remains the - go-to-market *wedge* — see [uvp.md](../uvp.md); chat is the acquisition on-ramp, the workflow is the spread.) -- **The chat → workflow continuum is the moat.** No competitor turns a conversation into - a reviewable, committable artifact. Relavium does: a chat is auto-persisted, resumable, - and **exports to a `.relavium.yaml` scaffold** reviewed before commit - ([ADR-0026](../decisions/0026-session-export-to-workflow.md)). That continuum is what - no session-based tool offers — the same work that started as throwaway chat becomes a - versioned, shareable workflow. Relavium owns *both* ends of the lifecycle. -- **The workflow file is the viral mechanic.** A `.relavium.yaml` committed to a team - repo — including one that *began* as a chat export — is the invite: every teammate who - pulls the repo encounters it, installs the extension to run it, and becomes a user. - Value compounds with team size in a way no session-based tool can match. -- **Risk — chat is the most contested surface, and we compete on the *continuum*, not parity.** - Conversational parity with Claude Code / Cursor / Cline / Copilot is a high bar against - deeply-resourced incumbents, and Relavium's chat surfaces arrive *after* the engine (build phases - 2–4) on a deliberately minimal `LLMProvider` seam. So the bet is made with eyes open: the defensible - moat is **not** out-chatting the category leaders — it is the **chat → workflow continuum + the - git-native, portable workflow artifact + local-first**, which none of them produce. Relavium wins as - the *familiar on-ramp that graduates into something they can't*, not by winning the conversational - surface head-on. Treating "conversational parity" as the headline would be the strategic error to avoid. -- **Local-first is a Phase-1 trust moat.** Until a competitor offers conversational entry - + visual design + local execution + multi-model routing + multi-agent orchestration in - one product with zero data leaving the machine, Relavium owns the security-conscious - developer segment. Chat transcripts are persisted in the encrypted local `history.db`, - not a vendor database. -- **Phase-2 cloud is a business-model unlock, not a feature gap.** Cloud execution, - scheduling, and team governance are deliberately deferred (see - [product-constraints.md](../product-constraints.md)); they are gated on real Phase-1 - demand, not built speculatively. - -> Source: the 7-competitor analysis in the frozen `synthesis-raw.json` (`strategy` -> section), re-read through the agent-first pivot. See the -> [archive provenance map](_archive/README.md). diff --git a/docs/decisions/0009-git-native-workflow-yaml.md b/docs/decisions/0009-git-native-workflow-yaml.md index 9a303b18..61c19829 100644 --- a/docs/decisions/0009-git-native-workflow-yaml.md +++ b/docs/decisions/0009-git-native-workflow-yaml.md @@ -4,9 +4,13 @@ - **Date**: 2026-06-03 - **Related**: [0007-desktop-is-not-an-ide.md](0007-desktop-is-not-an-ide.md), [0008-local-first-phase-1-cloud-phase-2.md](0008-local-first-phase-1-cloud-phase-2.md), [0024-agent-first-entry-point-agentsession.md](0024-agent-first-entry-point-agentsession.md), [0026-session-export-to-workflow.md](0026-session-export-to-workflow.md), [product-constraints.md](../product-constraints.md), [tech-stack.md](../tech-stack.md) +> Amended 2026-06-10: the dated competitive-landscape analysis this ADR cited was removed from +> `docs/analysis/`; its surviving distillation is [uvp.md](../uvp.md), and the two citations below +> now point there. The decision and its reasoning are unchanged. + ## Context -A workflow and the agents inside it have to be *stored* somewhere. The choice of storage format is also a choice about whether workflows are first-class, shareable, reviewable artifacts or opaque rows in a private database. For a developer tool whose differentiator over chat-style competitors is **reusable, version-controllable, team-shareable workflows** (see the competitive analysis in [analysis/competitive-landscape-2026-06-03.md](../analysis/competitive-landscape-2026-06-03.md)), this is a defining decision. +A workflow and the agents inside it have to be *stored* somewhere. The choice of storage format is also a choice about whether workflows are first-class, shareable, reviewable artifacts or opaque rows in a private database. For a developer tool whose differentiator over chat-style competitors is **reusable, version-controllable, team-shareable workflows** (see the competitive framing in [uvp.md](../uvp.md)), this is a defining decision. Two facts constrain it. First, the desktop app is an agent-management center with a *visual* canvas (see [ADR-0007](0007-desktop-is-not-an-ide.md)), while the VS Code extension edits the *same* definitions as text — so the format must be editable cleanly in both modalities without one corrupting the other. Second, the product is local-first with no database-of-record in the cloud (see [ADR-0008](0008-local-first-phase-1-cloud-phase-2.md)); the files on disk *are* the source of truth, and teams must be able to share them the way they share code — by committing, branching, and reviewing in a PR. @@ -30,7 +34,7 @@ Critically, **the YAML schema is treated as a public API from day one**. Because ### Positive -- Workflows and agents are first-class source artifacts: committable, branchable, diff-able, and reviewable in a PR — the core differentiator over chat/CLI-only competitors (see [analysis/competitive-landscape-2026-06-03.md](../analysis/competitive-landscape-2026-06-03.md)). +- Workflows and agents are first-class source artifacts: committable, branchable, diff-able, and reviewable in a PR — the core differentiator over chat/CLI-only competitors (see [uvp.md](../uvp.md)). - Human-friendly authoring: multi-line prompts read naturally, comments capture intent, and diffs are clean and reviewable. - One source of truth for two editing modalities — the visual canvas ([ADR-0007](0007-desktop-is-not-an-ide.md)) and the VS Code text editor edit the very same files. - No database-of-record and no server needed to define or share a workflow, reinforcing the local-first model of [ADR-0008](0008-local-first-phase-1-cloud-phase-2.md). diff --git a/docs/decisions/0034-mcp-client-sdk-dependency.md b/docs/decisions/0034-mcp-client-sdk-dependency.md new file mode 100644 index 00000000..bb4ee239 --- /dev/null +++ b/docs/decisions/0034-mcp-client-sdk-dependency.md @@ -0,0 +1,91 @@ +# ADR-0034: MCP client implementation — the official TypeScript SDK, scheduled in build phase 2 + +- **Status**: Accepted +- **Date**: 2026-06-10 +- **Related**: [ADR-0006](0006-os-keychain-for-api-keys.md), [ADR-0011](0011-internal-llm-abstraction.md), [ADR-0019](0019-cli-node-keychain-library.md), [ADR-0029](0029-tool-policy-hardening.md), [mcp-integration.md](../reference/shared-core/mcp-integration.md), [architectural-principles.md](../standards/architectural-principles.md) + +## Context + +Relavium's MCP integration has been **contract-complete but implementation-unscheduled** since +Phase 0: the canonical spec ([mcp-integration.md](../reference/shared-core/mcp-integration.md)) +defines both directions (agents consuming MCP tools; workflows published as MCP servers), the +`McpServerRef` shape lives in the agent/workflow schemas, `[[mcp_servers]]` registration lives in +[config-spec.md](../reference/contracts/config-spec.md), and the desktop IPC contract reserves +`list_mcp_servers` — yet **no roadmap workstream builds the client**, and the implementation +requires a runtime dependency that the no-new-runtime-dependency-without-an-ADR rule +(CLAUDE.md non-negotiable #2; [architectural-principles.md](../standards/architectural-principles.md) +§9) says must be decided in an ADR **before** any implementation work starts. Leaving the gap open risks the contract and the +implementation drifting apart, or the dependency being adopted ad hoc inside a feature PR. + +Two questions are settled here: **which client implementation** (hand-rolled protocol code vs the +official SDK) and **when** (which build phase owns the workstream). The engine-first critical path +(M2: parser → runner → checkpoint → harness) must not grow; the CLI build phase already resolves +the `[[mcp_servers]]` config and is the first surface that can exercise a live MCP round-trip +end-to-end. + +## Decision + +**We will implement the MCP client (and later the workflow-as-MCP-server adapter) on the official +TypeScript MCP SDK — `@modelcontextprotocol/sdk` — and bind the client implementation to a +dedicated workstream at the start of build phase 2 (the CLI phase), off the M3 critical path.** +The exact version is pinned in the pnpm `catalog:` and recorded in +[tech-stack.md](../tech-stack.md) when the package lands with that workstream, not here. + +Considered alternatives: + +- **Hand-roll the MCP protocol client** (rejected) — Relavium's build-in-house rule targets the + *core* (engine, LLM seam, schedulers), not interop-protocol plumbing. MCP is an external, + still-evolving protocol with an official, actively maintained TS SDK that doubles as the + conformance reference; hand-rolling it buys no differentiation, adds a permanent + protocol-tracking burden, and repeats the class of mistake [ADR-0019](0019-cli-node-keychain-library.md) + avoided by choosing a maintained library over bespoke plumbing. +- **Implement in Phase 1, alongside the engine** (rejected) — the M2 critical path stays lean; the + engine consumes tools through the `ToolRegistry` abstraction either way, so MCP-backed tools are + additive. The CLI phase is the first place a real `mcp_servers` config, keychain-backed secrets, + and a live stdio server can be proven together. +- **Defer the decision itself until someone needs MCP** (rejected) — the contract has been ahead of + the implementation for two phases already; deciding the dependency and the slot now is what keeps + the contract honest and gives the workstream (2.R) an unambiguous starting point. + +Binding guardrails for the implementation (1–4 are existing decisions applied to MCP, cited not +restated; 5 is a new obligation this ADR records): + +1. **Secrets**: MCP server credentials resolve from the OS keychain / secret store into the server + `env` at spawn time and never appear in YAML, logs, or event payloads ([ADR-0006](0006-os-keychain-for-api-keys.md), + [keychain-and-secrets.md](../reference/desktop/keychain-and-secrets.md)). +2. **Egress**: MCP `url` endpoints pass the one shared SSRF primitive — the same range-block used + for provider base URLs and `http_request` ([ADR-0029](0029-tool-policy-hardening.md), + [security-review.md](../standards/security-review.md)). +3. **Type discipline**: SDK types are confined to the MCP integration layer, exactly as provider + SDK types are confined to `@relavium/llm` adapters ([ADR-0011](0011-internal-llm-abstraction.md)); + discovered tools surface to the `ToolRegistry` and the LLM seam only as Relavium/Zod tool + definitions, schema-validated before dispatch. +4. **Tool policy**: MCP-discovered tools enter an agent's tool surface only under the same + narrow-only tool policy as built-in tools ([ADR-0029](0029-tool-policy-hardening.md)); a + per-server `tools_allowlist` is the recommended authoring posture + ([mcp-integration.md](../reference/shared-core/mcp-integration.md#tool-discovery)). +5. **Process hygiene**: stdio servers are spawned with an explicitly constructed environment (the + declared `env` plus a minimal base), never a blanket copy of the host process environment. + +## Consequences + +### Positive + +- The two-phase-old contract↔implementation gap gets an owner (workstream 2.R) and a decided + dependency before any code is written — no ad-hoc adoption inside a feature PR. +- The official SDK tracks protocol revisions for us; conformance against real MCP hosts (Claude + Desktop, Cursor) is testable from day one. +- The engine critical path (M2) is untouched; MCP arrives where it can first be proven end-to-end + (CLI), behind the existing `ToolRegistry` abstraction. + +### Negative + +- A new runtime dependency in the tool path — mitigated by confining SDK types to the integration + layer (guardrail 3), pinning the version in the catalog, and the dependency being absent from + `@relavium/core`'s import graph (the engine sees only `ToolRegistry` shapes). +- MCP support ships later than the engine (build phase 2, not 1) — mitigated by the fact that no + Phase-1 milestone needs a live MCP server, and the schemas/config keep authored files valid in + the meantime. +- The workflow-as-MCP-server direction (outbound) waits further still — it depends on a running + engine plus a surface to host the adapter; 2.R scopes the **client** (inbound) and leaves + outbound to a later workstream rather than inflating the slot. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 1cc58077..2c415c76 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -77,6 +77,7 @@ flowchart TD | 0031 | [`@relavium/llm` seam-shape amendment — first-class multimodal I/O](0031-llm-seam-shape-amendment-multimodal-io.md) | Accepted | 2026-06-08 | | 0032 | [Desktop Rust-side media de-inlining on the egress path (amends 0018)](0032-desktop-rust-media-de-inline-amends-0018.md) | Accepted | 2026-06-08 | | 0033 | [Local config files are strictly validated too (amends 0023)](0033-strict-config-files-amends-0023.md) | Accepted | 2026-06-09 | +| 0034 | [MCP client implementation — the official TypeScript SDK, scheduled in build phase 2](0034-mcp-client-sdk-dependency.md) | Accepted | 2026-06-10 | ## Creating a new ADR diff --git a/docs/ideas/README.md b/docs/ideas/README.md index fc966816..e7e87f8e 100644 --- a/docs/ideas/README.md +++ b/docs/ideas/README.md @@ -24,6 +24,7 @@ matures into a real choice, it graduates to an ADR (and usually appears in | Note | Summary | Status | |------|---------|--------| | [scheduled-and-webhook-triggers.md](scheduled-and-webhook-triggers.md) | Time-based (cron) and HTTP webhook triggers for workflows. Out of scope for Phase 1 because they require an always-on listener. | Phase-2 idea, out of Phase-1 scope | +| [evaluation-harness.md](evaluation-harness.md) | Run a workflow against a git-committed dataset and score the outputs — deterministic scorers first, LLM-as-judge strictly opt-in, in-process via the normal engine. | Phase-2+ idea, not committed | ## When to add a note here diff --git a/docs/ideas/evaluation-harness.md b/docs/ideas/evaluation-harness.md new file mode 100644 index 00000000..626d7de9 --- /dev/null +++ b/docs/ideas/evaluation-harness.md @@ -0,0 +1,54 @@ +# Idea: Workflow Evaluation Harness (datasets + judge) + +- **Status**: Idea — not committed, not on the roadmap +- **Phase**: Phase 2+ (after the CLI regression harness exists) +- **Related**: [roadmap/README.md](../roadmap/README.md), [reference/contracts/workflow-yaml-spec.md](../reference/contracts/workflow-yaml-spec.md), [reference/shared-core/llm-provider-seam.md](../reference/shared-core/llm-provider-seam.md), [standards/testing.md](../standards/testing.md) + +The idea: run a git-committed workflow against a **committed dataset of test cases** and +score the outputs — so a team can change a prompt, a model, or a fallback chain and see +*measured* quality movement instead of eyeballing one run. + +> This is an **idea note, not committed work**. Phase 1's quality story is the CLI +> regression harness (deterministic fixtures, exit codes, event-stream assertions); an +> evaluation harness is the *statistical* complement, and it only makes sense once real +> workflows and the `--json` runner exist. + +## The idea + +- A **dataset** is itself a git-committed artifact (rows of `inputs` + optional expected + outputs / assertions), versioned next to the workflow it exercises — the same + share-by-PR mechanics as the workflow file. +- An **evaluation run** executes the workflow once per row through the normal engine + (in-process, the same `WorkflowEngine` — never an HTTP self-call) and records per-row + outputs, per-row cost (from the existing `CostTracker` micro-cent accounting), and + pass/fail per assertion. +- **Scoring is deterministic-first**: schema checks, exact/regex/numeric assertions, and + diff-style comparisons are the default scorers. An **LLM-as-judge scorer is opt-in, + per-evaluation**, with its own model/provider declared (multi-provider, like any agent) + and its calls metered like any other run cost — never a hidden default that silently + multiplies spend. +- Results land in the local run history (same SQLite, new tables when designed), so the + cost dashboard and run views can show "evaluation #N: 38/40 pass, $0.42". + +## Why it fits Relavium + +The two differentiators do real work here: workflows are **git objects**, so dataset + +workflow + score travel together through PRs (a reviewable quality gate, not a SaaS +dashboard); and the engine is a **library**, so evaluation runs in-process in CI exactly +like the regression harness — no server, no latency tax, no second execution path. + +## Open questions + +- Dataset schema (one canonical shape vs per-workflow assertion plugins). +- Where the boundary with the 2.K regression harness sits (the harness asserts the + *engine* is correct; the eval harness asserts a *workflow* is good — they must not blur). +- Judge reliability: rubric prompts, score calibration, and whether judge verdicts need + N-vote majority before they gate anything. +- CI ergonomics: budget caps for eval runs (ADR-0028 applies per run; an eval multiplies + runs), and a `--json` summary shape for pipelines. + +## Promotion path + +If taken up, this becomes a roadmap workstream plus an ADR for the dataset/score +contracts (and any new dependency). The canonical schemas would live under +[reference/](../reference/README.md); this note then links forward. diff --git a/docs/ideas/scheduled-and-webhook-triggers.md b/docs/ideas/scheduled-and-webhook-triggers.md index 13156669..ca092041 100644 --- a/docs/ideas/scheduled-and-webhook-triggers.md +++ b/docs/ideas/scheduled-and-webhook-triggers.md @@ -61,6 +61,19 @@ deliberately does not ship. enumerates `manual`, `file_change`, and reserves `schedule` / `webhook`). The canonical schema is in [workflow-yaml-spec.md](../reference/contracts/workflow-yaml-spec.md); any real implementation updates that one home, not this note. +- **Execution-model candidate (the part that needs an ADR).** Whatever hosts the + scheduler, the *execution model* should be: **durable and run-id'd** — schedule state + lives in the database keyed to real run records + ([ADR-0022](../decisions/0022-run-references-workflow-by-uuid.md)), never a loose + side-file, so a crash can neither lose a fire nor double-fire (claim-idempotency: a + fire is claimed transactionally before it enqueues, and a zombie claim from a dead + host is reaped on startup); **deterministic catch-up policy** declared per workflow + (skip vs run-once-on-recovery — never an unbounded replay storm); and **scheduling + stays a host/surface concern outside `@relavium/core`** — the engine exposes + `start(workflowId, input)` and stays free of timers and platform imports + ([ADR-0003](../decisions/0003-pure-ts-engine-not-langgraph-python.md)), so the same + scheduler core could drive a Phase-2 cloud queue or a long-running local host without + forking the engine. ## Open questions diff --git a/docs/reference/cli/commands.md b/docs/reference/cli/commands.md index 384109c3..bb0471d4 100644 --- a/docs/reference/cli/commands.md +++ b/docs/reference/cli/commands.md @@ -33,6 +33,15 @@ The CLI auto-detects its environment and switches presentation accordingly: Exit codes are CI-friendly (see [Exit codes](#exit-codes)). +> **Candidate (non-gating): an agent-readable command surface.** Because Relavium's own thesis is +> that work starts in an agent, the CLI is a natural *tool surface for other agents* — two cheap +> affordances are candidates for the build-phase-2 implementation, with neither gating M3: a +> **machine-readable help mode** (`--help` emitting the command/flag surface as JSON, so an agent +> can discover the CLI without scraping prose) and a per-command **`effect` annotation** +> (`read` / `write` / `destructive`) in that output, so an agent's tool policy can gate +> destructive commands behind approval the same way workflow tools are gated. If adopted, the +> shapes are locked here. + ## Commands The command set below is the confirmed surface. Subcommands marked _(planned)_ are intended but not yet locked. diff --git a/docs/reference/contracts/agent-session-spec.md b/docs/reference/contracts/agent-session-spec.md index 5df43829..7451e37e 100644 --- a/docs/reference/contracts/agent-session-spec.md +++ b/docs/reference/contracts/agent-session-spec.md @@ -108,9 +108,11 @@ interface SessionMessage { `SessionMessage` is **mapped to the seam's `LlmMessage` at call time, never copied** — when the session calls a provider, the `AgentRunner` projects the persisted messages into the `LlmMessage` shape owned by [llm-provider-seam.md](../shared-core/llm-provider-seam.md). **No vendor SDK type -crosses the seam** ([ADR-0011](../../decisions/0011-internal-llm-abstraction.md)); the -`DurableContentPart`/`ContentPart` unions are the same Relavium types the seam uses — the -projection resolves durable handles for egress, it never invents a new shape. +crosses the seam** ([ADR-0011](../../decisions/0011-internal-llm-abstraction.md)): both unions are +Relavium-owned types from `@relavium/shared`, but they are **distinct by design** — +`DurableContentPart` is the persisted form (handle-only media, signature-less reasoning), while +`ContentPart` is the in-flight form `LlmMessage` carries. The projection bridges the two existing +types (resolving durable handles for egress); it never invents a new shape. > **Relationship to the run `messages` table.** A session's messages are persisted in > **`session_messages`**, bound to a **session** — distinct from the existing per-step run `messages` diff --git a/docs/reference/contracts/sse-event-schema.md b/docs/reference/contracts/sse-event-schema.md index 48487fde..6e72fa41 100644 --- a/docs/reference/contracts/sse-event-schema.md +++ b/docs/reference/contracts/sse-event-schema.md @@ -218,9 +218,9 @@ Within a turn, the conversational work reuses the **same** `agent:token` / `agen `node:failed.error.code` and `run:failed.error.code` are a closed **`ErrorCode`** enum (not a free string), so surfaces can branch on cause and `retryable` is unambiguous: -`validation` · `provider_auth` · `provider_rate_limit` · `provider_unavailable` · `tool_denied` · `tool_failed` · `budget_exceeded` · `run_timeout` · `cancelled` · `sandbox_error` · `internal` +`validation` · `provider_auth` · `provider_rate_limit` · `provider_unavailable` · `tool_denied` · `tool_failed` · `budget_exceeded` · `run_timeout` · `turn_limit` · `cancelled` · `sandbox_error` · `internal` -The retryable/fatal mapping is owned by [error-handling.md](../../standards/error-handling.md) (e.g. `provider_rate_limit`/`provider_unavailable` retryable; `provider_auth`/`validation`/`tool_denied`/`cancelled` fatal). Messages remain user-safe and secret-free. +The retryable/fatal mapping is owned by [error-handling.md](../../standards/error-handling.md) (e.g. `provider_rate_limit`/`provider_unavailable` retryable; `provider_auth`/`validation`/`tool_denied`/`turn_limit`/`cancelled` fatal). `turn_limit` is the limit-family code for a **hard** agent/session turn/round cap (the exact knob is settled with `AgentSession`, 1.V) — distinct from `run_timeout`/`budget_exceeded` so a capped conversation surfaces its own cause rather than a silent stop; continuing past it is an explicit user action, never a retry. It is **not** the `[chat].max_messages` knob, which is a session-history **trim** threshold ([config-spec.md](config-spec.md)) — trimming continues the session and emits no error. Messages remain user-safe and secret-free. ## Forward-compatibility diff --git a/docs/reference/contracts/workflow-yaml-spec.md b/docs/reference/contracts/workflow-yaml-spec.md index d9293329..244046b8 100644 --- a/docs/reference/contracts/workflow-yaml-spec.md +++ b/docs/reference/contracts/workflow-yaml-spec.md @@ -341,6 +341,19 @@ edges: | `label` | no | Display label. | | `condition` | no | JS expression; the edge is followed only when truthy. Omit for unconditional. | | `data_mapping` | — | **Reserved / engine-internal in v1.0** — not an authored field; reshape state via a `transform` node or a custom `merge_fn` instead. | +| `on_error` | — | **Reserved (forward-compat; not authorable in v1.0)** — see the reservation note below. | + +> **Reserved: `on_error` error-routing edges.** A per-node error port — an edge kind that routes a +> node's *failure* (rather than its output) to a designated handler branch — is **reserved** for a +> post-v1.0 schema revision: the slot is named in this contract so adding it later is additive, but +> there is no v1.0 authored field and no Phase-1 engine handler. Authoring `on_error` today is +> rejected at parse like any unknown key +> ([ADR-0023](../../decisions/0023-strict-authored-yaml-validation.md)). *Considered for v1.0 and +> deliberately deferred:* v1.0 failure semantics stay the simple, predictable pair — per-node +> `retry` plus run failure (a required node is never silently skipped) — and graph-level error +> routing interacts with fan-in strategies, gate timeouts, and checkpoint/resume in ways that +> deserve their own design pass, recorded in a future ADR before the engine grows a third failure +> path. ## Complete example diff --git a/docs/reference/shared-core/node-types.md b/docs/reference/shared-core/node-types.md index 34e3694f..53a42038 100644 --- a/docs/reference/shared-core/node-types.md +++ b/docs/reference/shared-core/node-types.md @@ -145,4 +145,4 @@ A fan-in / aggregator node combines N branch results with **one** named strategy ## Edges -Edges connect nodes by `source_node_id` → `target_node_id`, with an optional `label`, an optional `condition` (the edge is followed only when truthy; null means unconditional), and an optional `data_mapping` that remaps state keys as execution traverses the edge. **`data_mapping` is engine-internal / reserved in v1.0** — it is not an authored YAML field; reshape state via a `transform` node or a `merge_fn` instead. Branch handles on a `condition`/`fan_out` node are referenced from edges as `nodeId:handleName`. See [../contracts/workflow-yaml-spec.md](../contracts/workflow-yaml-spec.md#edges). +Edges connect nodes by `from` → `to` (source node id → target node id, per the authored contract), with an optional `label`, an optional `condition` (the edge is followed only when truthy; null means unconditional), and an optional `data_mapping` that remaps state keys as execution traverses the edge. **`data_mapping` is engine-internal / reserved in v1.0** — it is not an authored YAML field; reshape state via a `transform` node or a `merge_fn` instead. An **`on_error` error-routing edge kind is likewise reserved (forward-compat; not authorable in v1.0)** — the same reserved-slot pattern as `LoopNode`/`best_of_n`, applied to an edge kind: the slot is named in the contract, but v1.0 failure semantics remain per-node `retry` + run failure; the canonical reservation note (with the deferral rationale) lives in [../contracts/workflow-yaml-spec.md](../contracts/workflow-yaml-spec.md#edges). Branch handles on a `condition`/`fan_out` node are referenced from edges as `nodeId:handleName`. See [../contracts/workflow-yaml-spec.md](../contracts/workflow-yaml-spec.md#edges). diff --git a/docs/roadmap/current.md b/docs/roadmap/current.md index 61f8a663..340b17e0 100644 --- a/docs/roadmap/current.md +++ b/docs/roadmap/current.md @@ -2,7 +2,7 @@ > Status: Living -> Last updated: 2026-06-08 +> Last updated: 2026-06-10 - **Related**: [README.md](README.md), [phases/phase-0-foundations.md](phases/phase-0-foundations.md), [phases/phase-1-engine-and-llm.md](phases/phase-1-engine-and-llm.md), [../project-structure.md](../project-structure.md), [../tech-stack.md](../tech-stack.md) @@ -103,9 +103,14 @@ seam amendment — 1.G** (OpenAI/DeepSeek) ‖ **1.H** (Gemini), the **ADR-0030* (reasoning channel + `responseFormat` + `providerExecuted`), and **1.J** (conformance green) — merged in **PR #9** (2026-06-07). All three adapters now pass the shared conformance spec in fixture mode (live nightly reserved, pending keys), with classified errors and capability gating, behind the frozen -`LLMProvider` seam — **🎯 M1 achieved**. Per the -[sequencing plan](phases/phase-1-engine-and-llm.md#sequencing--parallelization), the next work runs -two parallel lanes: +`LLMProvider` seam — **🎯 M1 achieved**. The **ADR-0031 multimodal seam-shape amendment — 1.AD** +(the `media` `ContentPart` arm + handle-only durable fork, the `media_start/delta/end` `StreamChunk` +triad, the `CapabilityFlags.media` matrix with `vision` as its derived alias, `Usage.mediaUnits`, +`LlmRequest.outputModalities`, and the reserved `generateMedia?`/`pollMediaJob?` methods — **shape +only**, with honest all-false adapter matrices and a fail-fast media guard until 1.AE) — merged in +**PR #11** (2026-06-10), landing the union members **before the seam's exhaustive consumers** exist. +Per the [sequencing plan](phases/phase-1-engine-and-llm.md#sequencing--parallelization), the next +work runs two parallel lanes: 1. **Seam policy lane** — **1.K — `FallbackChain` runner** (retryable/fatal routing on `LlmError`, per-attempt usage → `CostTracker`, and the ADR-0030 strip-the-reasoning-signature-on-failover @@ -117,16 +122,23 @@ two parallel lanes: field-named errors (**zero platform imports**), then 1.L → 1.L2 → 1.M → 1.N → 1.R, converging at the **1.O join** (which waits on the fallback runner, 1.K) toward **M2**. -> **Multimodal I/O decided (2026-06-08) — a second pre-freeze seam amendment.** First-class -> image/audio/video I/O (input **and** output, incl. generate-media-by-rule) is now designed and decided: +> **Multimodal I/O — the shape is landed (1.AD ✅ Done, PR #11, 2026-06-10).** First-class +> image/audio/video I/O (input **and** output, incl. generate-media-by-rule) was decided on 2026-06-08: > [ADR-0031](../decisions/0031-llm-seam-shape-amendment-multimodal-io.md) (the seam amendment) + > [ADR-0032](../decisions/0032-desktop-rust-media-de-inline-amends-0018.md) (desktop Rust-side > de-inline), from [multimodal-io-design-2026-06-07.md](../analysis/multimodal-io-design-2026-06-07.md) -> (nine maintainer decisions A1–A9). It adds the **1.AD–1.AH** sub-spine (1.m6, -> [phase-1](phases/phase-1-engine-and-llm.md)): **1.AD lands the seam shape NOW — before the exhaustive -> consumers 1.K/1.O** so the `ContentPart`/`StreamChunk` media union members are non-breaking (the same -> cheap-window move as ADR-0030); **1.AE–1.AH (media input/engine/output + surfaces) are additive and do -> NOT gate M2.** So the immediate seam-lane work is **1.AD → then 1.K**, both before the 1.O join. +> (nine maintainer decisions A1–A9), as the **1.AD–1.AH** sub-spine (1.m6, +> [phase-1](phases/phase-1-engine-and-llm.md)). **1.AD landed the seam shape before the exhaustive +> consumers 1.K/1.O** (the same cheap-window move as ADR-0030), so the media union members are +> non-breaking; the seam doc carries the full amendment section. **1.AE–1.AH (media +> input/engine/output + surfaces) are additive and do NOT gate M2** — the seam lane proceeds straight +> to **1.K**. + +> **MCP client scheduled (2026-06-10).** The long-standing contract-ahead-of-implementation gap is +> closed at the decision layer: [ADR-0034](../decisions/0034-mcp-client-sdk-dependency.md) pins the +> official TypeScript MCP SDK as the client implementation and binds the inbound client to +> **workstream 2.R** at the start of [build phase 2](phases/phase-2-cli.md) (off the M3 critical +> path). No Phase-1 work changes. Carry-over hardening is tracked in [deferred-tasks.md](deferred-tasks.md) — pick items up as Phase 1 first touches each file. diff --git a/docs/roadmap/deferred-tasks.md b/docs/roadmap/deferred-tasks.md index 5893aa30..3d6cefd7 100644 --- a/docs/roadmap/deferred-tasks.md +++ b/docs/roadmap/deferred-tasks.md @@ -2,7 +2,7 @@ > Status: Living -> Last updated: 2026-06-09 +> Last updated: 2026-06-10 - **Related**: [current.md](current.md), [README.md](README.md), [phases/phase-0-foundations.md](phases/phase-0-foundations.md) @@ -34,6 +34,15 @@ Severity is the review's verified rating. Check an item off in the PR that resol > multimodal forward-obligations** below (SSRF primitive, async-job ADR, media cost estimate, `partialRef` > semantics, `workspace` authz scope, retention/GC table, `vision`-alias retirement) so nothing is lost. +> **2026-06-10 engine/tooling review pass:** a review of the engine, tool, and CI surfaces against +> the current contracts produced a small set of additions, recorded in their sections below: the +> **tool-output size gate + spill-to-disk** (1.T), **conformance tool-loop / cache-hit scenarios** +> (1.F follow-up), a **token-estimate accuracy** watch item (1.AC), the **Leakwatch CI gate** +> (deferred pending a distribution path), and a **dependency-bump cooling window** (pending a pnpm +> major). The same pass settled three decisions outside this file: the MCP client dependency and +> scheduling (ADR-0034 / workstream 2.R), the reserved `on_error` edge kind +> (workflow-yaml-spec.md), and the `turn_limit` `ErrorCode` (constants.ts + sse-event-schema.md). + ## Decisions needed (maintainer call) - [x] **Workflow `agents:` `$ref` support** — the @@ -84,12 +93,15 @@ Severity is the review's verified rating. Check an item off in the PR that resol ### Multimodal forward-obligations (carry the not-yet-coded pieces — see ADR-0031) -- [ ] **Media-arm integrity metadata (Y3) — DECIDED 2026-06-09 (ADR-0031 amended), land at 1.AD.** The +- [x] **Media-arm integrity metadata (Y3) — DECIDED 2026-06-09 (ADR-0031 amended), land at 1.AD.** The durable form (`DurableMediaPart`) carries an optional **`byteLength?`** + audio/video **`durationMs?`**, host-populated at the `deInlineMedia` boundary; **no `checksum`** (the `media://sha256-` handle IS the sha256); **no `width`/`height`** in Phase A (render-only). **Must ship in the 1.AD seam shape** (before 1.K/1.O exhaustive consumers) — adding a union-arm field later is breaking. `byteLength` is what the byte-delivery Range check bounds against. *(ADR-0031 "Amended 2026-06-09"; multimodal-io-design §3.2; 1.AD)* + **✅ Landed at 1.AD (PR #11, 2026-06-10):** `byteLength?`/`durationMs?` ship on `DurableMediaPart` only + (the in-flight arm stays lean — parse-stripped, tested), with the `durationMs`-is-audio/video-only rule + enforced on both the standalone schema and the durable union. - [ ] **Shared SSRF range-primitive (the `url`-carrier precondition)** — the one shared HTTPS-only / block-private-loopback-link-local-metadata-CGNAT / DNS-resolution + per-hop-redirect-revalidation / IPv4-mapped-IPv6-decode primitive that `assertHttpsBaseUrl` (openai.ts) is the best-effort placeholder @@ -98,7 +110,11 @@ Severity is the review's verified rating. Check an item off in the PR that resol the landing-gate CI test (url rejected while flag off). **A runtime-*derived* base URL** (auto-selected / resolved, not literally user-supplied) is re-checked through this same primitive against its **post-resolution IP**; an explicit-local-endpoint opt-out **narrows, never removes**, the - metadata-IP/link-local block. *(security-review.md; openai.ts; 1.AE)* + metadata-IP/link-local block. **The primitive must also pin the connection to the IP it validated** + (connect-by-validated-IP / a lookup-pinned agent): validating one resolution and letting the HTTP + client re-resolve is a TOCTOU window DNS-rebinding walks through — the check and the connect must + see the same address (a binding control in [security-review.md](../standards/security-review.md)). + *(security-review.md; openai.ts; 1.AE)* - [ ] **Async media-job ADR (`generateMedia`/`pollMediaJob` behavior, A5)** — the seam shape is reserved now (1.AD); the engine-owned **poll / checkpoint / resume / cancel loop** for minute-scale LROs (Sora/Veo) — in the run loop (1.N) + checkpointer (1.R), reusing `LlmError` classification — gets **its @@ -152,6 +168,20 @@ Severity is the review's verified rating. Check an item off in the PR that resol surfaces add i18n: a CI test that **fails** on a missing/extra translation key (parity), **zero conditional logic in translation data** (data ≠ code), and a dead/unused-string lint. Recorded now; lands with the Phase-2/3/4 surface i18n work (no consumer yet). *(a `docs/standards/` entry or skill; Phases 2–4)* +- [ ] **Tool-output size/token gate + spill-to-disk (1.T).** Today only the *event* `outputSummary` is + truncated; the tool result handed back to the model has no bound, so one oversized `read_file` / + `http_request` / MCP result can blow the context window (a cost/DoS surface ADR-0028's pre-egress + governor cannot see, since the damage lands in the *next* request). Add to the `ToolRegistry` + dispatch path (1.T): a byte/token ceiling per tool result with an explicit truncation marker, and + for over-threshold output (e.g. >2000 lines / >50KB) spill the full output to a workspace-scoped + file and hand the model a bounded preview + the path (readable via the normal FS-scope-tiered + tools). Behavior belongs in [built-in-tools.md](../reference/shared-core/built-in-tools.md) when + implemented. *(1.T; built-in-tools.md)* +- [ ] **Pre-egress token-estimate accuracy — watch item (1.AC).** The ADR-0028 governor blocks on + `worstCaseNextEstimate(maxTokens)` from `[defaults].max_tokens_estimate`. Record the open + question: does the estimate need provider-accurate token counting (from the seam's model meta / + usage feedback) to avoid systematic over/under-blocking, or is the declared estimate enough? + No change now — re-evaluate with real 1.AC telemetry. *(1.AC; ADR-0028)* ## Schema / validation hardening @@ -252,6 +282,14 @@ Severity is the review's verified rating. Check an item off in the PR that resol rejects, pinning the record boundary. *(nit · run.test.ts, run-event.test.ts)* - [x] **Round-trip fixture verbatim** — the workflow no-drift fixture paraphrases multi-line prompts; transcribe them verbatim from the spec or soften the "verbatim" claim. *(nit · workflow.test.ts)* +- [ ] **Conformance: tool-loop + cache-hit recorded scenarios (1.F follow-up)** — the shared + conformance suite covers text / single tool-call / usage / stop / error, but not a **multi-turn + tool loop** (call → result → continuation on the same provider, the path every agent node + exercises) or a **prompt-cache-hit** response (cached-token usage fields folding into the one + canonical `Usage`). Add both as recorded scenarios, and grow a small provider-quirk fixture bank + (reasoning-field variants, tool-call adjacency rules) as quirks are met in the adapters — the + suite is where quirk knowledge belongs, not adapter comments. *(packages/llm conformance; next + adapters-touching PR wave)* ## Tooling / CI @@ -284,6 +322,20 @@ Severity is the review's verified rating. Check an item off in the PR that resol uncomment the `schedule:` lane and add the provider API keys (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY` / `GEMINI_API_KEY` / `DEEPSEEK_API_KEY`) as CI secrets. Until then live coverage is a known gap. *(minor → keys · ci.yml, packages/llm/src/conformance/*.conformance.test.ts)* +- [ ] **Leakwatch secret-scanning CI gate** — CI has no secret-scan step. The HodeTech standard + scanner is **Leakwatch** (never gitleaks); the blocking `ci.yml` step is wired once a + distribution path for the binary onto Actions runners exists (private release / action). Until + then scanning runs locally with the installed binary, and test fixtures keep building any + key-shaped strings via `join()` so no contiguous key literal ever sits in the tree. Exceptions, + when the gate lands, are documented per finding — never blanket-ignored. + *(blocked → distribution path · ci.yml, security-review.md)* +- [ ] **Dependency-bump cooling window** — adopt a "no same-day upgrades" posture for runtime + dependency bumps: a freshly published version waits a cooling period before entering the + lockfile (supply-chain compromise of a new release is typically detected within days), with a + documented security-exception path (a CVE fix may skip the window, recorded in the PR). pnpm 9 + has no native knob for this; enforce as review policy now and revisit native enforcement + (e.g. a minimum-release-age setting) when the toolchain moves to a pnpm major that has one. + *(policy now, tooling later · pnpm-workspace.yaml, architectural-principles.md)* ## Docs diff --git a/docs/roadmap/phases/phase-1-engine-and-llm.md b/docs/roadmap/phases/phase-1-engine-and-llm.md index a95b9743..8b6545ba 100644 --- a/docs/roadmap/phases/phase-1-engine-and-llm.md +++ b/docs/roadmap/phases/phase-1-engine-and-llm.md @@ -21,6 +21,8 @@ > The new sub-spine **1.AD–1.AH** (1.m6) lands the **shape (1.AD) before the exhaustive consumers 1.K/1.O** > so the `ContentPart`/`StreamChunk` media union members are non-breaking; the **behavior (1.AE–1.AH) is > additive and does NOT gate M2** (it threads into the engine and Phases 2–6, like the agent-first sub-spine). +> **1.AD is ✅ Done (PR #11, 2026-06-10)** — the shape landed with all-false adapter matrices and the +> fail-fast media guard; 1.K is unblocked. - **Related**: [../README.md](../README.md), [phase-0-foundations.md](phase-0-foundations.md), [phase-2-cli.md](phase-2-cli.md), [../../architecture/shared-core-engine.md](../../architecture/shared-core-engine.md), [../../architecture/execution-model.md](../../architecture/execution-model.md), [../../architecture/multi-llm-providers.md](../../architecture/multi-llm-providers.md), [../../reference/shared-core/llm-provider-seam.md](../../reference/shared-core/llm-provider-seam.md), [../../reference/shared-core/node-types.md](../../reference/shared-core/node-types.md), [../../reference/shared-core/built-in-tools.md](../../reference/shared-core/built-in-tools.md), [../../reference/contracts/sse-event-schema.md](../../reference/contracts/sse-event-schema.md), [../../standards/testing.md](../../standards/testing.md), [../../standards/error-handling.md](../../standards/error-handling.md), [../../decisions/0011-internal-llm-abstraction.md](../../decisions/0011-internal-llm-abstraction.md) @@ -395,6 +397,14 @@ budgets. Adapters stay dumb; this owns the policy. - On a **classified-retryable** `LlmError`, exhaust the entry's `max_attempts` with backoff, then advance to the next entry; on a **fatal** `LlmError`, stop immediately (never mask a real bug by falling through). +- Three behavior nuances the chain must honor: an **auth-class error is never blindly + retried** (a `provider_auth` failure repeats deterministically — at most one out-of-band + credential-refresh path, never the attempt loop); a **rate-limited entry is parked in a + short cooldown** so an immediately-following call on the same chain does not hammer the + saturated provider again; and there is **no failover after the first content chunk** — + once a stream has begun emitting, a mid-stream error surfaces to the node-retry layer + (1.S) instead of silently re-issuing on the next provider (which would duplicate tokens + and tool side effects). - Record **per-attempt usage** and feed it to the `CostTracker` (1.B) so cost stays accurate across failover. - Surface the final outcome plus the attempt trace (which providers were tried, why @@ -413,6 +423,10 @@ part or `signature` into the next provider's request**; and the **fail-over/stop function of the classified `LlmError` discriminant** (`kind`/`retryable`, 1.I), **never** content / string-sentinel inspection — a response body containing `'Error:'` or an empty/malformed body does not by itself trigger failover, and the normalized `LlmError.message` is secret-free (security-review.md). +Additionally: a `provider_auth` failure is not re-attempted on the same entry; an error after the +first streamed content chunk does **not** advance the chain; and every failover is **visible** — +the attempt trace (which entry failed, why, where the chain advanced) reaches the run event/log, +never a silent provider switch. *(Score/quality-threshold fallback is deliberately out of scope for Phase 1 — deferred-tasks.md.)* ### 1.L.0 — Reconcile `@relavium/shared` to the 2026-06-05 contract — ✅ **Done (PR #6)** · *critical path, do first* @@ -470,7 +484,10 @@ The engine entry point: load a `.relavium.yaml` and validate it against the **Acceptance:** valid reference example workflows parse to a typed `WorkflowDefinition`; a battery of malformed files each fail with a field-named, -secret-free error; round-trip (parse → object) preserves all node config blocks; and **parsing is pure / +secret-free error — including an authored **`on_error` edge field, which is a *reserved* edge kind +(not a v1.0 field; [workflow-yaml-spec.md §Edges](../../reference/contracts/workflow-yaml-spec.md#edges)) +and must be rejected at parse with a field-named error like any unknown key**; round-trip +(parse → object) preserves all node config blocks; and **parsing is pure / side-effect-free** — no file mutation, no `process.env` mutation, no import-time singleton init (validation only). *(Note for a future maintainer: the `config.*` schemas are `.strict()` by a **deliberate choice beyond** ADR-0023's lenient-config default — "parity with the authored YAML" — not a bug to "fix".)* @@ -533,10 +550,24 @@ The loop that dispatches ready nodes and emits the canonical event stream. consumes). - Define the execution-mode interface seam (local now; cloud later) so the loop is identical across modes. +- Dispatch is a **completion-driven ready queue** (a node becomes ready when its last + dependency settles) — never a sleep/poll loop; the JS event loop is the scheduler. +- Cancellation is **cooperative end-to-end**: the `AbortSignal` threads through the + `AgentRunner` into the seam call and tool dispatch, so an in-flight provider stream or + tool is actually aborted, not orphaned to finish in the background. +- Condition branches **skip-propagate**: when a branch is not selected, every node + reachable *only* through it is marked skipped (recursively), and a downstream fan-in + counts skipped branches against its join strategy instead of waiting forever. +- The bus has exactly **one** producer-side translation point (engine internals → the + canonical `RunEvent` union); persistence, cost, and UI subscribe as **passive + consumers** — observation never mutates run state, and intervention happens only through + the engine API (`cancel` / `resume`), never through a listener. **Acceptance:** running a multi-node plan emits events in a gap-free `sequenceNumber` order asserted against the canonical schema by colon-namespaced name; cancellation -emits `run:cancelled` and stops dispatch; and **every run terminates in EXACTLY ONE terminal event** +emits `run:cancelled` and stops dispatch **and aborts the in-flight provider call/tool +(cooperative, asserted mid-stream)**; an unselected condition branch's entire subtree is +skipped and a fan-in over it still joins; and **every run terminates in EXACTLY ONE terminal event** (`run:completed` | `run:failed` | `run:cancelled`) — including on an **uncaught node-handler exception** (inject a throw → assert a single `run:failed`) and on **crash-then-restart of a non-resumable run** (reconciliation emits `run:failed`, never a stuck `run:started`) — so a zombie / never-terminating run is @@ -564,6 +595,15 @@ Executes a single agent node end-to-end against `@relavium/llm`. feedback path. - Thread `AbortSignal` for cancellation; map a final node failure to `node:failed` with a user-safe message + internal correlation id. +- Structure the turn loop as **small, separately-testable steps** (assemble → call → + categorize chunk → dispatch tool → persist/emit), each a function with explicit + inputs/outputs — never one monolithic loop body with shared mutable flags. +- A node sees only its **declared inputs** (the resolved `{{ … }}` references and its + config block), never the whole run state or another node's transcript — context + isolation is what keeps prompts small and node behavior reproducible. +- Tool results enter message assembly **as data through the typed untrusted boundary** + ([security-review.md §Prompt-injection posture](../../standards/security-review.md#prompt-injection-posture), + binding): never into `system`, never string-concatenated into an instruction template. **Acceptance:** an agent node with a tool streams tokens, performs a tool round-trip, emits a correct `cost:updated`, and completes with `node:completed`; a forced @@ -628,10 +668,27 @@ Persist state at every node boundary and reconstruct a run from it. in-flight runs from their last checkpoint rather than losing them). - Use a stable idempotency key (`runId + nodeId + retryCount`) so a retry never double-applies side effects; align gap detection to `sequenceNumber`. +- **Checkpoint writes are crash-safe and loud.** A `Checkpointer` implementation persists + atomically (a transaction for the DB-backed store; write-temp → fsync → rename for any + file-backed one) so a crash mid-write can never leave a torn checkpoint; a checkpoint + **write failure surfaces as a typed error/event** ([error-handling.md](../../standards/error-handling.md) + no-silent-catches), never a warn-and-continue that quietly forfeits resumability. +- **Resume validates identity.** `resume` checks the run's frozen + `workflow_definition_snapshot` against the plan being resumed and **refuses a mismatch** + with a typed error (an edited workflow resumes via retry-from-node against the snapshot, + not by replaying a stale plan over a new graph). +- The derived `CheckpointState` carries a **`schemaVersion`** so a later engine revision + can detect, migrate, or refuse an older derivation instead of misreading it. Forward + note for the Phase-2 persistence wiring: bound `run_events` row width — an + over-threshold node output is stored once out-of-row and referenced, keeping event-log + replay cheap (canonical DDL change goes to + [database-schema.md](../../reference/desktop/database-schema.md) when wired). **Acceptance:** a run interrupted after node 2 resumes from the checkpoint and completes nodes 3..N without re-running 1..2; re-applying a checkpointed node is -idempotent; gap detection triggers a resync rather than trusting a partial view. +idempotent; gap detection triggers a resync rather than trusting a partial view; +resume against a non-matching workflow snapshot is refused with a typed error; and a +forced checkpoint-write failure is visible as a typed error/event, not a silent skip. ### 1.S — Retry + fallback wiring (node budget above the chain) — *critical path* @@ -668,6 +725,24 @@ The engine-side registry that dispatches built-in tools the `AgentRunner` invoke - Plumb `invoke_agent` so an agent can dispatch another agent node by id (the orchestrator delegation mechanism), without building the full router selection logic this phase. +- **Bound tool results** (the model-facing result, not just the event summary): a + byte/token ceiling per result with an explicit truncation marker; the over-threshold + spill-to-file + bounded-preview behavior is the + [deferred-tasks.md](../deferred-tasks.md) tool-output-gate entry, landed here or in the + first pass that hits a real oversized output (behavior documented in + [built-in-tools.md](../../reference/shared-core/built-in-tools.md) when implemented). +- Tool resolution is **exact-match by id**: an unknown or misspelled tool name is a typed + error that lists the available tools — never fuzzy/nearest-name matching (the + wrong-tool-executed failure mode), consistent with ADR-0029's exact-match posture. +- Tool parameter schemas distinguish **LLM-visible vs config-only** parameters: + config-pinned values are merged at dispatch and never enter the LLM-facing JSON Schema + (smaller prompts, and a config value can't be overridden by a model-supplied argument). + The concrete field shape is documented in + [built-in-tools.md](../../reference/shared-core/built-in-tools.md) when this lands. +- A tool that spawns a process passes an **explicitly constructed environment** (its + declared vars + a minimal base) — never a blanket copy of the host environment, which + would hand every subprocess the host's secrets and hijack vectors (`PATH`, + `NODE_OPTIONS`, …). **Acceptance:** an agent tool call dispatches through the registry, maps I/O correctly, and emits sanitized tool events; an unlisted `run_command` is refused; @@ -700,7 +775,7 @@ reproduces the same final output — **M2 achieved**. These build the `AgentSession` entry point ([ADR-0024](../../decisions/0024-agent-first-entry-point-agentsession.md)). They run **parallel** to 1.L–1.U and do **not** feed the 1.U workflow harness — each is proven by its own harness (1.AA). The `WorkflowEngine` is unchanged; `AgentSession` is an additional entry point on the same substrate. -- **1.V — `AgentSession` entry point.** Wrap `AgentRunner` in a multi-turn session (session context, one bound agent + its fallback chain). *Acceptance:* a session runs a multi-turn conversation with a tool round-trip through the same `AgentRunner` path a workflow agent node uses. +- **1.V — `AgentSession` entry point.** Wrap `AgentRunner` in a multi-turn session (session context, one bound agent + its fallback chain). This workstream also settles the session's **hard turn/round cap** knob — deliberately distinct from `[chat].max_messages`, which is a history-**trim** threshold ([config-spec.md](../../reference/contracts/config-spec.md)) that continues the session. A session that reaches the hard cap ends **loudly**: `session:turn_completed` carries `error.code: 'turn_limit'` ([sse-event-schema.md](../../reference/contracts/sse-event-schema.md#error-code-taxonomy)) — never a silent stop — and the behavior is pinned by a dedicated regression test (a refactor of the turn loop must not be able to silently drop the cap signal). Context compaction (when it lands, later phases) is **append-only by principle**: the persisted transcript is never rewritten or trimmed in place; `agent:context_compacted` is the reserved signal ([sse-event-schema.md](../../reference/contracts/sse-event-schema.md#workflow-governance-and-reserved-events)). *Acceptance:* a session runs a multi-turn conversation with a tool round-trip through the same `AgentRunner` path a workflow agent node uses; a session driven to its hard turn cap emits the `turn_limit`-coded event, regression-pinned. - **1.W — `session:*` event namespace.** Emit session lifecycle events on the shared `RunEventBus` with the same `sequenceNumber` gap/resync logic ([sse-event-schema.md](../../reference/contracts/sse-event-schema.md)). *Acceptance:* session events are disjoint from `run:*` and gap-detected identically. - **1.X — Session persistence.** `agent_sessions` + `session_messages` via `@relavium/db` into `history.db` ([database-schema.md](../../reference/desktop/database-schema.md)). *Acceptance:* a session round-trips to the DB and resumes. **Note:** adding these two tables requires a regenerated Drizzle migration snapshot (the schema-migration drift CI gate). **ADR-0030 ephemerality:** a `reasoning` part's `signature`/`redacted` continuity token must **not** be persisted to `session_messages` — strip it (keep reasoning *text* if a transcript needs it, drop the opaque signature). *Acceptance also asserts:* a round-tripped session row carries no reasoning `signature`. - **1.Y — Session checkpoint/resume.** Reuse the idempotency-key logic so a session resumes after a restart. @@ -715,7 +790,7 @@ Per [ADR-0027](../../decisions/0027-expression-sandbox.md): a deterministic, res ### 1.AC — Resource governor (pre-egress budget) — folds into 1.O -Per [ADR-0028](../../decisions/0028-workflow-resource-governance.md): the **pre-egress** budget check, a run `timeout_ms`, and a parallel concurrency cap, with `pause_for_approval` reusing the human-gate seam and emitting `budget:warning` / `budget:paused` / `run:timeout`. The cost formula and `on_exceed` semantics are owned by ADR-0028; this workstream wires them into 1.O. +Per [ADR-0028](../../decisions/0028-workflow-resource-governance.md): the **pre-egress** budget check, a run `timeout_ms`, and a parallel concurrency cap, with `pause_for_approval` reusing the human-gate seam and emitting `budget:warning` / `budget:paused` / `run:timeout`. The cost formula and `on_exceed` semantics are owned by ADR-0028; this workstream wires them into 1.O. The estimator itself is a **pure function** (model meta + declared estimate in → budget verdict out, no I/O, no ambient state) so it is unit-testable in isolation and reusable wherever a context/token budget is computed (e.g. session context assembly), and its accuracy is a recorded watch item ([deferred-tasks.md](../deferred-tasks.md)). **Acceptance:** a run that would exceed its budget fails or pauses **before** the next LLM call; the concurrency cap bounds a wide fan-out. @@ -725,13 +800,13 @@ First-class multimodal I/O (image / audio / video, **input AND output**, incl. a **generates** a media file) per [ADR-0031](../../decisions/0031-llm-seam-shape-amendment-multimodal-io.md) + [ADR-0032](../../decisions/0032-desktop-rust-media-de-inline-amends-0018.md), designed in [multimodal-io-design-2026-06-07.md](../../analysis/multimodal-io-design-2026-06-07.md). The **shape** -(1.AD) is a second pre-freeze seam amendment in the ADR-0030 mould — it lands **before the exhaustive +(1.AD) is a second pre-freeze seam amendment in the ADR-0030 mould — it landed **before the exhaustive consumers** (1.K `FallbackChain`, 1.O `AgentRunner`) so adding the `ContentPart`/`StreamChunk` media union members is non-breaking. The **behavior** (1.AE–1.AH) is **additive — it does NOT gate M2** (like the agent-first sub-spine): media plumbing layers onto the proven engine and threads into the surface phases (2–6). Each phase below maps to the design doc's Phase A–E. -- **1.AD — Multimodal seam amendment (Phase A) — land NOW, before 1.K/1.O.** Add to `@relavium/shared` +- **1.AD — Multimodal seam amendment (Phase A) — landed before 1.K/1.O — ✅ Done (PR #11, 2026-06-10).** Add to `@relavium/shared` + the `@relavium/llm` seam types, *shape only, no behavior*: the MIME-discriminated `media` `ContentPart` arm **and** the distinct handle-only `DurableMediaPart`/`DurableContentPart` (the durable form also carrying optional `byteLength?` + audio/video `durationMs?` — Y3, ADR-0031 amended); the @@ -818,7 +893,7 @@ the latter being the critical-path milestone for the whole product. | 1.m4 | Agent + non-agent node handlers, gate, checkpoint/resume, retry, tools, **expression sandbox** + pre-egress budget | 1.O, 1.P, 1.Q, 1.R, 1.S, 1.T, **1.AB**, **1.AC** | | **M2** | **Engine end-to-end from a Node harness (stream + checkpoint + retry + fallback) — CRITICAL-PATH MILESTONE** | **1.U** | | 1.m5 | Agent-first sub-spine: `AgentSession` + session events + persistence + checkpoint/resume + export, proven by its own harness (**additive, parallel — does NOT gate M2**) | 1.V, 1.W, 1.X, 1.Y, 1.Z, 1.AA | -| 1.m6 | Multimodal I/O: seam amendment (**1.AD — lands before 1.K/1.O so the union members are non-breaking**), then media input/engine/output behavior (**additive — does NOT gate M2**) + surfaces threaded into Phases 2–6 ([ADR-0031](../../decisions/0031-llm-seam-shape-amendment-multimodal-io.md)/[0032](../../decisions/0032-desktop-rust-media-de-inline-amends-0018.md)) | **1.AD**, 1.AE, 1.AF, 1.AG, 1.AH | +| 1.m6 | Multimodal I/O: seam amendment (**1.AD ✅ Done, PR #11 — landed before 1.K/1.O so the union members are non-breaking**), then media input/engine/output behavior (**additive — does NOT gate M2**) + surfaces threaded into Phases 2–6 ([ADR-0031](../../decisions/0031-llm-seam-shape-amendment-multimodal-io.md)/[0032](../../decisions/0032-desktop-rust-media-de-inline-amends-0018.md)) | **1.AD ✅**, 1.AE, 1.AF, 1.AG, 1.AH | ## Sequencing & parallelization @@ -977,7 +1052,7 @@ flowchart LR | 1.Y | C | 1.X, 1.R | 1.AA | ◇ | | 1.Z | C | 1.V, 1.L | 1.AA | ◇ | | 1.AA | C | 1.V, 1.W, 1.X, 1.Y, 1.Z | **1.m5** | ◇ | -| 1.AD | D | 1.A (seam types) | **must precede 1.K, 1.O** (non-breaking union members); 1.AE | ⬤ shape-only, precedes consumers | +| 1.AD | D | 1.A (seam types) | **must precede 1.K, 1.O** (non-breaking union members); 1.AE | ⬤ shape-only — ✅ **Done (PR #11)** | | 1.AE | D | 1.AD, 1.G/1.H (adapters) | 1.AF | ◇ | | 1.AF | D | 1.AE, 1.K, 1.N, 1.R | 1.AG | ◇ | | 1.AG | D | 1.AF | 1.AH | ◇ | @@ -991,7 +1066,8 @@ flowchart LR > **Lane D (multimodal, 1.AD–1.AH)** is omitted from the waves Mermaid to keep it readable. Its one > scheduling constraint: **1.AD (shape only) must land before 1.K and 1.O** — the same cheap-window logic > as the ADR-0030 amendment — so the media union members are added before any exhaustive `switch` exists. -> Everything after 1.AD (1.AE–1.AH) is **additive and never gates M2**, mirroring Lane C. +> That constraint is **satisfied: 1.AD is ✅ Done (PR #11, 2026-06-10)**. Everything after 1.AD +> (1.AE–1.AH) is **additive and never gates M2**, mirroring Lane C. ### Solo vs. multi-track diff --git a/docs/roadmap/phases/phase-2-cli.md b/docs/roadmap/phases/phase-2-cli.md index 8a3580eb..3e443386 100644 --- a/docs/roadmap/phases/phase-2-cli.md +++ b/docs/roadmap/phases/phase-2-cli.md @@ -56,6 +56,11 @@ the [config resolution order](../../reference/contracts/config-spec.md), and the `run_events`, `run_costs`) so `list`/`logs`/`status` read durable state. - A CLI-driven engine regression harness: a small set of example workflows run with `--json` and asserted on in CI on every engine change. +- The **MCP client** (inbound direction of + [mcp-integration.md](../../reference/shared-core/mcp-integration.md)): connect/spawn the + `[[mcp_servers]]` registrations resolved by 2.B and surface their tools through the engine's + `ToolRegistry`, per [ADR-0034](../../decisions/0034-mcp-client-sdk-dependency.md) (2.R — off the + M3 critical path). ### Explicitly out of scope @@ -77,7 +82,10 @@ the [config resolution order](../../reference/contracts/config-spec.md), and the The workstreams are ordered for execution. Critical-path workstreams (`2.A`, `2.B`, `2.D`, `2.F`, `2.K`) gate the global milestone **M3**. Earlier workstreams -unblock later ones; build them in this order. +unblock later ones; build them in this order. (`2.R` is the one exception to the +documentation order: it appears after the chat workstreams below but is +**scheduled early** — implementation needs only 2.B + 2.C, while its end-to-end +acceptance fixture additionally needs 2.F's `relavium run` to exist.) ```mermaid flowchart LR @@ -94,6 +102,8 @@ flowchart LR F --> K["2.K regression harness"] I --> K K --> L["2.L packaging & publish"] + B --> R["2.R MCP client"] + C --> R ``` ### 2.A — CLI skeleton (`commander.js`) and process contract @@ -124,6 +134,10 @@ flags) every command inherits. - Centralize error rendering: human-readable to a TTY, structured `RunEvent`-shaped error object to stdout under `--json`; never leak a stack trace as the primary output. +- Keep each command's **core framework-free**: the logic lives in a plain module + (parsed args in, typed result out) that imports neither `commander` nor `ink`; the + CLI layer is thin wiring around it. This is what makes commands unit-testable + without a TTY and keeps the TUI and `--json` paths renderers over one core. **Acceptance:** `relavium --help` and every subcommand's `--help` print the documented surface; `relavium ` exits `2`; output mode is correctly @@ -419,6 +433,40 @@ The interactive agent entry point on the CLI ([ADR-0024](../../decisions/0024-ag **Acceptance:** an interactive `relavium chat` streams a multi-turn conversation with a tool call, persists, and resumes; `chat --json` emits a deterministic `session:*` stream; `agent run` invokes a single agent headlessly; chat `/exit` returns exit code 4. +### 2.R — MCP client integration (inbound) + +Implement the **inbound** half of [mcp-integration.md](../../reference/shared-core/mcp-integration.md) +— agents consuming MCP tools — on the official TypeScript SDK per +[ADR-0034](../../decisions/0034-mcp-client-sdk-dependency.md). Scheduled **early** (implementation +needs only 2.B config + 2.C keys; the end-to-end acceptance below additionally requires 2.F's +`relavium run`) but **off the M3 critical path**; the workflow-as-MCP-server (outbound) direction +is deliberately not in this slot. + +**Tasks:** + +- Add the SDK dependency (version pinned in the pnpm `catalog:`, recorded in tech-stack.md) per + [ADR-0034](../../decisions/0034-mcp-client-sdk-dependency.md); SDK types stay confined to the MCP + integration layer — discovered tools surface to the `ToolRegistry` only as Relavium/Zod tool + definitions (the same confinement discipline as provider SDKs in `@relavium/llm`). +- Resolve `[[mcp_servers]]` registrations through the 2.B config loader (global → project merge per + [config-spec.md](../../reference/contracts/config-spec.md)); spawn stdio servers / connect + SSE/WebSocket servers, run `tools/list` discovery, and register tools under the + `mcp_{server}_{tool}` namespace per [mcp-integration.md](../../reference/shared-core/mcp-integration.md). +- Inject server credentials from the keychain-backed secret seam (2.C) into the server `env` at + spawn time; construct the child environment **explicitly** (declared `env` + a minimal base), + never a blanket copy of the host environment; no secret in YAML, logs, or `--json` output. +- Validate every MCP `url` through the one shared SSRF primitive (same range-block as provider + base URLs and `http_request`, [ADR-0029](../../decisions/0029-tool-policy-hardening.md)); honor + the explicit local-endpoint opt-in. +- Enforce tool policy: MCP tools enter an agent's surface only under the narrow-only rules of + [ADR-0029](../../decisions/0029-tool-policy-hardening.md); honor a per-server `tools_allowlist`; + schema-validate every call against the server-reported JSON Schema before dispatch. + +**Acceptance:** a fixture agent declaring a stdio MCP server completes a real tool round-trip via +`relavium run` (and `relavium chat`), with the tool visible under its namespaced id; credentials +reach the server only via its spawn-time `env`; a private-range MCP `url` is rejected without the +explicit opt-in; the import-zone check confirms no SDK type leaks past the integration layer. + ## Milestones | In-phase milestone | Completed by | Global milestone | @@ -432,6 +480,7 @@ The interactive agent entry point on the CLI ([ADR-0024](../../decisions/0024-ag | CLI adopted as the engine regression harness | 2.D, 2.F, 2.K | **M3** | | Published, installable binary verified on all OSes | 2.L | — | | **Agent-first CLI** — `relavium chat` + session commands (resume / list / export / `agent run` / `gate list`): the **first user-facing `AgentSession` surface**, a committed build-phase-2 deliverable (off the M3 critical path, but a phase exit item — the agent-first headline is demonstrable here) | 2.M, 2.N, 2.O, 2.P, 2.Q | — | +| **MCP client live** — a fixture agent completes a real stdio MCP tool round-trip behind the `ToolRegistry`, per [ADR-0034](../../decisions/0034-mcp-client-sdk-dependency.md) (off the M3 critical path) | 2.R | — | ## Dependencies diff --git a/docs/roadmap/phases/phase-3-desktop.md b/docs/roadmap/phases/phase-3-desktop.md index e6e34d0e..620eb6f5 100644 --- a/docs/roadmap/phases/phase-3-desktop.md +++ b/docs/roadmap/phases/phase-3-desktop.md @@ -491,7 +491,10 @@ Git-native interoperability and a fast first-run experience. `.relavium/` files grouped by tag with last-run status; New / Import / Duplicate / Delete. - Bundle 3–5 curated starter templates at install, surfaced in the Dashboard - gallery and the onboarding flow. + gallery and the onboarding flow. Candidate set (settled when this workstream + starts): a code-review pipeline, a changelog/release-notes generator, and a + **research pipeline** (plan → search → extract → synthesize as an explicit DAG — + the template that shows off fan-out, a condition, and a human gate together). **Acceptance:** importing a hand-authored, git-committed workflow, editing it on the canvas, and exporting produces a diff limited to the intended edit; a starter diff --git a/docs/roadmap/phases/phase-5-managed-inference.md b/docs/roadmap/phases/phase-5-managed-inference.md index 5d0c1938..4a50f793 100644 --- a/docs/roadmap/phases/phase-5-managed-inference.md +++ b/docs/roadmap/phases/phase-5-managed-inference.md @@ -213,6 +213,11 @@ local, only egress is proxied" behind the seam. - Enforce the seam boundary: the gateway is the only path that touches Relavium's keys; surfaces and `@relavium/core` never see a managed key (mirrors the BYOK secret rule, [local-first-and-security.md](../../architecture/local-first-and-security.md)). +- Gateway health/error reporting is **secret-free by classification**: an upstream or + probe failure maps to a closed category + actionable hint — never a raw exception + string or a URL that may carry credentials — and multi-provider health checks run + under a deadline hierarchy (per-probe, per-provider, overall ceiling) so one hung + upstream cannot stall the health surface. **Acceptance:** a workflow runs end-to-end in `managed` mode with the engine still local and only LLM egress proxied; the `ManagedGatewayProvider` returns the @@ -231,7 +236,11 @@ Where Relavium's own provider keys live and how the gateway draws from them safe client-facing payloads (extends the never-reinvent-security-primitives rule). - Implement **per-provider key pools**: multiple keys per provider to spread org rate limits, with selection, **zero-downtime rotation**, and a **429-cooldown** that - parks a saturated key and advances to the next. + parks a saturated key and advances to the next. Pool retry discipline mirrors the + seam's fallback rules ([1.K](phase-1-engine-and-llm.md)): transient errors back off with jitter, while an + **auth-class failure on a pooled key is never blindly retried** — it parks the key + for investigation (possible revocation) and advances, since a deterministic auth + failure only repeats. - Add **cross-provider fallback** for the managed lane (consistent with the BYOK `FallbackChain` policy, but over Relavium's pooled keys) so a provider outage degrades gracefully. @@ -335,6 +344,10 @@ analysis) ([ADR-0012](../../decisions/0012-managed-inference-dual-mode.md)). managed mode never repeats the Cursor/Copilot "silent devaluation" backlash. - Keep routing a **managed-mode policy only**; BYOK mode honors the agent's model selection exactly (no managed routing crosses into BYOK). +- Evaluate a **dynamic model catalog for the managed lane** (a remote, versioned + price/limit/capability dataset refreshed out-of-band) so routing and pricing updates + do not require a client release; BYOK keeps the in-code pricing table as its source + of truth either way. **Acceptance:** a managed workflow without an explicit frontier model runs on the cheap default lane; prompt caching reduces measured token COGS on a cache-friendly diff --git a/docs/standards/code-style-typescript.md b/docs/standards/code-style-typescript.md index c4e27ecf..540f8e12 100644 --- a/docs/standards/code-style-typescript.md +++ b/docs/standards/code-style-typescript.md @@ -52,6 +52,11 @@ This is the load-bearing rule of the in-house multi-LLM decision importing a provider SDK outside the adapter folder. Adding a vendor dependency to the core path requires an [ADR](../decisions/README.md) and a [code review](code-review.md) sign-off. +- The fence has a **size half**, CI-enforced: each engine package's runtime `dependencies` + must stay within an explicit, reviewed allowlist (`tools/engine-deps/check.mjs`, hard-zero + — growing the list happens only together with the dependency's ADR). The same + import-zone technique extends to `packages/ui`'s internal layer boundaries when that + package lands in build phase 3. - The same discipline applies to other framework seams (DB driver, keychain, IPC): the vendor lives behind a Relavium interface, and only that package imports it. diff --git a/docs/standards/documentation-style.md b/docs/standards/documentation-style.md index 05251425..00314f18 100644 --- a/docs/standards/documentation-style.md +++ b/docs/standards/documentation-style.md @@ -50,7 +50,7 @@ The H1 is the document's canonical title and should match the intent of the file `local-dev-setup.md`, `multi-llm-providers.md`. - ADRs are zero-padded four digits plus a slug: `0001-tauri-v2-over-electron.md`. - Dated analysis docs suffix an ISO date: - `competitive-landscape-2026-06-03.md`. + `managed-inference-business-model-2026-06-03.md`. - Review records use a full ISO timestamp slug (see [reviews/README.md](../reviews/README.md)): `YYYY-MM-DDTHH-MM-SS--review.md`. diff --git a/docs/standards/error-handling.md b/docs/standards/error-handling.md index 75a3c1c5..d7c900da 100644 --- a/docs/standards/error-handling.md +++ b/docs/standards/error-handling.md @@ -72,6 +72,16 @@ We distinguish the two and never leak one as the other: - Errors surfaced through the run-event stream use the canonical `node:failed` and `run:failed` events (see the [SSE event schema](../reference/contracts/sse-event-schema.md)); they carry a user-safe message plus an internal correlation id, not a raw exception. +- **Resource/limit codes are fatal-without-user-action, never silent.** `budget_exceeded`, + `run_timeout`, and `turn_limit` (a **hard** agent/session turn/round cap — distinct from + the `[chat].max_messages` history-**trim** threshold of + [config-spec.md](../reference/contracts/config-spec.md), which continues the session and + emits no error) end the work with a typed event carrying that code — the engine never + loops past a cap or quietly stops under one. They are not retryable by policy: + continuing past a limit is an explicit user decision (raise the cap, resume the + session), not something a runner retries into. Note `budget_exceeded` is the + **fail-path** code only: ADR-0028's `on_exceed: warn` / `pause_for_approval` branches + emit `budget:warning` / `budget:paused` events and do not use this code. ## Validation at boundaries diff --git a/docs/standards/security-review.md b/docs/standards/security-review.md index 4e264f39..b96371d5 100644 --- a/docs/standards/security-review.md +++ b/docs/standards/security-review.md @@ -151,6 +151,12 @@ unless the user has explicitly opted into a local endpoint) applies to all three **engine** (never an adapter), through this **same** range-primitive — no second parser. It ships **feature-flag-OFF** until the one shared primitive lands. *(Not yet built; recorded here so it binds to the same primitive when it does.)* +- **The check and the connect must see the same address (no TOCTOU).** The primitive resolves the + hostname, validates **every** resolved IP against the range-block, and then **pins the connection + to a validated IP** (connect-by-validated-IP / a lookup-pinned HTTP agent). Validating one + resolution and letting the HTTP client re-resolve at connect time is the DNS-rebinding window — + an attacker-controlled DNS answer can pass the check and then point the actual connection at a + private address. Redirects re-run the full resolve-validate-pin cycle per hop. - All provider calls are HTTPS; we do not disable TLS verification. *(Per-host/per-provider TLS granularity is a deferred draft-proposal, not a current rule — see [deferred-tasks.md](../roadmap/deferred-tasks.md); the global never-disable stance holds until a private-CA self-hosted consumer needs an opt-IN behind a @@ -246,6 +252,18 @@ never hand-roll a security primitive; rule #2: a new runtime dependency needs an text in a fetched document or prior tool result cannot silently escalate privilege. - We do not concatenate untrusted content into a position where it can override the system prompt; the `system` field is set by Relavium, not by tool output. +- **Untrusted-content-as-data is a *structural* engine guarantee, not a per-call-site + discipline.** Every piece of content the model's caller does not author — a tool result + (`run_command` stdout, `read_file` contents, an `http_request` body), an MCP server + response, fetched media text — enters message assembly through a **typed untrusted + boundary** (a branded wrapper / taint marker on the engine side, the same compile-time + technique as the ADR-0029(c) secret taint), and the assembly layer can place such a value + **only** in a data position (`user`/`tool` content), **never** in `system` and never + string-concatenated into an instruction template. The reason it must be structural: with + N tool call-sites, "remember to wrap it" fails open at exactly one forgotten site — the + type boundary makes the unsafe path unrepresentable. This binds the `AgentRunner` / + `ToolRegistry` / `AgentSession` implementations (1.O / 1.T / 1.V); it is an engine-layer + rule and changes **no** seam shape — `LlmMessage` stays as the seam defines it. ## Never hand-roll crypto diff --git a/docs/uvp.md b/docs/uvp.md index 525e917a..1db83360 100644 --- a/docs/uvp.md +++ b/docs/uvp.md @@ -1,7 +1,7 @@ # Unique Value Proposition - **Status**: Accepted -- **Related**: [vision.md](vision.md), [product-constraints.md](product-constraints.md), [analysis/competitive-landscape-2026-06-05.md](analysis/competitive-landscape-2026-06-05.md) +- **Related**: [vision.md](vision.md), [product-constraints.md](product-constraints.md), [analysis/README.md](analysis/README.md) ## Statement @@ -62,8 +62,9 @@ both. The five properties that, taken together, no one else offers: ## Competitive Framing -A condensed read; the full matrix with per-competitor strengths and gaps lives in -[analysis/competitive-landscape-2026-06-05.md](analysis/competitive-landscape-2026-06-05.md). +A condensed read of the per-competitor strengths and the gaps Relavium fills — this table is +the surviving canonical competitive framing. *(The dated competitive-landscape analyses it +distilled were removed on 2026-06-10.)* | Competitor | Their strength | The gap Relavium fills | |-----------|----------------|------------------------| diff --git a/eslint.config.mjs b/eslint.config.mjs index 5e03b4b4..df4a1faa 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -128,6 +128,16 @@ export default tseslint.config( 'no-restricted-syntax': seamSyntaxRules, }, }, + { + // The repo's Node tooling scripts (the seam-fence assert + the engine-deps guard). + // This is the "globals.node for Node tooling" slot promised above, granted narrowly + // and inline (no `globals` package) — only what the scripts actually touch. ESLint 9 + // flat config ignores `/* eslint-env */` comments, so the grant must live here. + files: ['tools/**/*.{js,mjs,cjs}'], + languageOptions: { + globals: { console: 'readonly', process: 'readonly' }, + }, + }, { // The seam's ONE legal zone: provider SDKs are imported only inside the adapters, // which translate vendor shapes to Relavium/Zod seam types (ADR-0011). Every seam diff --git a/package.json b/package.json index 6c80936f..c7fa8f2b 100644 --- a/package.json +++ b/package.json @@ -16,8 +16,9 @@ "typecheck:tools": "tsc -p tsconfig.tools.json", "test": "turbo run test", "coverage": "vitest run --coverage", - "ci": "turbo run lint typecheck test && pnpm typecheck:tools && turbo run build format:check && pnpm lint:fence-check", + "ci": "turbo run lint typecheck test && pnpm typecheck:tools && turbo run build format:check && pnpm lint:fence-check && pnpm lint:engine-deps", "lint:fence-check": "node tools/lint-fixtures/assert-fence.mjs", + "lint:engine-deps": "node tools/engine-deps/check.mjs", "format": "prettier --write .", "format:check": "prettier --check ." }, @@ -31,5 +32,11 @@ "typescript": "catalog:", "typescript-eslint": "catalog:", "vitest": "catalog:" + }, + "pnpm": { + "onlyBuiltDependencies": [ + "better-sqlite3", + "esbuild" + ] } } diff --git a/packages/shared/src/constants.ts b/packages/shared/src/constants.ts index ffd0d97b..41747214 100644 --- a/packages/shared/src/constants.ts +++ b/packages/shared/src/constants.ts @@ -63,6 +63,11 @@ export type SessionEventType = (typeof SESSION_EVENT_TYPES)[number]; * `node:failed` / `run:failed` / `session:turn_completed` carry one of these as * `error.code` (never a free string), so a surface can branch on cause. The * retryable/fatal mapping is owned by docs/standards/error-handling.md. + * `turn_limit` is the limit-family code for a HARD agent/session turn/round cap + * (the knob is settled at 1.V): distinct from `run_timeout`/`budget_exceeded` so a + * capped conversation surfaces its own cause instead of a silent stop. Not to be + * confused with `[chat].max_messages`, which is a history-trim threshold + * (config-spec.md) — trimming continues the session and emits no error. */ export const ERROR_CODES = [ 'validation', @@ -73,6 +78,7 @@ export const ERROR_CODES = [ 'tool_failed', 'budget_exceeded', 'run_timeout', + 'turn_limit', 'cancelled', 'sandbox_error', 'internal', diff --git a/packages/shared/src/run-event.test.ts b/packages/shared/src/run-event.test.ts index 693d7fab..5f0ec799 100644 --- a/packages/shared/src/run-event.test.ts +++ b/packages/shared/src/run-event.test.ts @@ -398,6 +398,18 @@ describe('SessionEvent union — the agent-first namespace', () => { ).toBe(true); }); + it('pins turn_limit as the ErrorCode for a capped conversation (never a silent stop)', () => { + // A session hitting a hard turn/round cap must be expressible as its own cause, + // fatal-without-user-action — not folded into run_timeout/budget_exceeded. (Distinct + // from [chat].max_messages, which is a history-trim threshold, not a stop.) + expect( + SessionEventSchema.safeParse({ + ...validSession['session:turn_completed'], + error: { code: 'turn_limit', message: 'session reached its turn cap', retryable: false }, + }).success, + ).toBe(true); + }); + it('rejects a session:started selection whose startLine exceeds endLine', () => { const withSelection = (sel: { file: string; startLine: number; endLine: number }) => SessionEventSchema.safeParse({ diff --git a/tools/engine-deps/check.mjs b/tools/engine-deps/check.mjs new file mode 100644 index 00000000..87d45559 --- /dev/null +++ b/tools/engine-deps/check.mjs @@ -0,0 +1,74 @@ +/** + * Engine dependency-allowlist guard — the SIZE half of the seam fence. + * + * The import-syntax fence (tools/lint-fixtures/assert-fence.mjs) polices WHAT crosses the + * `@relavium/llm` seam; this guard polices HOW BIG the engine packages' runtime dependency + * graphs may get. Each engine package's `dependencies` must be a subset of an explicit, + * reviewed allowlist — hard-zero, no warn-only mode, no allowlist growth inside a feature + * PR. Adding a runtime dependency to an engine package means editing THIS file in the same + * change as the ADR that justifies the dependency (architectural-principles.md §9, + * ADR-0003 zero-platform-imports, ADR-0011 seam discipline). + * + * Checked: `dependencies` + `optionalDependencies` (both install and ship — an optional + * runtime dep must not bypass the gate). Not checked: `devDependencies` (never ship), + * `peerDependencies` (declared, not shipped; the strict CI peer gate polices drift), and + * `bundledDependencies` (unused in this repo). `@relavium/db` is deliberately NOT an + * engine package — it is host-bound by design (better-sqlite3). When `packages/ui` lands + * (build phase 3), its internal layer boundaries get the same treatment via an ESLint + * import-zone config rather than this list. + * + * Exits non-zero so CI fails loudly. Run from the repo root: + * node tools/engine-deps/check.mjs + */ +import { readFileSync, existsSync } from 'node:fs'; +import { join } from 'node:path'; + +/** + * package path → allowed RUNTIME dependency names. A missing package is skipped with a + * note (packages/core is scaffolded at 1.L); an existing package with a dep outside its + * list fails the build. + */ +const ENGINE_ALLOWLISTS = { + // The contract package: Zod is its ONLY runtime dependency (tech-stack.md). + 'packages/shared': ['zod'], + // The seam package: the contract + Zod + the official provider SDKs the adapters wrap + // (ADR-0011 — SDK types stay confined to packages/llm/src/adapters/*). + 'packages/llm': ['@relavium/shared', 'zod', '@anthropic-ai/sdk', 'openai', '@google/genai'], + // The engine: Relavium packages only, plus the ADR-0027 sandbox runtime once the 1.AB + // perf spike pins the package (added here, with the catalog pin, in that change). + // @relavium/db is deliberately ABSENT: the engine runs in the Tauri WebView with zero + // platform imports (CLAUDE.md rule 5), and @relavium/db pulls the native better-sqlite3 + // runtime. Core may use its TYPES via a devDependency (the Checkpointer interface / + // Drizzle schema types); the store itself is injected by the host surface (1.R). + 'packages/core': ['@relavium/shared', '@relavium/llm', 'zod'], +}; + +let failed = false; + +for (const [pkgDir, allowed] of Object.entries(ENGINE_ALLOWLISTS)) { + const manifestPath = join(pkgDir, 'package.json'); + if (!existsSync(manifestPath)) { + console.log(`- ${pkgDir}: not scaffolded yet — skipped (allowlist already reserved).`); + continue; + } + const manifest = JSON.parse(readFileSync(manifestPath, 'utf8')); + const deps = [ + ...Object.keys(manifest.dependencies ?? {}), + ...Object.keys(manifest.optionalDependencies ?? {}), + ]; + const offenders = deps.filter((d) => !allowed.includes(d)); + if (offenders.length > 0) { + failed = true; + console.error( + `✗ ${pkgDir}: runtime dependencies outside the engine allowlist: ${offenders.join(', ')}\n` + + ` Allowed: ${allowed.join(', ')}\n` + + ' A new engine runtime dependency needs an ADR (architectural-principles.md §9) and a\n' + + ' deliberate edit to tools/engine-deps/check.mjs in the same change.', + ); + } else { + console.log(`✓ ${pkgDir}: ${deps.length} runtime dep(s), all on the allowlist.`); + } +} + +if (failed) process.exit(1); +console.log('✓ Engine dependency graphs are within their reviewed allowlists.');