Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/decisions/0011-internal-llm-abstraction.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

- **Status**: Accepted
- **Date**: 2026-06-03
- **Related**: [0004-vercel-ai-sdk-multi-llm.md](0004-vercel-ai-sdk-multi-llm.md) (supersedes), [0003-pure-ts-engine-not-langgraph-python.md](0003-pure-ts-engine-not-langgraph-python.md), [0006-os-keychain-for-api-keys.md](0006-os-keychain-for-api-keys.md), [0018-desktop-execution-and-rust-egress.md](0018-desktop-execution-and-rust-egress.md) (per-host egress + key handling), [0024-agent-first-entry-point-agentsession.md](0024-agent-first-entry-point-agentsession.md) (seam reused by chat-mode agents), [tech-stack.md](../tech-stack.md)
- **Related**: [0004-vercel-ai-sdk-multi-llm.md](0004-vercel-ai-sdk-multi-llm.md) (supersedes), [0003-pure-ts-engine-not-langgraph-python.md](0003-pure-ts-engine-not-langgraph-python.md), [0006-os-keychain-for-api-keys.md](0006-os-keychain-for-api-keys.md), [0018-desktop-execution-and-rust-egress.md](0018-desktop-execution-and-rust-egress.md) (per-host egress + key handling), [0024-agent-first-entry-point-agentsession.md](0024-agent-first-entry-point-agentsession.md) (seam reused by chat-mode agents), [0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md](0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md) (amends the seam shape), [tech-stack.md](../tech-stack.md)

## Context

Expand Down Expand Up @@ -33,6 +33,13 @@ The seam is the immovable contract; the adapter implementation behind it is deli
> agents call providers through the identical contract, so no vendor SDK type crosses the seam for
> sessions either. The seam's types and contract are unchanged.

> Amended 2026-06-07: the seam *shape* grows by three additive features — a reasoning channel,
> `LlmRequest.responseFormat`, and `providerExecuted` — per
> [ADR-0030](0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md). The
> ADR-0011 decision is unchanged (an internal, provider-agnostic seam in Relavium/Zod types, no
> vendor SDK type crossing it); only the seam's shape is extended, at the M1 freeze boundary before
> any consumer narrows on it.

Considered options:

1. **Internal abstraction over official provider SDKs (`@relavium/llm`)** — owned seam, thin per-provider adapters, no framework. *Chosen.*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# ADR-0030: `@relavium/llm` seam-shape amendment — reasoning channel, responseFormat, providerExecuted

- **Status**: Accepted
- **Date**: 2026-06-07

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Use a non-future ADR acceptance date.

Line 4 records Accepted with 2026-06-07, which is in the future relative to this review date (2026-06-06). Use the actual acceptance date at merge time to keep ADR history trustworthy.

✏️ Suggested edit
-- **Date**: 2026-06-07
+- **Date**: 2026-06-06
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- **Date**: 2026-06-07
- **Date**: 2026-06-06
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@docs/decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md`
at line 4, The ADR's Accepted date is set to a future date ("Accepted:
2026-06-07"); update the Accepted date entry in the document (the line
containing "Accepted") to the actual acceptance/merge date (today's date or the
commit/merge date) so the ADR history is not in the future.

- **Related**: [0011-internal-llm-abstraction.md](0011-internal-llm-abstraction.md) (the seam ADR this amends), [0029-tool-policy-hardening.md](0029-tool-policy-hardening.md) (the same "tighten the contract before it has consumers" move; the tool allowlist that makes `providerExecuted` matter), [0006-os-keychain-for-api-keys.md](0006-os-keychain-for-api-keys.md), [../reference/shared-core/llm-provider-seam.md](../reference/shared-core/llm-provider-seam.md) (the seam's one canonical home), [../standards/error-handling.md](../standards/error-handling.md)

## Context

The `@relavium/llm` seam — the request/result/stream/usage/content shapes in
[`packages/llm/src/types.ts`](../../packages/llm/src/types.ts) and
[`packages/shared/src/content.ts`](../../packages/shared/src/content.ts) — is the immovable contract
of [ADR-0011](0011-internal-llm-abstraction.md). The three Phase-1 adapters (Anthropic, the shared
OpenAI/DeepSeek adapter, Gemini) now pass the shared conformance suite, so the seam is at the **M1
freeze boundary**. Crucially, **no consumer beyond the adapters exists yet** — the `FallbackChain`
(1.K), the engine (`AgentRunner`/`WorkflowEngine`, 1.O/1.N), the session layer (1.V–1.Z) and the
surfaces are all unbuilt. This is the same situation [ADR-0029](0029-tool-policy-hardening.md) acted
on: a contract change is nearly free before it has consumers and a breaking change after.

The ADR-0011 seam rule (recorded in [llm-provider-seam.md](../reference/shared-core/llm-provider-seam.md))
distinguishes two kinds of change: **adding a provider id is additive** (no ADR — it alters no
existing type); **changing the seam *shape*** — the request/result/stream/usage/content types — is a
real amendment that requires an ADR. Three shape gaps were assessed against the actual pinned SDKs
and the already-merged contracts. Each is a genuine **cross-provider** concern the current seam
cannot express, not a single-provider quirk (those go through `providerOptions` + a capability flag):

1. **Reasoning is advertised but undeliverable.** `CapabilityFlags.reasoning` exists and is set
`true` for Anthropic/Gemini/DeepSeek, yet the seam has **no channel** to carry the reasoning it
promises, so all three adapters silently drop it today. The three providers expose reasoning in
three incompatible native shapes (Anthropic first-class `thinking`/`signature` block-deltas;
Gemini `thought`-flagged parts with a base64 `thoughtSignature`; DeepSeek/Kimi an untyped
`reasoning_content` field over the OpenAI-compatible wire). Reasoning text can only reach a
consumer today as a vendor-shaped blob on `LlmResult.raw` — re-introducing exactly the
vendor-coupling ADR-0011 exists to prevent, and making the ephemeral-signature guarantee
unenforceable at the seam.

2. **`responseFormat` is the missing mechanism for an already-merged contract.** `output_schema` is
already shipped on `agent`/`transform` nodes ([`packages/shared/src/node.ts`](../../packages/shared/src/node.ts)
`OutputSchemaSchema`), but the LLM seam has no way to ask a model for structured output, so the
node feature is unimplementable until the seam can carry it.

3. **`providerExecuted` distinguishes server-run tools from engine-run tools.** Providers increasingly
run tools on their own side (Anthropic `web_search`/`code_execution`, Gemini `googleSearch`,
OpenAI Responses built-ins). Without a discriminator, the engine `ToolDispatcher` (1.T, allowlist
[ADR-0029](0029-tool-policy-hardening.md)) cannot tell "I must run this" from "the provider already
ran this" — risking double-execution and mis-applying the engine's tool-permission model to a call
the engine never makes.

An adversarial assessment refined the urgency: only a change that adds a **member to a discriminated
union** (`StreamChunk` / `ContentPart`) is genuinely breaking-to-add-later, because every consumer's
exhaustive `switch` + `never`-exhaustiveness check breaks at compile time. Adding an **optional
field** (to `LlmRequest`, or to an existing union arm) is backwards-compatible and could in principle
be deferred to the consumer that needs it. We nonetheless settle all three **now**, in one amendment,
because (a) the reasoning channel and the provider-executed stream chunk *are* union-member additions
that must land before consumers narrow on the frozen shape, and (b) doing the one ADR + one seam edit
once — while the only consumers are the three adapters we are already editing — is cheaper and less
error-prone than three separate future amendments, each re-touching every adapter.

## Decision

**We will extend the `@relavium/llm` seam shape with three additive features, recorded as an
amendment to (not a supersession of) [ADR-0011](0011-internal-llm-abstraction.md).** ADR-0011's
decision — an internal, provider-agnostic seam in Relavium/Zod types with no vendor SDK type crossing
it — is unchanged; this only grows the seam's shape. The canonical types live in
[llm-provider-seam.md](../reference/shared-core/llm-provider-seam.md); the additions are:

**1. Reasoning channel (additive).**
- `ContentPart` gains a `reasoning` arm: `{ type: 'reasoning', text, signature?, redacted? }`.
- `StreamChunk` gains `reasoning_start` / `reasoning_delta` / `reasoning_end` (mirroring the
`tool_call_*` triad; `id` correlates deltas to the terminating `reasoning_end`, which carries the
optional `signature`/`redacted`).
- `Usage` gains an optional `reasoningTokens` — **observability only**; the cost math is unchanged
(every provider counts reasoning inside `outputTokens` for billing, so `CostTracker` keeps billing
`outputTokens` whole — `reasoningTokens` is never an additional cost line).

**2. `LlmRequest.responseFormat` (additive, optional).** A discriminated union
`{ type: 'text' } | { type: 'json', schema, name?, strict? }` (`schema` is the one canonical
`JSONSchema7`). Each adapter lowers `json` to the provider's **native** structured-output mode where
one exists (OpenAI `response_format: json_schema`; Gemini `responseJsonSchema` + JSON mime type;
Anthropic `output_config`/forced tool) — native-vs-forced-tool is a per-adapter implementation detail,
not a seam concern. We deliberately **drop** the opencode `{ type: 'tool' }` variant: "force a
specific tool" is already expressed by `toolChoice: { name }`, so a third variant would be a
redundant second way to force a tool.

**3. `providerExecuted` (additive).**
- `ContentPart` `tool_call` and `tool_result` gain an optional `providerExecuted?: boolean`.
- `StreamChunk` gains a `tool_result` arm carrying a provider-executed result
(`{ type: 'tool_result', id, name, result, isError?, providerExecuted: true }`) — distinct from the
engine-executed `tool_call_start/delta/end` triad. A `providerExecuted === true` call is **skipped**
by the engine `ToolDispatcher` (1.T): the engine neither runs it nor applies its allowlist to it;
it only records/forwards it.

**Alternatives weighed.** *(i)* `providerOptions` + the capability flag + `raw` (rejected:
`providerOptions` is request-inbound only and cannot carry reasoning/results back; `raw` is a
vendor-shaped `unknown`, so consumers would pattern-match vendor shapes — the exact coupling ADR-0011
forbids). *(ii)* Defer all three until their consumers exist (rejected for the union-member additions —
breaking-to-add-later; and bundling the optional fields into the same one-time amendment is cheaper
than three future re-touches of every adapter). *(iii)* A full ~16-member opencode-style event union
with `step-*`, media, audio, citations (rejected: speculative; those are deferrable optional/feature
additions to add with their capability when demanded — this amendment stays minimal).

**Guardrails (binding).**
- **Reasoning is ephemeral.** A provider-signed reasoning block (`signature`) is a same-provider,
same-turn continuity token. It is **never persisted** to a session, **never replayed across a
provider boundary** (the `FallbackChain`, 1.K, strips reasoning parts when failing over to another
provider), and **never written to a run event or log**. The engine does not interpret it; only the
originating adapter feeds it back. `signature` is an opaque `string` (no `Buffer`/Node type — the
seam stays platform-free, `tsconfig.seam.json` `types: []`).
- **No vendor type crosses the seam.** Each provider's native reasoning/structured-output/server-tool
shape is normalized to these canonical types inside the adapter; `responseFormat` carries one
canonical `JSONSchema7`.
- **`providerExecuted` and the engine tool-security model stay disjoint** — the dispatcher applies its
allowlist only to engine-executed calls; a provider-executed call is never run by the engine.
- **Usage stays NET** ([cost-tracker](../../packages/llm/src/cost-tracker.ts)); `reasoningTokens` is
an extra disjoint observability count, not a new billable class.

Per-workstream, this lands the **shape** plus the **reasoning + structured-output behavior** wired in
every adapter that supports it (Anthropic/Gemini/DeepSeek reasoning; all three structured output;
OpenAI chat emits no reasoning) with conformance scenarios. `providerExecuted` lands as **shape only**
(no Phase-1 server-tool support is common-path), reserved so 1.T/1.O are born handling it.

## Consequences

### Positive

- The seam is extended at its cheapest possible moment — three adapters, zero downstream consumers —
avoiding a future breaking discriminated-union change + superseding ADR + consumer rework.
- `CapabilityFlags.reasoning` stops being a dangling promise; reasoning reaches the UI/session as a
canonical, vendor-neutral channel with an enforceable ephemerality guarantee.
- `output_schema` becomes implementable; the engine can request structured output through one
canonical field, each adapter using the best native mechanism.
- The engine tool loop is born knowing the difference between a call it must run and one the provider
already ran — no double-execution, no mis-scoped permission.

### Negative

- A larger seam surface: more `StreamChunk`/`ContentPart` arms for every future consumer to handle
(mitigated — the additions are minimal and each carries an exhaustiveness obligation that catches
omissions at compile time).
- `providerExecuted` ships as reserved shape with no Phase-1 emitter, i.e. shape ahead of behavior
(accepted deliberately: the union-member reservation is the breaking-to-add-later part).
- The reasoning ephemerality guarantee is a standing correctness/data-handling obligation every later
consumer (fallback, session persistence, run-event logging) must uphold — called out as design
notes on 1.K and 1.Z.
1 change: 1 addition & 0 deletions docs/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ flowchart TD
| 0027 | [Expression sandbox for `condition` / `transform` / `merge_fn`](0027-expression-sandbox.md) | Accepted | 2026-06-05 |
| 0028 | [Workflow resource governance — pre-egress budget, run timeout, concurrency cap](0028-workflow-resource-governance.md) | Accepted | 2026-06-05 |
| 0029 | [Tool-policy hardening — command match, tool narrowing, secret interpolation, SSRF](0029-tool-policy-hardening.md) | Accepted | 2026-06-05 |
| 0030 | [`@relavium/llm` seam-shape amendment — reasoning channel, responseFormat, providerExecuted](0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md) | Accepted | 2026-06-07 |
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## Creating a new ADR

Expand Down
54 changes: 51 additions & 3 deletions docs/reference/shared-core/llm-provider-seam.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,19 +39,28 @@ interface LlmRequest {
temperature?: number;
maxTokens?: number; // REQUIRED downstream for Anthropic; we default it
stopSequences?: string[];
responseFormat?: ResponseFormat; // structured-output request (ADR-0030)
signal?: AbortSignal; // cancellation; host-injected transport (desktop aborts the Rust llm_stream egress, ADR-0018)
providerOptions?: Record<string, unknown>; // typed escape hatch (caching, reasoning, etc.)
}

// Structured-output contract (ADR-0030). Each adapter lowers `json` to the provider's native mode
// (OpenAI json_schema; Gemini responseJsonSchema; Anthropic output_config; DeepSeek json_object — no
// schema enforcement, so its fidelity is "parseable JSON", not schema-validated).
type ResponseFormat =
| { type: 'text' }
| { type: 'json'; schema: JSONSchema7; name?: string; strict?: boolean };

interface LlmMessage {
role: 'user' | 'assistant' | 'tool';
content: ContentPart[]; // normalized parts, not raw strings
}

type ContentPart =
| { type: 'text'; text: string }
| { type: 'tool_call'; id: string; name: string; args: unknown } // assistant -> wants tool
| { type: 'tool_result'; toolCallId: string; result: unknown; isError?: boolean };
| { type: 'reasoning'; text: string; signature?: string; redacted?: boolean } // ADR-0030; signature is ephemeral
| { type: 'tool_call'; id: string; name: string; args: unknown; providerExecuted?: boolean } // assistant -> wants tool
| { type: 'tool_result'; toolCallId: string; result: unknown; isError?: boolean; providerExecuted?: boolean };

interface ToolDef {
name: string;
Expand All @@ -74,15 +83,20 @@ interface Usage {
outputTokens: number;
cacheReadTokens?: number; // Anthropic/DeepSeek expose; others undefined
cacheWriteTokens?: number;
reasoningTokens?: number; // ADR-0030 — OBSERVABILITY only; a subset of outputTokens (≤), never billed separately
costMicrocents?: number; // integer micro-cents (canonical unit defined below); computed by a pricing table keyed on canonical model id
}

// Normalized streaming — one discriminated union for ALL providers
type StreamChunk =
| { type: 'text_delta'; text: string }
| { type: 'reasoning_start'; id: string } // ADR-0030 — reasoning channel
| { type: 'reasoning_delta'; id: string; text: string }
| { type: 'reasoning_end'; id: string; signature?: string; redacted?: boolean } // signature/redacted both surfaced on the stream
| { type: 'tool_call_start'; id: string; name: string }
| { type: 'tool_call_delta'; id: string; argsJsonDelta: string } // partial JSON
| { type: 'tool_call_delta'; id: string; argsJsonDelta: string } // partial JSON; count/timing is provider-dependent — accumulate, parse at tool_call_end
| { type: 'tool_call_end'; id: string }
| { type: 'tool_result'; id: string; name: string; result: unknown; isError?: boolean; providerExecuted: true } // ADR-0030 — provider-run tool; engine records, never runs
| { type: 'stop'; stopReason: StopReason; usage: Usage }
| { type: 'error'; error: LlmError };

Expand Down Expand Up @@ -173,6 +187,40 @@ would require a real (superseding) ADR is changing the seam shape itself: the
request/result/stream types, the normalization rules, or the `LlmError` contract
above.

### Seam-shape amendments ([ADR-0030](../../decisions/0030-llm-seam-shape-amendment-reasoning-response-format-provider-executed.md))

Three cross-provider shape additions were made under ADR-0030 (a real amendment to
ADR-0011, decided before the seam froze at M1, while the only consumers were the
adapters):

- **Reasoning channel.** `ContentPart` gains a `reasoning` arm
(`{ type: 'reasoning', text, signature?, redacted? }`); `StreamChunk` gains
`reasoning_start` / `reasoning_delta` / `reasoning_end` (mirroring the
`tool_call_*` triad; `reasoning_end` carries the optional `signature` and
`redacted` flag — both surfaced on the streaming path, symmetric with the
non-streaming `reasoning` content part); `Usage` gains an optional
`reasoningTokens` (**observability only** — already inside `outputTokens` for
billing on Anthropic/OpenAI; on Gemini, thinking tokens are billed *separately*
from candidates, so the adapter sums both into `outputTokens` and surfaces the
thinking subset as `reasoningTokens`). **Reasoning is ephemeral:** a provider-signed
`signature` is never persisted to a session, never replayed across a provider
boundary on fallback, and never written to a run event or log — the engine does
not interpret it; only the originating adapter feeds it back (a same-provider,
same-turn obligation owned by the 1.K `FallbackChain` strip-on-failover, not yet
exercised — no consumer beyond the adapters exists).
- **`responseFormat`** on `LlmRequest` — `{ type: 'text' } | { type: 'json', schema, name?, strict? }`,
one canonical JSON-Schema each adapter lowers to the provider's native
structured-output mode (OpenAI `response_format`, Gemini `responseJsonSchema`,
Anthropic `output_config`). This is the seam mechanism that realizes a node's
`output_schema`. (The opencode `{ type: 'tool' }` variant is deliberately not
adopted — `toolChoice: { name }` already forces a specific tool.)
- **`providerExecuted`** — an optional flag on `ContentPart` `tool_call`/`tool_result`
plus a provider-executed `tool_result` `StreamChunk` arm, distinguishing a tool
the **provider** ran on its own side (server-side/built-in) from one the engine
runs. The engine `ToolDispatcher` skips `providerExecuted` calls (no
double-execution, and the allowlist applies only to engine-run calls). Phase-1
adapters reserve the shape but emit no server-tool calls (off the common path).

## What must be normalized

The seam's value is entirely in the normalization the adapters perform. Each of
Expand Down
Loading
Loading