[5283] fix(agent): Preserve Pi history after cold resume

docs/design/agent-workflows/projects/session-workspace-materialization/README.md

@@ -0,0 +1,16 @@
+# Pi session resources and continuity
+
+This workspace plans a launch-safe fix for two failures caused by Pi's temporary per-run agent
+directory: skill paths become invalid after cold resume, and native transcripts can be deleted
+while Agenta still treats their session IDs as resumable. The immediate gate verifies native loads
+and falls back to the canonical transcript replay. The filesystem follow-up uses append-only skill
+snapshots and durable private transcript storage without deleting workspace files.
+
+## Files
+
+- `context.md` - Problem, goals, non-goals, and terminology.
+- `research.md` - Current Agenta behavior and Pi 0.80.6 discovery rules.
+- `design.md` - Proposed cwd ownership and materialization contract.
+- `plan.md` - Phased implementation sequence.
+- `qa.md` - Unit, integration, lifecycle, and security verification.
+- `status.md` - Current decisions, open questions, and next steps.

docs/design/agent-workflows/projects/session-workspace-materialization/context.md

@@ -0,0 +1,70 @@
+# Context
+
+## Problem
+
+Local Pi creates a random per-run `PI_CODING_AGENT_DIR` whenever a run has configured skills or a
+custom system prompt. Pi stores both injected configuration and native transcripts under this
+directory. Teardown deletes it after the turn.
+
+This produces two independent failures. Skill paths remembered by the model become invalid after a
+cold resume. More seriously, Agenta can retain a native session ID after deleting its transcript.
+`pi-acp` can then create a blank replacement transcript at the stale path, report a successful
+load, and cause the runner to send only the newest message instead of the canonical cold replay.
+
+Claude already receives configured skills in the session cwd under `.claude/skills`. That path
+shares the session workspace lifecycle. Its current copier merges files and does not remove stale
+content. Changing that behavior as part of an urgent Pi fix adds unnecessary launch risk.
+
+The design separates four lifecycles:
+
+1. user and agent data that should persist with the session;
+2. agent configuration rendered as harness-readable files;
+3. private native transcripts that should persist with the conversation;
+4. ephemeral runner transport files.
+
+## Goals
+
+1. Make uncertain native loads fall back to the canonical replay before launch.
+2. Never treat an inferred or requested native session ID as proof that history loaded.
+3. Put configured Pi skills in immutable snapshots at
+   `<cwd>/agents/skills/<skill-set-digest>/<name>`.
+4. Give Pi transcripts a private durable lifetime aligned with the Agenta conversation.
+5. Keep credentials, executable extensions, per-run settings, and relay files ephemeral.
+6. Avoid deletion-based reconciliation and leave Claude unchanged in the urgent work.
+7. State when the runner refreshes startup resources and when it recreates a harness.
+8. Preserve progressive disclosure: skill metadata is eager, while full skill content remains
+   on-demand.
+
+## Non-goals
+
+- Making every Pi project resource trusted.
+- Loading arbitrary `.pi/extensions`, `.pi/settings.json`, prompts, or themes from a mutable
+  workspace.
+- Changing the public agent configuration or `/run` wire shape.
+- Moving model credentials or OAuth files into the session mount.
+- Reloading instructions or skills inside an already-running harness without recreating it.
+- Making native-load correctness depend on warm reuse.
+- Garbage-collecting old skill snapshots during environment acquisition.
+- Changing Claude skill materialization in the urgent implementation.
+
+## Terms
+
+- **Session cwd:** The workspace passed to the harness as its working directory. It is durable when
+  a session mount is available and ephemeral for an ad hoc run.
+- **Agent directory:** Pi's private configuration directory selected by `PI_CODING_AGENT_DIR`.
+- **Canonical replay:** The complete Agenta transcript prepared in `plan.turnText` for a cold
+  harness session.
+- **Verified native load:** A load whose transcript exists, is readable, and identifies the
+  requested native session. Transport success alone is not verification.
+- **Literal `./agents/skills`:** The non-hidden `<cwd>/agents/skills` directory. Pi does not
+  discover it by convention; the runner supplies one exact snapshot as an explicit skill path.
+- **Project `.agents/skills`:** Pi's hidden conventional project directory. It is trust-gated and
+  is not used for runner-injected skills.
+- **Trust-gated:** Loaded only after the user trusts the project in Pi's consent flow. Pi stores
+  the decision in `<agent-dir>/trust.json` keyed by the cwd. Trust authorizes the project's own
+  in-cwd configuration: `.pi/settings.json`, executable `.pi/extensions`, `.pi/skills`,
+  `.agents/skills`, prompts, themes, and `SYSTEM.md`. RPC runs cannot prompt, so the default `ask`
+  policy leaves projects untrusted and loads none of these.
+- **Immutable skill snapshot:** A content-addressed directory that is written once and never
+  reconciled in place. Immutability is a convention the runner upholds, not something the
+  filesystem enforces.

docs/design/agent-workflows/projects/session-workspace-materialization/design.md

@@ -0,0 +1,163 @@
+# Design
+
+## Decision
+
+Fix correctness before storage. The runner may send only the newest message when native history was
+verified, not merely when an ACP request returned successfully. Any missing, corrupt, mismatched, or
+uncertain native session falls back to a new harness session with the canonical Agenta replay.
+
+After that gate is safe, separate resources by lifecycle:
+
+| Resource | Role | Lifecycle | Destination |
+| --- | --- | --- | --- |
+| `AGENTS.md` | declarative instructions | configuration | session cwd |
+| Pi skills | declarative capability packages | immutable configuration snapshot | `<cwd>/agents/skills/<digest>` |
+| Claude skills | current Claude project configuration | existing behavior | `<cwd>/.claude/skills` |
+| Pi native transcripts | private harness state | Agenta conversation | dedicated private session storage |
+| system prompt files and private settings | harness configuration | environment | temporary Pi agent directory |
+| credentials and custom models | secrets/private configuration | credential epoch | temporary Pi agent directory |
+| Agenta Pi extension | executable runtime infrastructure | runner build/environment | temporary Pi agent directory |
+| relay and tool files | transport scratch | turn/environment | ephemeral runner paths |
+
+The urgent implementation changes only native-load verification and replay selection. It does not
+move skills, add a reconciler, or alter Claude.
+
+## Gate 1: verified native load
+
+The current `loaded: boolean` has ambiguous meaning. Transport success and native-history success
+are different facts. The internal continuation result must carry an explicit history outcome:
+
+```text
+verified
+  transcript exists and is readable
+  requested native ID equals transcript header ID
+  adapter reports the same actual native ID
+
+unavailable
+  mapping or transcript is missing
+
+invalid
+  transcript is corrupt or its header ID differs
+
+unverified
+  the adapter cannot prove which history it loaded
+```
+
+Only `verified` permits last-message-only prompting. Every other outcome must:
+
+1. invalidate the stale continuity record;
+2. create a clean native session;
+3. send `plan.turnText`, the canonical cold replay;
+4. record the new native ID only after the replayed turn completes successfully.
+
+The sandbox-agent wrapper must not substitute `requestedSessionId` for a missing response ID and
+then present it as loaded identity. A requested identifier is routing input, not evidence.
+
+Verification evidence belongs in the layer that owns the native transcript mapping. For Pi, that
+is `pi-acp`. But `pi-acp` is an external prebuilt dependency pinned at 0.0.29, and its load
+response carries no identity evidence at all. The launch gate therefore must not depend on
+changing it. At launch, the runner applies the conservative rule alone: every cold Pi load is
+`unverified`, and every cold continuation replays the canonical transcript. That is the intended
+launch behavior, not a regression. It trades repeated replay tokens for guaranteed history.
+
+Verified loads return as evidence lands, from two acceptable sources in order of preference:
+
+1. `pi-acp` validates existence and header identity and returns the actual loaded ID, through an
+   upstream fix, a version bump, or a patch file like the existing sandbox-agent patch.
+2. The runner checks the mapped transcript itself. On local runs this is a pure file read of
+   information the runner already has: it shares the filesystem and `HOME` with Pi, knows the
+   agent directory it created, holds the expected native ID in its continuity store, and already
+   parses transcript headers in `pi-error.ts`. On Daytona the same check reuses the existing
+   in-sandbox exec and file channel but is new code on the read-back direction.
+
+Either source may upgrade an outcome to `verified`. Nothing relaxes the conservative rule.
+
+## Existing mapping migration
+
+Old `session-map.json` entries may already point to deleted or replaced files. The first
+post-change continuation must treat them as unverified, invalidate the pointer, and replay the
+canonical transcript. Migration must not fail the user turn merely because native state is stale.
+
+## Durable native transcript state
+
+The private transcript directory must live as long as the Agenta conversation. Do not make the
+whole `PI_CODING_AGENT_DIR` durable because it also contains credentials, settings, extensions,
+and system prompt files with different lifecycles.
+
+Pi has a supported seam for this: a session directory override, resolved from `--session-dir`,
+`PI_CODING_AGENT_SESSION_DIR`, or the `sessionDir` key in the agent directory's `settings.json`.
+The leading option is to write `sessionDir` into the per-run agent directory's `settings.json`,
+pointing at stable private per-conversation storage. The settings form is required rather than the
+env var because `pi-acp` resolves its scan directory from the same settings key and ignores the
+env var. The settings form keeps Pi's writes, `pi-acp`'s session listing, and its validating
+map-miss fallback on one directory. The alternatives remain extending the harness-session mount
+model to local execution, or linking the temporary agent directory's `sessions` child to stable
+storage. Confirm the choice with fault-injection tests that prove teardown, runner restart, and
+storage-failure behavior.
+
+Durability improves efficiency, but it does not relax the verified-load gate. A durable file can
+still be missing, corrupt, or mismatched.
+
+Two processes must never append to the same native transcript concurrently. Reuse the existing
+session ownership and serialization boundary and fail closed if exclusive ownership cannot be
+proven. This is deliberately different from history uncertainty. Uncertain history degrades to a
+clean session with canonical replay because reading stale history risks nothing. Uncertain write
+ownership refuses the turn because two writers can corrupt the transcript for every later turn.
+
+## Append-only Pi skill snapshots
+
+Use the literal non-hidden path:
+
+```text
+<cwd>/
+|-- AGENTS.md
+`-- agents/
+    `-- skills/
+        `-- <skill-set-digest>/
+            |-- .agenta-skill-set.json
+            `-- <configured-skill>/
+                `-- SKILL.md
+```
+
+The digest covers normalized skill names and complete package contents. A cold resume with the same
+configuration gets the same path. A changed or removed skill produces a new path.
+
+The runner writes the completion record last and supplies only a complete current snapshot as an
+explicit Pi skill source. It never trusts the whole project. It never deletes or replaces an old
+snapshot during acquisition. Old paths remain readable for references already present in a resumed
+transcript, but Pi does not advertise them as current skills.
+
+Snapshots are immutable by convention, not enforcement. The cwd is agent-writable, so a run can
+edit its own snapshot files, and the completion record lists expected files without hashing their
+contents. This is the accepted threat model: the workspace already belongs to the same user and
+agent, so an in-place edit crosses no trust boundary. Acquisition-time content verification is out
+of scope.
+
+This intentionally trades bounded storage growth for launch safety. Session retention can remove
+the whole workspace when the session is deleted. Fine-grained garbage collection requires a
+separate post-launch design.
+
+## Refresh timing
+
+```text
+same warm environment and same fingerprint
+  -> reuse the loaded harness and resource paths
+
+cold start, cold resume, or changed fingerprint
+  -> mount/create cwd and private transcript storage
+  -> materialize or validate the current skill snapshot
+  -> attempt verified native load
+  -> verified: send newest message only
+  -> anything else: create clean native session and send canonical replay
+```
+
+## Failure behavior
+
+- Native history uncertainty falls back to canonical replay, not a failed user turn.
+- A missing or malformed skill snapshot fails before Pi starts. It never falls back to temporary
+  skill paths.
+- An existing snapshot with a mismatched completion record is left untouched and rejected.
+- Environment acquisition performs no recursive workspace deletion.
+- Transcript storage never includes credentials, provider settings, or executable extensions.
+- Logs include IDs, path hashes, existence, header identity, load outcome, and replay choice. They
+  never include transcript or skill content.

docs/design/agent-workflows/projects/session-workspace-materialization/plan.md

@@ -0,0 +1,103 @@
+# Plan
+
+## Phase 0: launch gate for conversation correctness
+
+1. Turn the supplied `session-7e9ad207` run into a production-shaped regression case.
+2. Replace inferred `loaded=true` with an explicit verified-history outcome. With pinned
+   `pi-acp@0.0.29` no load can produce evidence, so every cold Pi continuation replays. That is
+   the intended launch behavior.
+3. On missing, corrupt, mismatched, or unverified history, invalidate continuity, create a clean
+   native session, and send `plan.turnText`.
+4. Record the new native ID only after the replayed turn succeeds.
+5. Add structured logs and counters without transcript content.
+
+Exit criterion: after deleting or replacing the mapped Pi transcript, turn 1 still receives turn 0
+through canonical replay. Only a verified native load sends the newest message alone.
+
+This phase is the release gate. It contains no workspace reconciliation, skill move, transcript
+mount, `pi-acp` change, or Claude behavior change. The gate must ship even if no verified path
+exists yet.
+
+## Phase 0b: restore verified loads
+
+1. Stop the sandbox-agent wrapper from substituting the requested ID when the adapter omits actual
+   loaded identity. This cleanup is required before any Pi load can become verified, but it is not
+   required for the Phase 0 gate because Phase 0 bypasses cold Pi loading entirely.
+2. Preferred: fix `pi-acp` so `session/load` rejects a missing transcript before spawning Pi with
+   `--session`, requires the transcript header ID to equal the requested mapping ID, and returns
+   the actual loaded ID. Deliver through an upstream fix, a version bump, or a patch file like the
+   existing sandbox-agent patch. Name the mechanism in the implementation PR.
+3. Acceptable interim: the runner performs the existence and header-identity check itself on local
+   runs. It already holds the agent directory, the expected native ID, and a transcript header
+   parser in `pi-error.ts`, so this adds no new access path.
+4. Either source upgrades the history outcome to `verified` and re-enables newest-message-only
+   prompting. The conservative fallback from Phase 0 stays in place regardless.
+
+## Phase 1: durable local Pi transcripts
+
+1. Point Pi's session directory at stable private per-conversation storage by writing the
+   `sessionDir` key into the per-run agent directory's `settings.json`. Use the settings form,
+   not the env var: `pi-acp` resolves its scan directory from the same settings key and ignores
+   `PI_CODING_AGENT_SESSION_DIR`. Keep the harness-session mount extension and the
+   `sessions`-child link as fallbacks if the probe fails.
+2. Probe the override end to end with pinned versions: Pi writes new transcripts to the durable
+   directory, `pi-acp`'s session listing and map-miss fallback scan the same directory, and a
+   deleted `session-map.json` entry recovers through the header-matching scan.
+3. Persist only native transcripts. Keep auth, settings, custom models, extensions, and system
+   prompt files ephemeral.
+4. Align transcript retention and cleanup with the Agenta conversation lifecycle.
+5. Enforce single-writer ownership for each native session.
+6. Migrate stale existing mappings through replay rather than failing the turn.
+7. Test runner restart, teardown, remount, and unavailable-storage behavior.
+
+Exit criterion: a valid local Pi native transcript survives full environment teardown and runner
+restart, while a lost transcript still recovers through Phase 0.
+
+## Phase 2: prove the explicit Pi skill seam
+
+1. Use pinned Pi 0.80.6 and `pi-acp@0.0.29`.
+2. Create an untrusted cwd with a marker under `agents/skills/<digest>`.
+3. Add only that absolute snapshot path to the isolated agent directory's global `skills` list.
+4. Add different markers under trust-gated `.agents/skills` and `.pi/extensions`.
+5. Confirm only the explicit snapshot loads, the extension does not execute, and no trust decision
+   is persisted.
+6. If the seam fails, stop. Review a narrow `pi-acp` `--skill` passthrough separately.
+
+Exit criterion: Pi loads one literal cwd snapshot without trusting project resources.
+
+## Phase 3: append-only Pi skill snapshots
+
+1. Compute a deterministic digest from normalized Pi skill names and complete contents.
+2. Materialize `agents/skills/<digest>/<name>` after the cwd mount and before harness startup.
+3. Write and validate a completion record after all expected files succeed.
+4. Reuse a complete matching snapshot without rewriting it.
+5. Fail without mutation on collisions or mismatched completion records.
+6. Configure Pi with only the current snapshot.
+7. Stop copying configured skills into the temporary local and Daytona Pi agent directories.
+8. Keep old snapshots. Do not add acquisition-time garbage collection.
+9. Leave Claude's `.claude/skills` implementation unchanged.
+
+Exit criterion: cold resumes retain stable skill paths, removed skills are not advertised by new
+runs, and old transcript references remain readable.
+
+## Phase 4: verification and rollout
+
+1. Run focused runner tests for load identity, replay selection, local teardown, Daytona, partial
+   writes, collisions, and concurrency.
+2. Run the canonical services test entrypoint.
+3. Run the local and Daytona Pi cells from the agent-workflows QA matrix.
+4. Replay both supplied incidents: stale skill paths and lost turn 0.
+5. Confirm the implementation diff contains no Claude runtime changes and no recursive cwd delete.
+6. Roll out the verified-load gate before the storage and skill phases.
+
+## Delivery shape
+
+Use separate reviewable implementation PRs linked to one issue:
+
+1. conservative fallback gate (Phase 0);
+2. verified-load evidence (Phase 0b, may follow the gate);
+3. durable local Pi transcript state;
+4. append-only cwd skill snapshots.
+
+Do not combine the launch gate with reconciliation, the `pi-acp` change, or the filesystem
+migrations.