[5650] feat(ci): clone-based Railway previews behind a mode switch (WP3)

[mmabrouk]

Part of the clone-based Railway preview redesign (#5650, WP3 of 3, final). Do not merge before #5665; merge order is bottom-up through the stack.

Today every preview is constructed imperatively (~75 Railway mutations per run), which is why previews keep failing in new ways. The spike (#5658) proved the replacement: clone a prepared template environment, patch the image tags in one batched mutation, deploy, delete on close. This PR wires that flow into the real preview workflows behind a switch, with the legacy path untouched.

What changes

• One switch: the mode job in workflow 14 resolves vars.RAILWAY_PREVIEW_MODE || 'legacy' once per run and passes it down, so a single repo variable flips previews between modes, and rollback is flipping it back. The variable does not exist yet; everything defaults to legacy.
• preview-clone-create.sh: create-or-update of the per-PR clone environment. Fresh PR: clone the template (skipInitialDeploys, poll-by-name on ambiguous create), one environmentPatchCommit for all app images, ordered deploys (infra → migrations → rest, supertokens after alembic), one retry for a slow first Postgres deploy, then smoke checks. Subsequent push: just patch and redeploy. --verify-only gives workflow 43 a mutation-free re-check that emits the same outputs the legacy deploy step did, so the tests job and PR comment work unchanged.
• preview-clone-destroy.sh: idempotent environment deletion plus a stale-clone sweep (--stale-hours) for the cleanup cron, with pr-template and production double-protected.
• Workflows 41/43/45: legacy steps gated mode != 'clone' with their command lines byte-identical; clone steps added alongside. 45 also gains the stale-clone sweep in its daily cron (runs in both modes deliberately, so a rollback to legacy cannot strand clone environments).
• Workflow 48: the evidence harness. Dispatchable and PR-triggered; runs N consecutive create→smoke→destroy cycles with the production scripts and fails on any cycle failure.

Verification

• Live, against the test bed: fresh create+patch+deploy+smoke green in 89s / 24 API calls (7 mutations); idempotent re-run 3s (empty patch-set handled, nothing redeployed); verify-only 2s; destroy and absent-destroy both green. Full redacted transcript in the PR comment below.
• Legacy equivalence: with the variable unset, every legacy step's run/env/with block is byte-identical; the only expressions that changed are output fallbacks (steps.deploy.X || steps.verify.X) that render identical values in legacy mode. Stated precisely in the equivalence note in the comment.
• shellcheck, bash -n, and actionlint clean.
• This PR's own CI run of workflow 48 provides the "consecutive green clone cycles from a real Actions run" acceptance evidence.

Refs #5650

https://claude.ai/code/session_011zdniFW44QDqcs4Nz8n3hr

[mmabrouk]

Live verification transcript (WP3 acceptance) + legacy-equivalence note:

# WP3 verification — clone-mode preview scripts (issue #5650)
#
# Date: 2026-08-02. Live run of the PRODUCTION scripts
# (hosting/railway/oss/scripts/preview-clone-{create,destroy}.sh) against the
# test bed: project agenta-oss-clone-spike, template env pr-template,
# preview env pr-clone-wp3, patch tag pr-5651-a46168f (template app tag is
# v0.107.0, so the patchCommit no-op trap does not apply).
# Transcript is verbatim script output (redaction-checked; the scripts never
# print tokens, and error paths go through rw_redact).
#
# Static validation (same session):
#   bash -n  preview-clone-create.sh preview-clone-destroy.sh  -> OK
#   shellcheck -x -P SCRIPTDIR (v0.10.0, default severity)     -> clean
#   actionlint (v1.7.7) on workflows 14/41/43/45/48            -> clean

## Phase 1 — create (fresh clone + patch + deploy + smoke)
$ RAILWAY_PREVIEW_ENV_NAME=pr-clone-wp3 IMAGE_TAG=pr-5651-a46168f preview-clone-create.sh
Cloning template 'pr-template' into environment 'pr-clone-wp3'.
environmentPatchCommit: 8 service(s) -> api worker-streams worker-queues cron alembic web services runner
Smoke-checking https://gateway-pr-clone-wp3.up.railway.app
OK: /w
OK: /api/health
OK: /services/health
Preview ready (create): https://gateway-pr-clone-wp3.up.railway.app/w
Timings: created=6s deployed=88s smoked=89s total=89s api_calls=24
preview-clone-create (create): 24 Railway API call(s) so far
exit: 0

## Phase 2 — verify-only (workflow 43's clone-mode step; mutation-free)
$ RAILWAY_PREVIEW_ENV_NAME=pr-clone-wp3 preview-clone-create.sh --verify-only
Smoke-checking https://gateway-pr-clone-wp3.up.railway.app
OK: /w
OK: /api/health
OK: /services/health
Preview ready (verify): https://gateway-pr-clone-wp3.up.railway.app/w
Timings: created=-s deployed=-s smoked=-s total=2s api_calls=5
preview-clone-create (verify): 29 Railway API call(s) so far
exit: 0

## Phase 3 — idempotent re-run (subsequent-push/update path, same tag)
$ RAILWAY_PREVIEW_ENV_NAME=pr-clone-wp3 IMAGE_TAG=pr-5651-a46168f preview-clone-create.sh
Environment 'pr-clone-wp3' already exists; converging it to tag pr-5651-a46168f.
All app images already at pr-5651-a46168f; nothing to patch (explicit deploys cover the no-op trap).
Smoke-checking https://gateway-pr-clone-wp3.up.railway.app
OK: /w
OK: /api/health
OK: /services/health
Preview ready (update): https://gateway-pr-clone-wp3.up.railway.app/w
Timings: created=1s deployed=3s smoked=3s total=3s api_calls=11
preview-clone-create (update): 40 Railway API call(s) so far
exit: 0

## Phase 4 — stale sweep dry-run (verifies environments.createdAt live;
##           pr-template and production correctly excluded)
$ RAILWAY_PREVIEW_DRY_RUN=true preview-clone-destroy.sh --stale-hours 24
KEEP: 'pr-clone-wp3' (age: 0h, max: 24h)
Stale sweep complete (dry-run). Would delete: 0, kept: 1 (max age: 24h)
preview-clone-destroy: 43 Railway API call(s) so far
exit: 0

## Phase 5 — destroy
$ RAILWAY_PREVIEW_ENV_NAME=pr-clone-wp3 preview-clone-destroy.sh
Deleted preview environment 'pr-clone-wp3'
preview-clone-destroy: 47 Railway API call(s) so far
exit: 0

## Phase 6 — destroy again (idempotency: absent environment = success)
$ RAILWAY_PREVIEW_ENV_NAME=pr-clone-wp3 preview-clone-destroy.sh
Environment 'pr-clone-wp3' does not exist in project 'agenta-oss-clone-spike'. Nothing to delete.
preview-clone-destroy: 50 Railway API call(s) so far
exit: 0

## Totals
# Full cycle (create 89s + destroy ~4s): green. 50 Railway API calls for the
# ENTIRE session (create 24, verify 5, update 11, stale dry-run 3, destroy 4,
# idempotent destroy 3) — all phases share one RW_CALLS_FILE counter.
# The create phase alone: 24 calls, of which 7 mutations (environmentCreate,
# 3 infra deploys, 1 environmentPatchCommit covering all 8 app images,
# supertokens + gateway deploys; no domain create needed — Railway regenerated
# the gateway domain in the clone). The other 17 are bounded reads: project/env
# resolution, populate check, domain check, and ~15s-interval status polls.
# No cycle failure; the single-retry Postgres path was not needed.

Legacy byte-equivalence: with RAILWAY_PREVIEW_MODE unset/legacy, every legacy step's run/env/with block is byte-identical to before. Changes on the legacy path are exactly: mode-gates that evaluate true, output fallbacks (steps.deploy.X || steps.verify.X) that render identical values, an inert env var in workflow 45, the additive mode job in 14, and the deliberately mode-independent stale-clone sweep in the daily cron (so a rollback cannot strand clone environments).

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agenta-documentation	Ready	Preview	Aug 3, 2026 6:37am

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6b7f5bdf-4e3b-4bba-87d5-65d4898d1e3f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[mmabrouk]

Acceptance evidence (real Actions run): workflow 48 run 30766969925 — 3 consecutive clone cycles with the production scripts, all green: cycle 1 total 133s / 27 API calls, cycle 2 total 86s / 24 calls, cycle 3 green (see run summary for the table). Every cycle passed all three smoke checks (/w, /api/health, /services/health) and destroyed its environment cleanly.

[github-actions]

Railway Preview Environment

Status	Destroyed (PR closed)

Updated at 2026-08-03T06:56:11.690Z