Off-platform database backups, and an alarm when they stop - #36
Conversation
Time Travel already covers the failure modes that actually happen — a bad migration, a DELETE without a WHERE — for 30 days, free. It cannot survive the database being deleted, and it cannot get a copy of the data off the platform. This closes only that gap, rather than rebuilding what the platform already does well. The export runs nightly in CI, not in the cron: D1's export is an account-scoped REST operation that needs an account token with no business being in a Worker, and it is polled to completion, which no cron invocation's CPU budget allows. The dump is encrypted to a public key before it is uploaded, and the job fails rather than ever falling back to plaintext. Artifacts on a public repository are downloadable by anyone, and this dump holds refresh tokens, private drafts and the email list — uploading it in the clear would be a worse breach than the loss it prevents. CI can write backups it cannot read; only the holder of the secret key can restore. A dump that exists is not a dump that is usable, so the job checks size, the expected tables and the presence of real rows before it counts. It stamps a heartbeat only after a verified, encrypted, uploaded backup exists. The hourly cron reads that heartbeat, because the failure this really guards against is silence: a backup job that has quietly stopped looks exactly like one that is working. A stale heartbeat is an invariant failure and rides the existing alert path rather than growing a parallel one. Inert until configured — with no recipient key or secrets set, the cron correctly reports having no backup, which is the honest state.
📝 WalkthroughWalkthroughAdds a nightly encrypted D1 export workflow, validates and uploads backup artifacts, records heartbeat rows, monitors freshness through scheduled jobs, alerts on failures, and documents Time Travel and encrypted-dump restoration. ChangesD1 backup lifecycle
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant Wrangler
participant Age
participant D1
participant ScheduledCron
participant Webhook
GitHubActions->>Wrangler: export remote D1 database
GitHubActions->>Age: encrypt verified SQL dump
GitHubActions->>D1: record backup_runs heartbeat
ScheduledCron->>D1: read latest heartbeat
ScheduledCron->>Webhook: alert on freshness failures
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
goldroad-staging | cb7cab7 | Commit Preview URL Branch Preview URL |
Jul 30 2026, 01:29 PM |
![coderabbitai[bot]](https://avatars.github.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/backup.yml:
- Around line 49-50: Replace the mutable action tags in
.github/workflows/backup.yml at lines 49-50, 52-53, 55-56, and 121-122 with
reviewed full immutable commit SHAs: respectively pin actions/checkout@v7,
pnpm/action-setup@v6.0.9, actions/setup-node@v7, and actions/upload-artifact@v4.
In `@docs/BACKUPS.md`:
- Around line 137-140: Update the “From Time Travel — the usual case” section to
remove the claim that the restore “loses nothing”; explain that restoring
overwrites the database in place and discards later writes, and recommend
recording or exporting the current bookmark before using `wrangler d1
time-travel restore`.
- Around line 13-16: Update the D1 Time Travel retention statements in
docs/BACKUPS.md lines 13-16 and SELF_HOSTING.md lines 88-90 to remove the
unconditional “every plan” claim and specify plan-dependent limits:
approximately 30 days for Workers Paid and up to 7 days for Workers Free; keep
the surrounding backup guidance unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e36818cd-55fb-47df-a519-a1083362c307
📒 Files selected for processing (11)
.github/workflows/backup.ymlSELF_HOSTING.mddocs/BACKUPS.mddrizzle/0007_tidy_havok.sqldrizzle/meta/0007_snapshot.jsondrizzle/meta/_journal.jsonsrc/__tests__/backup.test.tssrc/__tests__/scheduled.test.tssrc/db/schema.tssrc/lib/backup.tssrc/lib/scheduled.ts
| - name: Checkout | ||
| uses: actions/checkout@v7 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -nP '^\s*uses:\s+\S+@(?![0-9a-f]{40}(?:\s|$))' .github/workflows/backup.ymlRepository: peterkibuchi/goldroad
Length of output: 322
🌐 Web query:
actions/checkout v7 release commit
💡 Result:
The actions/checkout v7 release was officially released as version v7.0.0 on June 18, 2026 [1][2]. The merge commit associated with the final preparations for this release (pull request #2464) is 1044a6dea927916f2c38ba5aeffbc0a847b1221a [3][4]. As of July 30, 2026, the latest release in the v7 series is v7.0.1, which was published on July 20, 2026 [5][6]. The primary change introduced in the v7.0.0 release was a security update that changes the default behavior to refuse checking out fork pull request code in pull_request_target and workflow_run workflows, aimed at preventing "pwn request" vulnerabilities [2].
Citations:
- 1: https://github.com/actions/checkout/releases/tag/v7.0.0
- 2: https://github.blog/changelog/2026-06-18-safer-pull_request_target-defaults-for-github-actions-checkout/
- 3: getting ready for checkout v7 release actions/checkout#2464
- 4: actions/checkout@v6.0.3...v7
- 5: https://github.com/actions/checkout/releases/tag/v7.0.1
- 6: https://github.com/actions/checkout/releases
🌐 Web query:
pnpm/action-setup v6.0.9 tag commit
💡 Result:
The GitHub release v6.0.9 for pnpm/action-setup is associated with the commit SHA 0ebf47130e4866e96fce0953f49152a61190b271 [1][2][3]. This release was published on June 15, 2026, and its primary change was to update pnpm to v11.7.0 [4].
Citations:
- 1: pnpm/action-setup@v6.0.8...v6.0.9
- 2: raviqqe/stak@0aa1ffe
- 3: pnpm/action-setup@fc06bc1...0ebf471
- 4: https://github.com/pnpm/action-setup/releases/tag/v6.0.9
🌐 Web query:
actions/setup-node v7 commit
💡 Result:
The actions/setup-node v7.0.0 release is associated with commit 8207627 [1][2]. This version was published on July 14, 2026 [3]. Key updates in v7 include: - Migration of action internals to ESM (ECMAScript Modules) to ensure compatibility with updated @actions/* packages [4]. - An upgrade of the action's underlying runtime from node20 to node24 [5][4]. - Requirement of GitHub runner version v2.327.1 or later to ensure compatibility [5][4]. There are no changes to the action's inputs, outputs, or general behavior in this release [4].
Citations:
- 1: actions/setup-node@v6...v7
- 2: actions/setup-node@v6.4.0...v7.0.0
- 3: https://github.com/actions/setup-node/releases/tag/v7.0.0
- 4: https://github.com/actions/setup-node/tree/refs/heads/main
- 5: https://github.com/actions/setup-node
🌐 Web query:
actions/upload-artifact v4 commit
💡 Result:
The actions/upload-artifact action has progressed significantly beyond version 4. As of July 30, 2026, the latest available version is v7.0.1 [1][2]. Because v4 is a major version (released December 2023 [3][4]), it is not a single commit SHA, but a series of releases (v4.0.0 through v4.6.2) [5][2]. Users often pinned the v4 tag to a specific commit SHA for security and reproducibility [6][7]. For example, v4.6.2 was historically associated with commit SHA ea165f8d65b6e75b540449e92b4886f43607fa02 [6][7]. It is highly recommended to check the official GitHub Actions repository for the most recent version, as v4 is significantly outdated compared to current releases like v7 [1][2]. When pinning actions in your workflow, you can find the specific commit SHAs for any release tag directly on the official releases page of the actions/upload-artifact repository [2].
Citations:
- 1: https://github.com/actions/upload-artifact/tree/v4.0.0
- 2: https://github.com/actions/upload-artifact/releases
- 3: https://github.com/actions/upload-artifact/releases/tag/v4.0.0
- 4: https://github.blog/changelog/2023-12-14-github-actions-artifacts-v4-is-now-generally-available/
- 5: actions/upload-artifact@v4.0.0...v4.6.2
- 6: Pin
actions/upload-artifactto SHA actions/upload-pages-artifact#127 - 7: Arize-ai/arize-skills@d980b15
Pin the workflow actions to immutable commit SHAs.
Floating GitHub Action tags are mutable and can change before later steps receive Cloudflare credentials. Replace the version tags with reviewed full SHAs for:
actions/checkout@v7pnpm/action-setup@v6.0.9actions/setup-node@v7actions/upload-artifact@v4
🧰 Tools
🪛 zizmor (1.28.0)
[warning] 49-50: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 50-50: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
📍 Affects 1 file
.github/workflows/backup.yml#L49-L50(this comment).github/workflows/backup.yml#L52-L53.github/workflows/backup.yml#L55-L56.github/workflows/backup.yml#L121-L122
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/backup.yml around lines 49 - 50, Replace the mutable
action tags in .github/workflows/backup.yml at lines 49-50, 52-53, 55-56, and
121-122 with reviewed full immutable commit SHAs: respectively pin
actions/checkout@v7, pnpm/action-setup@v6.0.9, actions/setup-node@v7, and
actions/upload-artifact@v4.
Source: Linters/SAST tools
| Every D1 database has a 30-day point-in-time restore built in, on every plan. | ||
| It is the right tool for the failure modes that actually happen: a migration | ||
| that drops the wrong column, a `DELETE` without a `WHERE`, a backfill that | ||
| corrupts a column. There is no setup and no cost. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Relevant documentation snippets:\n'
if [ -f docs/BACKUPS.md ]; then
sed -n '1,40p' docs/BACKUPS.md | cat -n
else
fd -a 'BACKUPS\.md$' . | while read -r f; do echo "--- $f"; sed -n '1,40p' "$f" | cat -n; done
fi
if [ -f SELF_HOSTING.md ]; then
sed -n '75,100p' SELF_HOSTING.md | cat -n
else
fd -a 'SELF_HOSTING\.md$' . | while read -r f; do echo "--- $f"; sed -n '75,100p' "$f" | cat -n; done
fi
printf '\nSearch for Cloudflare D1/Time Travel references:\n'
rg -n --hidden -S "Time Travel|point-in-time|D1|Workers (Paid|Free)|30-day|30 day|7-day|7 day|production-backend|back-end|backend" -g '!node_modules' -g '!dist' -g '!build' .Repository: peterkibuchi/goldroad
Length of output: 26969
🌐 Web query:
Cloudflare D1 Time Travel Workers Paid 30 days Workers Free 7 days production backend documentation
💡 Result:
Cloudflare D1 Time Travel is a built-in feature that provides point-in-time recovery for your databases, allowing you to restore them to any specific minute within the retention period [1][2]. The retention period for Time Travel is determined by your Cloudflare Workers plan: - Workers Paid plan: 30 days of retention [1][3][4]. - Workers Free plan: 7 days of retention [1][3][4]. Key details regarding Time Travel: - Always On: Time Travel does not need to be enabled; it is automatically active for all D1 databases [1][2]. - No Extra Cost: Database history and restore operations incur no additional costs [1][2]. - Automatic Bookmarks: The system automatically generates bookmarks that you can use as restore points [1][2]. - Restore Limits: You can perform a maximum of 10 restore operations per 10 minutes per database [3][4]. - Long-term Storage: If you need to retain database state for longer than the standard retention period (30 days or 7 days), you can export your database into Cloudflare R2 storage using the REST API or Cloudflare Workflows [1]. - Management: You can manage and trigger restores via the Cloudflare API or using Wrangler (the Cloudflare command-line tool) with the command wrangler d1 time-travel restore <DATABASE_NAME> [5][6][7].
Citations:
- 1: https://developers.cloudflare.com/d1/reference/time-travel/
- 2: https://github.com/cloudflare/cloudflare-docs/blob/production/src/content/docs/d1/reference/time-travel.mdx
- 3: https://developers.cloudflare.com/d1/platform/limits/index.md
- 4: https://developers.cloudflare.com/d1/platform/limits/
- 5: https://developers.cloudflare.com/api/resources/d1/subresources/database/subresources/time_travel/methods/restore/
- 6: https://developers.cloudflare.com/api/python/resources/d1/subresources/database/subresources/time_travel/methods/restore/
- 7: https://97468409.preview.developers.cloudflare.com/d1/wrangler-commands/
🌐 Web query:
site:developers.cloudflare.com/d1/reference/time-travel "free" "retention" "Time Travel"
💡 Result:
The term "Time Travel" in database technology generally refers to a feature that enables point-in-time recovery (PITR) or the ability to query historical data as it existed at a specific moment in the past. While many enterprise database systems offer this, implementations and associated costs vary widely, with some services offering it as a standard, built-in feature without additional fees. A prominent example of "free" or included Time Travel functionality is Cloudflare D1. In this service, Time Travel is an always-on feature that allows users to restore a database to any minute within a specific retention period at no additional cost [1]. The retention period for this feature is tiered based on the user's plan: - Workers Paid plan: Up to 30 days of retention [1]. - Workers Free plan: Up to 7 days of retention [1]. Because this functionality is integrated and does not require manual backup initiation, it serves as an automated approach to data recovery from accidental deletions, incorrect updates, or failed schema migrations [1]. Users should always check the specific documentation for the database system they are using, as many other platforms may charge extra for the storage required to maintain historical snapshots or logs necessary for Time Travel capabilities.
Citations:
🌐 Web query:
site:developers.cloudflare.com/d1/reference/time-travel "backend" "production" "Time Travel"
💡 Result:
In the context of backend production systems, Time Travel refers to point-in-time recovery capabilities, specifically exemplified by Cloudflare D1's database management [1]. Time Travel allows developers to restore a database to a specific state from the past, typically to recover from issues like failed migrations, schema changes, or erroneous data modification queries (such as DELETE or UPDATE statements executed without proper conditions) [1][2]. Key characteristics of this implementation include: - Automatic operation: It is typically always on for supported databases, eliminating the need for manual backup management or scheduling [1]. - Granularity: It enables restoration to any minute within a set retention window (e.g., the last 30 days for certain plans) [1][3]. - Deterministic recovery: Timestamps are mapped to stable, internal bookmarks to ensure consistent recovery results [1]. - Infrastructure requirements: It requires the database to run on specific, modern storage backends. Users can verify their status using command-line tools like Wrangler (e.g., checking if the version output is production) [1][2]. While often associated with database recovery in backend production, the term is conceptually distinct from time-travel debugging, which typically refers to development-time tools that allow stepping through code execution backward and forward [1][2].
Citations:
- 1: https://developers.cloudflare.com/d1/reference/time-travel/
- 2: https://pedro-2025-04-30-docs-add-quotes.preview.developers.cloudflare.com/d1/reference/time-travel/
- 3: https://97468409.preview.developers.cloudflare.com/d1/reference/time-travel/
🌐 Web query:
site:developers.cloudflare.com/d1/reference/time-travel "Workers Free" "30 days" "7 days"
💡 Result:
The references to "30 days" and "7 days" in the context of Cloudflare Workers relate to the Time Travel feature for Cloudflare D1 databases [1][2]. Time Travel allows users to restore a database to a specific point in time [1][2]. The retention period for these restore points depends on the user's plan: - Workers Paid plan: You can restore a database back to a point in time up to 30 days in the past [1][2]. - Workers Free plan: You can restore a database back to a point in time up to 7 days in the past [1][2].
Citations:
- 1: https://pedro-2025-04-30-docs-add-quotes.preview.developers.cloudflare.com/d1/reference/time-travel/
- 2: https://developers.cloudflare.com/d1/reference/time-travel/
🌐 Web query:
site:developers.cloudflare.com/d1/platform/limits "Time Travel" "Workers Free" "Workers Paid"
💡 Result:
The term "Time Travel" in the context of Cloudflare Workers refers to the point-in-time recovery feature available for Cloudflare D1 databases [1][2]. The duration of time for which you can perform point-in-time recovery differs based on your subscription plan: - Workers Paid: You have access to 30 days of Time Travel duration [1][2]. - Free: You have access to 7 days of Time Travel duration [1][2]. For both plans, there is a limit of 10 restore operations per 10 minutes per database [1][2].
Citations:
- 1: https://developers.cloudflare.com/d1/platform/limits/
- 2: https://developers.cloudflare.com/d1/platform/limits/index.md
Correct the unconditional Time Travel retention claims. Cloudflare’s D1 Time Travel retention depends on the Workers plan, with about 30 days for Workers Paid and up to 7 days for Workers Free.
docs/BACKUPS.md#L13-L16: replace “every plan” with the plan-specific retention.SELF_HOSTING.md#L88-L90: align the summary with the actual retention limit.
📍 Affects 2 files
docs/BACKUPS.md#L13-L16(this comment)SELF_HOSTING.md#L88-L90
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/BACKUPS.md` around lines 13 - 16, Update the D1 Time Travel retention
statements in docs/BACKUPS.md lines 13-16 and SELF_HOSTING.md lines 88-90 to
remove the unconditional “every plan” claim and specify plan-dependent limits:
approximately 30 days for Workers Paid and up to 7 days for Workers Free; keep
the surrounding backup guidance unchanged.
| ### From Time Travel — the usual case | ||
|
|
||
| Use section 1. This is what you want for a bad migration or a bad delete: it is | ||
| faster, it needs no keys, and it loses nothing. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## Backups doc context"
if [ -f docs/BACKUPS.md ]; then
sed -n '1,220p' docs/BACKUPS.md | cat -n
else
echo "docs/BACKUPS.md not found"
fi
echo
echo "## Search for Time Travel references"
rg -n "Time Travel|loses nothing|bookmark|restor|restore|D1" docs README.md .github 2>/dev/null || trueRepository: peterkibuchi/goldroad
Length of output: 12019
🌐 Web query:
Cloudflare D1 Time Travel restore overwrites in place newer writes after timestamp loses nothing developers.cloudflare.com
💡 Result:
Cloudflare D1 Time Travel is a destructive, in-place operation that overwrites your current database state with the state from a chosen point in time [1][2][3]. Because it performs an in-place overwrite, any data written to the database after that specific timestamp is lost [2][4][3]. However, the operation is designed to be reversible [1][2]. When you perform a restore, Cloudflare provides a previous_bookmark that represents the state of the database immediately before the restore occurred [5][6][3]. You can use this bookmark to undo the restore operation and revert the database to the state it was in before the overwrite [1][2][3]. Key points regarding the restore process: - Destructive: It overwrites all data in the database in place [2][4][3]. - In-flight operations: Any queries or transactions currently in flight when the restore is initiated will be cancelled and return an error [1][2][3]. - Reversibility: You are provided with a bookmark representing the pre-restore state, which allows you to undo the action [5][6][3]. - Availability: You can restore to any minute within the last 30 days (on Workers Paid plans) or 7 days (on Workers Free plans) [1][2][3].
Citations:
- 1: https://developers.cloudflare.com/d1/reference/time-travel/
- 2: https://github.com/cloudflare/cloudflare-docs/blob/production/src/content/docs/d1/reference/time-travel.mdx
- 3: https://pedro-2025-04-30-docs-add-quotes.preview.developers.cloudflare.com/d1/reference/time-travel/
- 4: 🚀 Feature Request: D1 Time Travel cloudflare/workers-sdk#3577
- 5: https://developers.cloudflare.com/api/resources/d1/subresources/database/subresources/time_travel
- 6: https://developers.cloudflare.com/api/resources/d1/subresources/database/subresources/time_travel/methods/restore/
Warn that Time Travel restores discard later writes.
wrangler d1 time-travel restore overwrites the current database state with the selected historical state, so any legitimate writes after that point are lost unless backed up separately. Update the “loses nothing” sentence to warn about the in-place overwrite and recommend recording/exporting the current bookmark before restoring.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/BACKUPS.md` around lines 137 - 140, Update the “From Time Travel — the
usual case” section to remove the claim that the restore “loses nothing”;
explain that restoring overwrites the database in place and discards later
writes, and recommend recording or exporting the current bookmark before using
`wrangler d1 time-travel restore`.
1 participant
The database holds the only copy of several things: the launch-updates list, private drafts, the import ledger, follower history, and OAuth session rows. Published posts are safe — they live in each writer's own repository — but none of the above does. There was no backup at all.
Time Travel already does most of this job, so this doesn't rebuild it. D1 gives 30-day point-in-time restore on the free plan, and it covers the failures that actually happen: a bad migration, a
DELETEwithout aWHERE. What it can't do is survive the database being deleted, or get a copy of the data off the platform. This closes only that gap.The export runs nightly in CI rather than in the Worker cron, because D1's export is an account-scoped REST operation — it needs an account API token that has no business living in a Worker, and it's polled to completion, which no cron invocation's CPU budget allows.
The dump is encrypted before upload, and that isn't optional. Workflow artifacts on a public repository are downloadable by anyone, and this dump contains refresh tokens, private drafts and the email list — uploading it in the clear would be a worse breach than the data loss it prevents. Encryption is to a public key, so CI can write backups it cannot itself read; only the holder of the secret key can restore. If the recipient key is missing the job fails rather than falling back to plaintext.
A dump that exists is not a dump that is usable, so the job verifies size, expected tables, and the presence of real rows before counting it, and stamps its heartbeat only after a verified, encrypted, uploaded backup exists.
The cron watches that heartbeat, because the real risk here is silence: a backup job that has quietly stopped looks exactly like one that's working. A stale heartbeat is an invariant failure and rides the existing alert path instead of growing a parallel one.
Inert until configured — with no recipient key or secrets set, the cron correctly reports having no backup, which is the honest state. Setup, key generation and restore instructions are in
docs/BACKUPS.md.Gate:
pnpm checkandpnpm typecheckclean, 773 tests pass across 72 files.Summary by CodeRabbit
New Features
Documentation