fix(hosting): expose the object store for Daytona sandboxes on self-hosted deployments - #5320
Conversation
…osted deployments On a gh self-host deployment using Daytona cloud sandboxes, geesefs runs inside the sandbox and reaches the store over the internet. The bundled SeaweedFS is bound to loopback with traefik.enable=false, so the signed store endpoint is unreachable from Daytona and the runner skips the mount silently, dropping every file the agent writes. Add an opt-in traefik router on the seaweedfs service (off by default) that publishes the S3 endpoint on its own subdomain, gated by AGENTA_STORE_TRAEFIK_ENABLE and AGENTA_STORE_DOMAIN. The API's existing AGENTA_STORE_ENDPOINT_URL then carries the public endpoint into the signed mount credentials. Host-only routing keeps the request path intact, which SeaweedFS S3 SigV4 requires. Document the recipe on the Daytona self-host page and in the gh env examples. Claude-Session: https://claude.ai/code/session_01Hyn9365BLPXDmNZShrQkmH
|
@coderabbitai review |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
✅ Action performedReview finished.
|
Symptom
On a self-hosted (
gh) deployment using Daytona cloud sandboxes, every file an agent writes is silently lost. Working directories do not persist across turns, and the only trace is amount degradedline in the runner logs.Cause
A Daytona sandbox runs geesefs (FUSE over S3) inside the cloud, so it reaches the store over the internet using the endpoint in the signed mount credentials. The
ghcompose bundles SeaweedFS bound to loopback withtraefik.enable=false, so that endpoint (http://seaweedfs:8333) is unreachable from Daytona. The runner's reachability check fails, it finds no ngrok tunnel (those exist only in thedevcompose), and it skips the mount without failing the turn. Dev never reproduces this because the dev stack ships the ngrok tunnel.Fix (config + docs only, no runner/API code changes)
The API already carries the store endpoint into the signed mount credentials through the existing
AGENTA_STORE_ENDPOINT_URLvariable (MountCredentials.endpoint = store.endpoint_urlinapi/oss/src/core/mounts/service.py). The missing piece was a way to expose the bundled store publicly. This PR adds an opt-in traefik router on theseaweedfsservice, off by default:An operator opts in with two variables in the env file, plus a public store endpoint:
When unset,
traefik.enable=falsemakes traefik ignore the store, so the change is inert on existing deployments. The router matches on the hostname and never rewrites the path, which SeaweedFS S3 SigV4 requires, so the store gets its own subdomain rather than a path prefix.Applied to the three
ghvariants that can serve a public host:oss/docker-compose.gh.ssl.yml: full TLS via the existingmyResolverLet's Encrypt resolver.oss/docker-compose.gh.ymlandee/docker-compose.gh.yml: port-80 router, TLS terminated at the operator's own proxy (EE has no bundled-TLS stack).gh.localis left untouched: it is local-only and a Daytona cloud sandbox cannot reach it.Docs and env examples
docs/docs/self-host/agents/02-daytona.mdx: rewrote "Make the store reachable from the sandbox" to lead with the concrete traefik recipe (hostname, DNS record, the three env vars, verification), followed by the external-S3 and dev-only ngrok options.oss/env.oss.gh.exampleandee/env.ee.gh.example: documented the two new opt-in vars next toAGENTA_STORE_ENDPOINT_URL.Key finding
The endpoint chain works without any code change.
AGENTA_STORE_ENDPOINT_URLalready feeds the signed credentials, and it is already plumbed onto theapiandworkerservices in everyghcompose file. The one caveat: the API uses a single store endpoint for both its own in-network client and the sandbox credentials, so setting it to a public URL routes the API's own store traffic back through traefik (hairpin). That is the accepted single-endpoint design, and it is what the existing docs already recommend.Verification
--sslgh deployment, setAGENTA_STORE_TRAEFIK_ENABLE=true,AGENTA_STORE_DOMAIN=store.example.com,AGENTA_STORE_ENDPOINT_URL=https://store.example.com, and add a DNS record for the store host. Recreate the stack.curl -I https://store.example.comreturns an S3 response (not a 404 from the web app).mount degraded ... cause=sign_returned_no_mounttoremote mounted <bucket>:<prefix> -> <cwd> (verified alive).Without the override, step 4 shows the
mount degradedline and step 5 finds the file gone.https://claude.ai/code/session_01Hyn9365BLPXDmNZShrQkmH