Skip to content

fix(hosting): expose the object store for Daytona sandboxes on self-hosted deployments - #5320

Merged
mmabrouk merged 1 commit into
big-agentsfrom
fix-selfhost-daytona-store-exposure
Jul 14, 2026
Merged

fix(hosting): expose the object store for Daytona sandboxes on self-hosted deployments#5320
mmabrouk merged 1 commit into
big-agentsfrom
fix-selfhost-daytona-store-exposure

Conversation

@mmabrouk

Copy link
Copy Markdown
Member

Symptom

On a self-hosted (gh) deployment using Daytona cloud sandboxes, every file an agent writes is silently lost. Working directories do not persist across turns, and the only trace is a mount degraded line in the runner logs.

Cause

A Daytona sandbox runs geesefs (FUSE over S3) inside the cloud, so it reaches the store over the internet using the endpoint in the signed mount credentials. The gh compose bundles SeaweedFS bound to loopback with traefik.enable=false, so that endpoint (http://seaweedfs:8333) is unreachable from Daytona. The runner's reachability check fails, it finds no ngrok tunnel (those exist only in the dev compose), and it skips the mount without failing the turn. Dev never reproduces this because the dev stack ships the ngrok tunnel.

Fix (config + docs only, no runner/API code changes)

The API already carries the store endpoint into the signed mount credentials through the existing AGENTA_STORE_ENDPOINT_URL variable (MountCredentials.endpoint = store.endpoint_url in api/oss/src/core/mounts/service.py). The missing piece was a way to expose the bundled store publicly. This PR adds an opt-in traefik router on the seaweedfs service, off by default:

labels:
    - "traefik.enable=${AGENTA_STORE_TRAEFIK_ENABLE:-false}"
    - "traefik.http.routers.store.rule=Host(`${AGENTA_STORE_DOMAIN:-store.localhost}`)"
    - "traefik.http.routers.store.entrypoints=web,web-secure"   # web only on the plain gh stack
    - "traefik.http.routers.store.service=store"
    - "traefik.http.routers.store.tls=true"                     # ssl stack only
    - "traefik.http.routers.store.tls.certresolver=myResolver"  # ssl stack only
    - "traefik.http.services.store.loadbalancer.server.port=8333"

An operator opts in with two variables in the env file, plus a public store endpoint:

AGENTA_STORE_TRAEFIK_ENABLE=true
AGENTA_STORE_DOMAIN=store.example.com
AGENTA_STORE_ENDPOINT_URL=https://store.example.com

When unset, traefik.enable=false makes traefik ignore the store, so the change is inert on existing deployments. The router matches on the hostname and never rewrites the path, which SeaweedFS S3 SigV4 requires, so the store gets its own subdomain rather than a path prefix.

Applied to the three gh variants that can serve a public host:

  • oss/docker-compose.gh.ssl.yml: full TLS via the existing myResolver Let's Encrypt resolver.
  • oss/docker-compose.gh.yml and ee/docker-compose.gh.yml: port-80 router, TLS terminated at the operator's own proxy (EE has no bundled-TLS stack).

gh.local is left untouched: it is local-only and a Daytona cloud sandbox cannot reach it.

Docs and env examples

  • docs/docs/self-host/agents/02-daytona.mdx: rewrote "Make the store reachable from the sandbox" to lead with the concrete traefik recipe (hostname, DNS record, the three env vars, verification), followed by the external-S3 and dev-only ngrok options.
  • oss/env.oss.gh.example and ee/env.ee.gh.example: documented the two new opt-in vars next to AGENTA_STORE_ENDPOINT_URL.

Key finding

The endpoint chain works without any code change. AGENTA_STORE_ENDPOINT_URL already feeds the signed credentials, and it is already plumbed onto the api and worker services in every gh compose file. The one caveat: the API uses a single store endpoint for both its own in-network client and the sandbox credentials, so setting it to a public URL routes the API's own store traffic back through traefik (hairpin). That is the accepted single-endpoint design, and it is what the existing docs already recommend.

Verification

  1. On an --ssl gh deployment, set AGENTA_STORE_TRAEFIK_ENABLE=true, AGENTA_STORE_DOMAIN=store.example.com, AGENTA_STORE_ENDPOINT_URL=https://store.example.com, and add a DNS record for the store host. Recreate the stack.
  2. Confirm traefik issued a cert and the store answers: curl -I https://store.example.com returns an S3 response (not a 404 from the web app).
  3. Run an agent on Daytona and have it write a file.
  4. The runner log flips from mount degraded ... cause=sign_returned_no_mount to remote mounted <bucket>:<prefix> -> <cwd> (verified alive).
  5. Start a second turn and confirm the file written in step 3 is still there.

Without the override, step 4 shows the mount degraded line and step 5 finds the file gone.

https://claude.ai/code/session_01Hyn9365BLPXDmNZShrQkmH

…osted deployments

On a gh self-host deployment using Daytona cloud sandboxes, geesefs runs inside
the sandbox and reaches the store over the internet. The bundled SeaweedFS is
bound to loopback with traefik.enable=false, so the signed store endpoint is
unreachable from Daytona and the runner skips the mount silently, dropping every
file the agent writes.

Add an opt-in traefik router on the seaweedfs service (off by default) that
publishes the S3 endpoint on its own subdomain, gated by AGENTA_STORE_TRAEFIK_ENABLE
and AGENTA_STORE_DOMAIN. The API's existing AGENTA_STORE_ENDPOINT_URL then carries
the public endpoint into the signed mount credentials. Host-only routing keeps the
request path intact, which SeaweedFS S3 SigV4 requires. Document the recipe on the
Daytona self-host page and in the gh env examples.

Claude-Session: https://claude.ai/code/session_01Hyn9365BLPXDmNZShrQkmH
@mmabrouk

Copy link
Copy Markdown
Member Author

@coderabbitai review

@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Ready Ready Preview, Comment Jul 14, 2026 7:06pm

Request Review

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c52bad09-855d-414c-96ee-09c5d44959e2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-selfhost-daytona-store-exposure

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@mmabrouk
mmabrouk changed the base branch from main to big-agents July 14, 2026 19:03
@mmabrouk
mmabrouk marked this pull request as ready for review July 14, 2026 19:10
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. documentation Improvements or additions to documentation labels Jul 14, 2026
@mmabrouk
mmabrouk merged commit c55b30f into big-agents Jul 14, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant