-
Notifications
You must be signed in to change notification settings - Fork 1
docs: attached-executor deployment guide #603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
kartikeya-27
merged 4 commits into
ByteVeda:master
from
stromanni:docs/589-attached-executors
Aug 2, 2026
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
3d2eb89
docs: add the attached-executor deployment guide
stromanni e8056c0
docs: stop naming only two SDKs on the landing page
stromanni 5de0576
docs: make the Java install pill a real, versioned coordinate
stromanni c4332d2
docs: say the attach token is not transport security
stromanni File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| { "title": "Operations", "pages": ["backends", "inspection", "dashboard", "sso", "mesh", "autoscaling", "keda", "cli", "testing", "security", "troubleshooting", "deployment", "graalvm"] } | ||
| { "title": "Operations", "pages": ["backends", "inspection", "dashboard", "sso", "mesh", "autoscaling", "keda", "cli", "testing", "security", "troubleshooting", "deployment", "executor", "graalvm"] } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| { "title": "Operations", "pages": ["backends", "inspection", "dashboard", "dashboard-api", "sso", "mesh", "keda", "autoscaling", "cli", "testing", "security", "troubleshooting", "deployment", "migration"] } | ||
| { "title": "Operations", "pages": ["backends", "inspection", "dashboard", "dashboard-api", "sso", "mesh", "keda", "autoscaling", "cli", "testing", "security", "troubleshooting", "deployment", "executor", "migration"] } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,7 @@ | |
| "troubleshooting", | ||
| "security", | ||
| "deployment", | ||
| "executor", | ||
| "mesh", | ||
| "autoscaling", | ||
| "keda", | ||
|
|
||
198 changes: 198 additions & 0 deletions
198
docs/content/docs/shared/guides/operations/executor.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,198 @@ | ||
| --- | ||
| title: Attached Executors | ||
| description: "Run task bodies in your app container while a separate scheduler holds the database credentials." | ||
| --- | ||
|
|
||
| An **attached executor** splits a worker in two. The scheduler — `taskito-server` | ||
| — holds the database connection, claims jobs, and owns retries, the dead-letter | ||
| queue and retention. Your app container runs `taskito executor`, which dials the | ||
| scheduler, announces the tasks it can run, and executes whatever it is sent. | ||
|
|
||
| The app image needs no database credentials and no inbound port. | ||
|
|
||
| ``` | ||
| ┌────────────────────────┐ ┌──────────────────────────────┐ | ||
| │ taskito-server │◀────────│ app container (unchanged) │ | ||
| │ scheduler + dashboard │ dial │ <app entrypoint> │ | ||
| │ retry / DLQ / breaker │ out │ + taskito executor │ | ||
| │ retention / reaper │────────▶│ │ | ||
| └────────────────────────┘ jobs └──────────────────────────────┘ | ||
| │ | ||
| storage (source of truth) | ||
| ``` | ||
|
|
||
| <Callout type="info" title="This is opt-in"> | ||
| An in-process worker stays the default and the simplest deployment. Reach for | ||
| an executor when the app image is large enough that running a second copy of | ||
| it as a worker is the expensive part — an interpreter or JVM plus model | ||
| weights resident twice, and a multi-gigabyte image pull on every scale-up. | ||
| </Callout> | ||
|
|
||
| ## Running the scheduler | ||
|
|
||
| `taskito-server` is configured entirely through the environment — there are no | ||
| flags. At minimum it needs a DSN and an attach address: | ||
|
|
||
| ```bash | ||
| TASKITO_DSN=postgres://user:pass@db/taskito \ | ||
| TASKITO_LISTEN=0.0.0.0:7749 \ | ||
| TASKITO_ATTACH_TOKEN=$(openssl rand -hex 32) \ | ||
| taskito-server | ||
| ``` | ||
|
|
||
| The scheduler starts lazily, on the **first attach**. With nothing attached it | ||
| would claim jobs no executor advertises, and each would fail retryably once the | ||
| placement timeout elapsed — a retry storm against an idle deployment. | ||
|
|
||
| ## Running an executor | ||
|
|
||
| <CodeTabs> | ||
| <Tab sdk="python"> | ||
|
|
||
| ```bash | ||
| TASKITO_ATTACH=scheduler:7749 \ | ||
| TASKITO_ATTACH_TOKEN=... \ | ||
| taskito executor --app myapp:queue --slots 4 | ||
| ``` | ||
|
|
||
| </Tab> | ||
| <Tab sdk="node"> | ||
|
|
||
| ```bash | ||
| TASKITO_ATTACH=scheduler:7749 \ | ||
| TASKITO_ATTACH_TOKEN=... \ | ||
| taskito executor ./app.js --slots 4 | ||
| ``` | ||
|
|
||
| </Tab> | ||
| <Tab sdk="java"> | ||
|
|
||
| ```bash | ||
| TASKITO_ATTACH=scheduler:7749 \ | ||
| TASKITO_ATTACH_TOKEN=... \ | ||
| java -cp app.jar org.byteveda.taskito.cli.Cli executor --slots 4 | ||
| ``` | ||
|
|
||
| Handlers are discovered from the classpath through `META-INF/services`, so no | ||
| application `main` has to run to register them. | ||
|
|
||
| </Tab> | ||
| </CodeTabs> | ||
|
|
||
| `--slots` is how many jobs run at once; the scheduler dispatches a task name | ||
| only to executors that advertised it, so a handler missing from an executor is a | ||
| job that never reaches it. | ||
|
|
||
| <Callout type="warn" title="The token is read from the environment, never a flag"> | ||
| A secret in `argv` shows up in `ps` output and shell history. Every SDK reads | ||
| `TASKITO_ATTACH_TOKEN` from the environment and offers no flag for it. | ||
| </Callout> | ||
|
|
||
| ## Security | ||
|
|
||
| The attach port dispatches code, so the defaults are strict: | ||
|
|
||
| - A **non-loopback** `TASKITO_LISTEN` refuses to start without | ||
| `TASKITO_ATTACH_TOKEN`. | ||
| - The shared secret is compared in constant time, and never logged — an attach | ||
| failure is logged with the executor id and peer only. | ||
| - `hello` must arrive before any other frame; an unauthenticated socket is | ||
| dropped rather than queued, and never receives an acknowledgement. | ||
| - Use a Unix socket (`unix:/run/taskito.sock`) for a same-pod sidecar; it needs | ||
| no token because the filesystem permissions are the boundary. | ||
|
stromanni marked this conversation as resolved.
|
||
|
|
||
| <Callout type="warn"> | ||
| The token is a bearer credential, not transport security. It proves who is | ||
| attaching; it does not encrypt or integrity-protect the frames, and those frames | ||
| carry task payloads on a port that dispatches code. On anything but loopback or | ||
| a Unix socket, terminate mTLS in a proxy in front of the listener and treat the | ||
| token as the second factor. | ||
|
|
||
| `taskito-server` does not terminate TLS itself. Setting | ||
| `TASKITO_LISTEN_TLS_CERT` or `TASKITO_LISTEN_TLS_KEY` fails at startup rather | ||
| than being ignored, so a deployment cannot come up believing it is encrypted | ||
| when it is not. | ||
| </Callout> | ||
|
|
||
| ## What a task can do on an executor | ||
|
|
||
| A task body behaves the same as it does on an in-process worker. Progress, log | ||
| lines and published partials all work — the executor has no storage, so it | ||
| reports them to the scheduler, which applies them: | ||
|
|
||
| <CodeTabs> | ||
| <Tab sdk="python"> | ||
|
|
||
| ```python | ||
| @queue.task() | ||
| def resize(path: str) -> str: | ||
| current_job.update_progress(50) | ||
| current_job.log("halfway") | ||
| current_job.publish({"stage": "halfway"}) | ||
| return "done" | ||
| ``` | ||
|
|
||
| </Tab> | ||
| <Tab sdk="node"> | ||
|
|
||
| ```ts | ||
| queue.task("resize", async (path: string) => { | ||
| const job = currentJob(); | ||
| job?.setProgress(50); | ||
| job?.publish({ stage: "halfway" }); | ||
| return "done"; | ||
| }); | ||
| ``` | ||
|
|
||
| </Tab> | ||
| <Tab sdk="java"> | ||
|
|
||
| ```java | ||
| @TaskHandler("resize") | ||
| public String resize(String path) { | ||
| JobContext job = JobContext.current(); | ||
| job.setProgress(50); | ||
| job.log("halfway"); | ||
| job.publish(Map.of("stage", "halfway")); | ||
| return "done"; | ||
| } | ||
| ``` | ||
|
|
||
| </Tab> | ||
| </CodeTabs> | ||
|
|
||
| Middleware disabled from the dashboard is honoured too. The executor cannot read | ||
| settings, so the scheduler resolves the list and attaches it to each dispatch — | ||
| a toggle still takes effect on the next job, with nothing to restart. | ||
|
|
||
| ### What is not available | ||
|
|
||
| Anything that needs the database directly. An enqueue, a workflow submission or | ||
| a queue inspection raises rather than silently doing nothing, because an enqueue | ||
| that quietly vanished would be worse than one that failed. | ||
|
|
||
| A handful of job fields also arrive as zeros and nulls, because a dispatch frame | ||
| carries what running the task needs rather than the whole row: `created_at`, | ||
| `scheduled_at`, `priority`, `unique_key` and `notes`. `metadata` is the | ||
| exception — middleware reads it, so it rides the frame. A task that needs the | ||
| rest wants a worker, not an executor. | ||
|
|
||
| ## Version skew | ||
|
|
||
| The scheduler and its executors upgrade independently. The handshake carries a | ||
| capability list rather than a version both sides must match, so a newer executor | ||
| attached to an older scheduler never sends a frame that scheduler could not | ||
| parse — it degrades instead. | ||
|
|
||
| Concretely: a scheduler that does not advertise `side_channel` gets no progress | ||
| or task-log frames, and those calls become no-ops (logged once). Everything else | ||
| — dispatch, results, retries, cancellation — is unaffected. The executor logs | ||
| which capabilities it negotiated at attach. | ||
|
|
||
| ## Trade-offs | ||
|
|
||
| | Cost | Mitigation | | ||
| |---|---| | ||
| | CPU-heavy tasks compete with request latency in the app container | Cap `--slots`; or run a second replica set from the same image with the HTTP server off | | ||
| | A network hop per job | Use a Unix socket for same-pod sidecars; attach is a poor fit for microsecond-scale tasks | | ||
| | The attach port dispatches code | A Unix socket where the peer is same-pod; off loopback, proxy-terminated mTLS *and* the token, never the token alone | | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.