Skip to content

feat(node): report executor progress, logs and toggles over the side-channel - #601

Merged
kartikeya-27 merged 8 commits into
ByteVeda:masterfrom
stromanni:feat/589-side-channel-node
Aug 2, 2026
Merged

feat(node): report executor progress, logs and toggles over the side-channel#601
kartikeya-27 merged 8 commits into
ByteVeda:masterfrom
stromanni:feat/589-side-channel-node

Conversation

@stromanni

@stromanni stromanni commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Part 3 of #589, on top of #599 (core) and #600 (Python). Same shape as the Python side: a task running on an attached Node executor gets its progress, its published partials and its dashboard middleware toggles back, without the executor holding a database connection. Java follows in its own PR.

What changes

crates/taskito-node/src/executor.rs — the binding

JsExecutor holds the session's ExecutorSideChannel and exposes four methods to JS: supportsSideChannel, reportProgress, writeTaskLog and disabledMiddleware. All fire-and-forget — nothing blocks the calling task, nothing fails its job, and nothing is sent at all when the scheduler advertised no side_channel capability. supportsSideChannel exists so the shell can say so once rather than silently dropping a progress bar.

sdks/node/src/task-callback.ts — the seams

setProgress and writeTaskLog join isCancelled as optional overrides. Each defaults to the storage-backed path a worker uses, and an executor substitutes its own. That keeps one task-callback implementation for both, rather than a second copy that drifts.

middlewareFor now takes the job id as well as the task name. An attached executor resolves disables per dispatch — the scheduler attaches the list to the job frame — where a worker resolves them per task name from storage. The worker's implementation ignores the extra argument, with a comment saying why.

sdks/node/src/executor.ts and queue.ts — the wiring

ExecutorStartParams.middlewareFor takes the disable list rather than reading one, and runExecutor supplies the chain-minus-disabled directly instead of constructing a MiddlewareDisableStore it has no storage to query. Progress and task logs route through the attached executor.

Tests

  • sends progress and logs to a scheduler that advertised the side-channel
  • skips a middleware the dispatch says is disabled
  • puts progress and published partials into storage via the scheduler
  • honours a dashboard middleware toggle on an attached executor

The last two are in executorAttachServer.test.ts, gated on TASKITO_SERVER_BIN: they run a real taskito-server and assert on the rows that land in storage rather than on frames.

Verification

cargo test --workspace, cargo clippy --all-targets --all-features -- -D warnings and cargo fmt --check are clean. On the Node side, build:native, build:ts, build:dashboard, pnpm typecheck and pnpm lint are clean, and pnpm test passes 650 with 6 skipped. The six TASKITO_SERVER_BIN-gated tests, which CI skips, were run separately against a real server three times — 6/6 each.

Refs #589.

Summary by CodeRabbit

  • New Features

    • Added executor support for reporting task progress and writing task logs.
    • Added support for publishing partial task results through the scheduler.
    • Added per-job middleware disabling for attached executors.
  • Bug Fixes

    • Improved middleware selection to respect scheduler-provided job settings.
    • Added fallback behavior when side-channel capabilities are unavailable.
    • Improved callback handling during executor startup.
  • Tests

    • Added coverage for progress reporting, task logs, partial results, and middleware disabling.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@stromanni, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 57 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c3b811be-8881-48cb-b838-ec70bb1de514

📥 Commits

Reviewing files that changed from the base of the PR and between 521775e and 9103b5f.

📒 Files selected for processing (3)
  • .github/actions/provision-gradle/action.yml
  • .github/workflows/ci-java.yml
  • sdks/python/tests/core/test_shutdown.py
📝 Walkthrough

Walkthrough

JsExecutor now supports scheduler side-channel operations for progress, task logs, and disabled middleware. Node callbacks use these operations after attachment, while queue fallbacks preserve unsupported behavior. Tests cover capability negotiation, dispatch timing, frame delivery, middleware filtering, and CLI startup timing.

Changes

Executor side-channel integration

Layer / File(s) Summary
Side-channel callback wiring
crates/taskito-node/src/executor.rs, sdks/node/src/executor.ts, sdks/node/src/task-callback.ts
JsExecutor stores the scheduler side channel and exposes reporting and middleware APIs. Node task callbacks wait for native attachment and use injected handlers with queue fallbacks.
Per-job middleware dispatch
sdks/node/src/queue.ts, sdks/node/src/task-callback.ts, sdks/node/src/worker.ts
Middleware resolution receives the job-specific disabled list. Queue selection filters registered middleware. Worker documentation records task-name-based checks.
Attach protocol and integration validation
sdks/node/test/worker/executorAttach.test.ts, sdks/node/test/worker/executorAttachServer.test.ts, sdks/node/test/integrations/cli.test.ts
Tests cover capability negotiation, disabled middleware transmission, side-channel frame decoding, degraded behavior, same-tick dispatch, real-scheduler persistence, and longer CLI startup timing.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related issues

Possibly related PRs

Suggested labels: node, enhancement

Suggested reviewers: kartikeya-27

Sequence Diagram(s)

sequenceDiagram
  participant TaskCallback
  participant JsExecutor
  participant ExecutorSideChannel
  participant Scheduler
  participant Storage
  TaskCallback->>JsExecutor: report progress or write task log
  JsExecutor->>ExecutorSideChannel: send side-channel event
  ExecutorSideChannel->>Scheduler: forward event
  Scheduler->>Storage: persist progress or task log
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: Node executors report progress, logs, and toggles through the side-channel.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@sdks/node/src/executor.ts`:
- Around line 104-115: Gate dispatcher job processing until the executor
attachment is assigned: update the startup flow around start_executor and the
callbacks middlewareFor, isCancelled, setProgress, and writeTaskLog so no job
can run while attached is undefined. Add a native-ready barrier or otherwise
sequence attach completion before dispatch, and cover immediate dispatch after
attach while preserving the existing callback behavior once attached is
available.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bedeebd2-7cfc-4e90-a98f-02819a2a44bd

📥 Commits

Reviewing files that changed from the base of the PR and between 33722b0 and f0870cc.

📒 Files selected for processing (7)
  • crates/taskito-node/src/executor.rs
  • sdks/node/src/executor.ts
  • sdks/node/src/queue.ts
  • sdks/node/src/task-callback.ts
  • sdks/node/src/worker.ts
  • sdks/node/test/worker/executorAttach.test.ts
  • sdks/node/test/worker/executorAttachServer.test.ts
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • ByteVeda/taskito (manual)

Comment thread sdks/node/src/executor.ts
@kartikeya-27

Copy link
Copy Markdown
Contributor

@CodeRabbit full review

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@kartikeya-27
kartikeya-27 merged commit 99847e2 into ByteVeda:master Aug 2, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants