Skip to content

feat: upload_file tool — let MCP callers provide files for upload into page file inputs#620

Open
apakkhalin-sudo wants to merge 2 commits into
alibaba:mainfrom
apakkhalin-sudo:feat/upload-file-tool
Open

feat: upload_file tool — let MCP callers provide files for upload into page file inputs#620
apakkhalin-sudo wants to merge 2 commits into
alibaba:mainfrom
apakkhalin-sudo:feat/upload-file-tool

Conversation

@apakkhalin-sudo

Copy link
Copy Markdown

What

Implements the upload_file @todo: MCP callers can now provide local files that the agent uploads into <input type="file"> elements on the page.

How it works

  • The MCP caller offers files per task (absolute paths on the MCP host). Each file is validated and registered under an unguessable random id, and served only for the task's duration at GET /files/{id} on the existing localhost HTTP bridge — no new ports, no persistent exposure.
  • The hub registers the offered files and appends their ids to the task text; the new upload_file customTool in the extension fetches the bytes, base64-encodes them across the messaging boundary, and PageController injects them into the file input via DataTransfer + synthetic input/change events.
  • The element index may point at the visible upload button/label/dropzone — the real (often hidden) input is resolved from it, which matches how most real-world upload widgets are built (Workday/Ashby/LinkedIn-style wrappers).

Also included

f0219ab fixes a race that breaks every hub task that arrives with a config payload: onExecute awaits configure() before execute(), but setConfig re-renders useAgent and the [config] effect cleanup disposes the current agent — aborting the task it is about to run ("Task aborted" with zero LLM requests). The fix was found while developing this feature and the feature's MCP path exercises it, so it's included here; happy to split it into its own PR if you prefer.

Testing

  • Live end-to-end on Windows/Chrome: MCP execute_task with files: [...] against a page with a FileReader-backed file input — exact filename, size, and byte content read back by the page.
  • Verified the file bridge rejects unknown ids and that registrations are dropped when the task ends.

apakkhalin-sudo and others added 2 commits July 15, 2026 18:13
When the hub receives an execute message with a config payload, onExecute
awaits configure() and then calls execute(). configure()'s setConfig
re-renders useAgent, and the [config] effect cleanup disposes the current
agent — the very agent the caller is about to run (execute still resolves
agentRef to it). Its AbortController fires and every MCP task instantly
fails with 'Task aborted' (alibaba#570).

Two changes in useAgent.configure():
- Skip the agent rebuild entirely when the merged config is value-equal
  to the current one (the common MCP case: env config matches saved
  settings). Storage writes still happen.
- When config genuinely changed, resolve configure()'s promise only
  after the [config] effect has installed the new agent, so callers
  execute on the new agent instead of the disposed one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…o page file inputs

Implements the upload_file @todo. Files are offered per task by the MCP
caller (absolute paths on the MCP host), validated and registered under
unguessable random ids, and served only for the task's duration at
GET /files/{id} on the existing localhost HTTP bridge.

The hub registers offered files and appends their ids to the task text;
the new upload_file customTool (extension) fetches the bytes, base64s
them across the messaging boundary, and PageController injects them into
the file input via DataTransfer + input/change events. The index may
point at the visible upload button/label/dropzone — the real (often
hidden) input is resolved from it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 13c7703e65

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +576 to +577
const inForm = element.closest('form')?.querySelector('input[type="file"]')
if (isFileInput(inForm ?? null)) return inForm as HTMLInputElement

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid choosing the wrong file input in a form

When the indexed target is a visible upload button/dropzone inside a form with more than one file input (for example separate resume and cover-letter uploads wired by click handlers instead of labels), this fallback always returns the first input[type="file"] in the form. That silently uploads to the wrong field even though the selected index points at a different upload control; the form fallback should only be used when it is unambiguous or when a closer association can be proven.

Useful? React with 👍 / 👎.

mime: MIME_TYPES[extname(path).toLowerCase()] ?? 'application/octet-stream',
size: info.size,
}
this.#taskFiles.set(id, meta)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear task files when the hub disconnects

When a file-backed task is registered and the hub WebSocket disconnects before sending a result or error, these map entries remain because the close handler only rejects #pendingTask and never clears #taskFiles. The HTTP route then keeps serving /files/{id} after the task has ended, and even across later no-file tasks, leaving the local file readable to any client that has the id.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants