Skip to content

Enhance agent features and improve clientIO functionality#2631

Open
robgruen wants to merge 48 commits into
mainfrom
dev/robgruen/dogfooding3
Open

Enhance agent features and improve clientIO functionality#2631
robgruen wants to merge 48 commits into
mainfrom
dev/robgruen/dogfooding3

Conversation

@robgruen

@robgruen robgruen commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

This pull request introduces a new ServiceActionHandler for Windows service control commands, adds comprehensive unit tests for service-related features, and integrates the service control functionality into the action dispatcher and handler registration. The main changes establish the ability to handle service restart actions, including scenarios for fuzzy matching, elevation, and user confirmation, with full test coverage.

Added missing functionality from previous chat-ui refactor

  • Added back file upload button
  • Added speech button
  • Added camera button
  • Added history delineation
  • Allowed for queuing messages while dispatcher is loading
  • Allowed for chaining choice questions together.
  • Implemented missing clientIO interface methods in vscode-shell

New Service Control Handler and Integration:

  • Added ServiceActionHandler to handle Windows service control actions (e.g., RestartService), including logic for fuzzy matching, elevation, and user confirmation. (dotnet/autoShell/Handlers/ServiceActionHandler.cs)
  • Registered ServiceActionHandler in the action dispatcher factory and handler registration, and updated dispatcher and test constructors to include the new IServiceControlService dependency. (dotnet/autoShell/ActionDispatcher.cs, dotnet/autoShell.Tests/HandlerRegistrationTests.cs, dotnet/autoShell.Tests/ActionDispatcherIntegrationTests.cs) [1] [2] [3] [4] [5] [6] [7]
image

Unit Tests for Service Control:

  • Added ServiceActionHandlerTests to validate all major behaviors of the new handler, including matching by name/description, handling fuzzy matches, elevation, error cases, and confirmation payloads. (dotnet/autoShell.Tests/ServiceActionHandlerTests.cs)
  • Added ServiceMatcherTests to thoroughly test the service matching logic for various scenarios (exact, fuzzy, description-based, and fallback cases). (dotnet/autoShell.Tests/ServiceMatcherTests.cs)

@robgruen robgruen marked this pull request as ready for review July 9, 2026 02:00
Copilot AI review requested due to automatic review settings July 9, 2026 02:00
Comment thread ts/packages/vscode-shell/src/webview/main.ts Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands TypeAgent’s end-to-end “client ↔ dispatcher ↔ agent-server ↔ agents ↔ autoShell” capabilities by restoring missing chat-ui features (attachments, speech, interactions, history separation, dynamic displays, feedback/delete affordances, pre-init send queue) and adding a new Windows Service control action (restart service) with matching + elevation flows and tests.

Changes:

  • Added Windows service restart support (schema + desktop agent logic + new autoShell service-control implementation + handler registration + .NET tests).
  • Restored/implemented missing VS Code shell clientIO/webview bridging features: attachments, Azure Speech mic, dynamic displays, server-driven interactions/choices, user feedback syncing, developer-mode delete UI, and queuing sends while the dispatcher/session initializes.
  • Added developer-mode translation capture plumbing (DevTrace) and a new internal npm registry bootstrap script (getNPMRC).
Show a summary per file
File Description
ts/tools/scripts/getNPMRC.mjs New script to pull/push repo .npmrc from Key Vault and set pnpm tokenHelper auth.
ts/tools/scripts/getNPMRC.config.json Vault/secret/path configuration for getNPMRC.
ts/pnpm-workspace.yaml Adds more onlyBuiltDependencies entries for pnpm builds.
ts/packages/vscode-shell/src/webview/vscode-theme.css VS Code shell theme overlay tweaks (agent icon styling).
ts/packages/vscode-shell/src/webview/main.ts Webview wiring for attachments, speech token flow, interactions, dynamic displays, feedback, and history separator.
ts/packages/vscode-shell/src/webview/azureSpeechProvider.ts Azure Speech-backed SpeechInputProvider for VS Code webview.
ts/packages/vscode-shell/src/bridge/messages.ts Expands bridge message contracts (attachments, choices, interactions, feedback, speech token).
ts/packages/vscode-shell/src/bridge/clientIO.ts Implements previously-no-op ClientIO methods by forwarding to webview (choices/interactions/dynamic display/feedback/dev mode).
ts/packages/vscode-shell/src/agentServerBridge.ts Queues sends until session is ready; adds developer-mode hydration; bridges new messages (choice/interaction/feedback/speech token/dynamic display).
ts/packages/vscode-shell/package.json Adds Speech SDK dependency for VS Code shell webview.
ts/packages/shell/src/renderer/src/chatPanelBridge.ts Aligns Electron shell behavior: history separator label import, pre-init send queue, dev-mode delete hook, dev-mode seeding.
ts/packages/dispatcher/types/src/dispatcher.ts Adds getDeveloperMode() to Dispatcher interface.
ts/packages/dispatcher/rpc/test/dispatcherRpc.spec.ts Updates stub dispatcher for new getDeveloperMode() method.
ts/packages/dispatcher/rpc/src/dispatcherTypes.ts Adds RPC surface for getDeveloperMode.
ts/packages/dispatcher/rpc/src/dispatcherServer.ts Implements getDeveloperMode RPC endpoint.
ts/packages/dispatcher/rpc/src/dispatcherClient.ts Adds getDeveloperMode RPC client method.
ts/packages/dispatcher/dispatcher/test/confirmTranslation.spec.ts Ensures confirm prompt is gated by confirmActions (not developerMode).
ts/packages/dispatcher/dispatcher/test/chainedChoice.spec.ts Regression test: choice callback returning a new pending choice must render the second card.
ts/packages/dispatcher/dispatcher/src/translation/interpretRequest.ts Adds DevTrace begin + translation capture persistence.
ts/packages/dispatcher/dispatcher/src/translation/confirmTranslation.ts Changes confirmation gating to confirmActions instead of developerMode.
ts/packages/dispatcher/dispatcher/src/dispatcher.ts Uses emitActionResult for choice callback results (enables chained pendingChoice rendering); adds getDeveloperMode.
ts/packages/dispatcher/dispatcher/src/context/system/handlers/configCommandHandlers.ts Implements @config dev on [--confirm] and @config dev off + client notifications.
ts/packages/dispatcher/dispatcher/src/context/devTrace.ts New DevTrace capture implementation writing dev-captures/translate-*.json.
ts/packages/dispatcher/dispatcher/src/context/commandHandlerContext.ts Adds confirmActions, devTrace, dispatcher option developerMode, and hooks prompt logging to DevTrace.
ts/packages/chat-ui/test/chatPanel.spec.ts Updates tests for removal of feedback “trash/hide” hook from ChatPanel.
ts/packages/chat-ui/styles/chat.css Adds styling for developer-mode per-message delete split button and menu.
ts/packages/chat-ui/src/webSpeechProvider.ts New default Web Speech API provider for browser hosts when no host speech provider is injected.
ts/packages/chat-ui/src/index.ts Exports formatHistorySeparatorLabel.
ts/packages/chat-ui/src/icons.ts Adds chevron-down icon for split-button menu.
ts/packages/chat-ui/src/feedbackWidget.ts Removes the old trash/hide affordance from feedback widget.
ts/packages/chat-ui/src/deleteControl.ts New developer-mode delete split-button + menu implementation.
ts/packages/chat-ui/src/chatPanel.ts Restores attach button w/ web-native picker, default speech provider, developer-mode delete affordance, and exports history separator label helper.
ts/packages/agentServer/server/src/speechToken.ts Implements server-side Azure Speech token vending w/ caching.
ts/packages/agentServer/server/src/server.ts Adds --dev startup option to enable developer mode by default.
ts/packages/agentServer/server/src/connectionHandler.ts Exposes getSpeechToken on the agent-server RPC surface.
ts/packages/agentServer/server/package.json Adds start:dev script to start server with --dev.
ts/packages/agentServer/protocol/src/protocol.ts Adds SpeechToken type + getSpeechToken() RPC in protocol.
ts/packages/agentServer/protocol/src/index.ts Exports SpeechToken.
ts/packages/agentServer/client/test/conversation-stubConnection.ts Updates stub connection with getSpeechToken.
ts/packages/agentServer/client/src/index.ts Re-exports SpeechToken type from protocol.
ts/packages/agentServer/client/src/agentServerClient.ts Adds getSpeechToken() client method.
ts/packages/agents/github-cli/test/githubCliBuildArgs.spec.ts Adds tests for mapping “unassigned” assignee requests to --search no:assignee.
ts/packages/agents/github-cli/src/github-cliSchema.ts Documents assignee option for issue/pr list actions.
ts/packages/agents/github-cli/src/github-cliSchema.agr Adds grammar for “unassigned issues …” mapping to assignee: "none".
ts/packages/agents/github-cli/src/github-cliActionHandler.ts Implements unassigned assignee sentinel mapping and exports buildArgs for tests.
ts/packages/agents/desktop/src/desktopSchema.agr Adds new service control grammar rules mapping to RestartService.
ts/packages/agents/desktop/src/actionsSchema.ts Adds RestartService action schema + parameters (matchBy/elevate).
ts/packages/agents/desktop/src/actionHandler.ts Adds choice-based confirmation + elevation flows for service restart results from autoShell.
ts/packages/agents/browser/src/extension/serviceWorker/dispatcherConnection.ts Adds getSpeechToken to browser extension’s agent-server adapter.
ts/package.json Adds getNPMRC and start:agent-server:dev scripts; updates onlyBuiltDependencies.
ts/.vscode/tasks.json Adds task for installing the VS Code chat extension locally.
ts/.vscode/settings.json Improves watcher/search excludes; sets a server start command.
ts/.vscode/extensions.json Recommends the typeagent.vscode-chat extension.
ts/.gitignore Ignores root /.npmrc and config.local.yaml.bak.
dotnet/autoShell/Services/WindowsServiceControlService.cs Implements service enumeration/matching + restart, with elevation fallback.
dotnet/autoShell/Services/ServiceMatcher.cs Pure matching logic (exact/fuzzy, description coverage) for services.
dotnet/autoShell/Services/IServiceControlService.cs Defines service control abstraction and result contract.
dotnet/autoShell/Handlers/ServiceActionHandler.cs Adds RestartService action handler returning confirm/elevation payloads.
dotnet/autoShell/autoShell.csproj Adds ServiceController package dependency.
dotnet/autoShell.Tests/ServiceMatcherTests.cs Unit tests for ServiceMatcher exact/fuzzy/description matching behavior.
dotnet/autoShell.Tests/ServiceActionHandlerTests.cs Unit tests for handler behavior incl. confirm/elevation payloads.
dotnet/autoShell.Tests/HandlerRegistrationTests.cs Registers new handler in test handler set.
dotnet/autoShell.Tests/ActionDispatcherIntegrationTests.cs Updates dispatcher construction for new service-control dependency.
dotnet/autoShell/ActionDispatcher.cs Wires WindowsServiceControlService + ServiceActionHandler into dispatcher creation.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 64/65 changed files
  • Comments generated: 5
  • Review effort level: Low

Comment thread ts/tools/scripts/getNPMRC.mjs Outdated
Comment thread ts/packages/chat-ui/src/deleteControl.ts
Comment thread ts/packages/vscode-shell/src/webview/vscode-theme.css Outdated
Comment thread ts/packages/vscode-shell/src/webview/main.ts Outdated
Comment thread dotnet/autoShell/Handlers/ServiceActionHandler.cs
Copilot AI review requested due to automatic review settings July 9, 2026 02:05
Comment thread ts/packages/agentServer/server/src/speechToken.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 64/65 changed files
  • Comments generated: 6
  • Review effort level: Low

Comment thread ts/tools/scripts/getNPMRC.mjs Outdated
Comment thread ts/tools/scripts/getNPMRC.mjs Outdated
Comment thread ts/tools/scripts/getNPMRC.config.json
Comment thread ts/packages/vscode-shell/src/webview/vscode-theme.css
Comment thread ts/packages/chat-ui/src/chatPanel.ts
Comment thread ts/.vscode/settings.json Outdated
Comment thread ts/packages/vscode-shell/src/webview/azureSpeechProvider.ts
…method call'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 9, 2026 02:53
@robgruen

robgruen commented Jul 9, 2026 via email

Copy link
Copy Markdown
Collaborator Author

…, webview templateServices, and promptLogger override
@curtisman

Copy link
Copy Markdown
Member

Sure

@robgruen robgruen added this pull request to the merge queue Jul 9, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to Branch Protection failures Jul 9, 2026
You're not authorized to push to this branch. Visit "About protected branches" for more information.
@robgruen robgruen enabled auto-merge July 10, 2026 04:32
@robgruen robgruen added this pull request to the merge queue Jul 10, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to Branch Protection failures Jul 10, 2026
You're not authorized to push to this branch. Visit "About protected branches" for more information.
@robgruen robgruen enabled auto-merge July 10, 2026 05:57
@robgruen robgruen added this pull request to the merge queue Jul 10, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to Branch Protection failures Jul 10, 2026
You're not authorized to push to this branch. Visit "About protected branches" for more information.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants