Skip to content

available_tools and excluded_tools filter custom tools registered via tools=, contradicting client.py docstrings #1179

@loganrosen

Description

@loganrosen

CopilotClient.create_session / resume_session document available_tools and excluded_tools as filtering built-in tools only:

# python/copilot/client.py L1241, L1533
available_tools: Allowlist of built-in tools to enable.
excluded_tools: List of built-in tools to disable.

But empirically both fields filter the entire merged tool catalog including custom tools registered via tools=, with no error or warning when a custom tool's name appears (or fails to appear, in the allowlist case). Custom tools just silently disappear from the model's tool catalog.

The TypedDict in session.py is silent on the built-in-vs-custom distinction:

# python/copilot/session.py L850-852, L923-925
# List of tool names to allow (takes precedence over excluded_tools)
available_tools: list[str]
# List of tool names to disable (ignored if available_tools is set)
excluded_tools: list[str]

The two docstrings disagree, and actual behavior matches the broader (session.py) reading — all tool names are filtered, not just built-ins.

Reproduction

import asyncio
from copilot import CopilotClient
from copilot.client import SubprocessConfig
from copilot.session import PermissionHandler
from copilot.tools import Tool, ToolResult


async def main():
    tool_called = []

    async def fetch_test_data(invocation):
        tool_called.append(invocation.tool_name)
        return ToolResult(text_result_for_llm='{"value": 42}')

    custom_tool = Tool(
        name="fetch_test_data",
        description="Returns test data. Call this tool to get the magic value.",
        parameters={"type": "object", "properties": {}},
        handler=fetch_test_data,
        skip_permission=True,
    )

    cases = [
        ("control: no allowlist/denylist", {}),
        ("available_tools=['skill'] excludes fetch_test_data", {"available_tools": ["skill"]}),
        ("available_tools=['skill','fetch_test_data']", {"available_tools": ["skill", "fetch_test_data"]}),
        ("excluded_tools=['fetch_test_data']", {"excluded_tools": ["fetch_test_data"]}),
        ("excluded_tools=['bash'] (real built-in)", {"excluded_tools": ["bash"]}),
    ]

    for label, kwargs in cases:
        tool_called.clear()
        client = CopilotClient(SubprocessConfig())
        await client.start()
        try:
            session = await client.create_session(
                on_permission_request=PermissionHandler.approve_all,
                model="claude-haiku-4.5",
                tools=[custom_tool],
                **kwargs,
            )
            events = []
            session.on(lambda ev: events.append(ev))
            await session.send("Call the fetch_test_data tool and tell me the value it returned.")
            for _ in range(120):
                await asyncio.sleep(0.5)
                if events and any("idle" in str(getattr(e, "type", "")) or "task_complete" in str(getattr(e, "type", "")) for e in events[-5:]):
                    break
        finally:
            await client.stop()
        print(f"[{label}] tool_called: {tool_called}")


asyncio.run(main())

Output:

[control: no allowlist/denylist] tool_called: ['fetch_test_data']
[available_tools=['skill'] excludes fetch_test_data] tool_called: []
[available_tools=['skill','fetch_test_data']] tool_called: ['fetch_test_data']
[excluded_tools=['fetch_test_data']] tool_called: []
[excluded_tools=['bash'] (real built-in)] tool_called: ['fetch_test_data']

The custom tool was registered, present in tools=, but the CLI hid it from the model whenever its name was missing from available_tools or present in excluded_tools — exactly as the merged-catalog reading predicts. Built-in-only filtering would leave the custom tool visible in all five cases.

Impact

This is silent and hard to debug:

  • No error or warning when a registered custom tool is omitted from available_tools or named in excluded_tools.
  • The model just doesn't have access to the tool; it falls back to other available tools or generic responses.
  • Looks identical to "the model decided not to use this tool."
  • Easy to overlook because client.py's docstring gives no hint that custom tools are subject to the filter.

There's a hidden landmine in the excluded_tools direction too: if a custom tool's name happens to collide with a built-in (bash, glob, view, etc.), denying the built-in silently denies the custom tool as well, with no warning.

We hit the available_tools direction trying to comply with an InfoSec recommendation that requires allowlist over blocklist. On a harness with ~40 in-process custom tools, naively setting available_tools=[<built-ins we want>] made every custom tool invisible. Trajectory score on our eval went from 1.0 (denylist) → 0.0 (allowlist with the same effective built-in catalog) on the same prompt because the model couldn't see its real tools.

Suggested fix

Either:

  1. Make the docstrings match behavior. Update both client.py docstrings and session.py comments to make clear the filter applies to all tool names — built-in and custom registered — and that the application is responsible for explicitly enumerating its custom tool names in any allowlist.

  2. Make the behavior match the (client.py) docstring. Custom tools registered via tools= should always be available regardless of available_tools/excluded_tools, since the application explicitly opted them in by passing them. Both fields would then filter only the CLI's built-in catalog. This is the more useful semantic — it lets applications build precise built-in policies without having to enumerate every custom tool name.

  3. At minimum, emit a warning when available_tools is provided and a registered custom tool's name is not in the list, or when excluded_tools names a tool that's both a built-in and a registered custom tool. Similar to what Documentation missing: MCP tools are prefixed with <server-key>- in available_tools / custom_agents[].tools #869 suggests for MCP tool prefixing.

Related

Versions

github-copilot-sdk 0.3.0 with bundled CLI 1.0.36-0. Reproduced 2026-04-30.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions