You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
But empirically both fields filter the entire merged tool catalog including custom tools registered via tools=, with no error or warning when a custom tool's name appears (or fails to appear, in the allowlist case). Custom tools just silently disappear from the model's tool catalog.
The TypedDict in session.py is silent on the built-in-vs-custom distinction:
# python/copilot/session.py L850-852, L923-925# List of tool names to allow (takes precedence over excluded_tools)available_tools: list[str]
# List of tool names to disable (ignored if available_tools is set)excluded_tools: list[str]
The two docstrings disagree, and actual behavior matches the broader (session.py) reading — all tool names are filtered, not just built-ins.
Reproduction
importasynciofromcopilotimportCopilotClientfromcopilot.clientimportSubprocessConfigfromcopilot.sessionimportPermissionHandlerfromcopilot.toolsimportTool, ToolResultasyncdefmain():
tool_called= []
asyncdeffetch_test_data(invocation):
tool_called.append(invocation.tool_name)
returnToolResult(text_result_for_llm='{"value": 42}')
custom_tool=Tool(
name="fetch_test_data",
description="Returns test data. Call this tool to get the magic value.",
parameters={"type": "object", "properties": {}},
handler=fetch_test_data,
skip_permission=True,
)
cases= [
("control: no allowlist/denylist", {}),
("available_tools=['skill'] excludes fetch_test_data", {"available_tools": ["skill"]}),
("available_tools=['skill','fetch_test_data']", {"available_tools": ["skill", "fetch_test_data"]}),
("excluded_tools=['fetch_test_data']", {"excluded_tools": ["fetch_test_data"]}),
("excluded_tools=['bash'] (real built-in)", {"excluded_tools": ["bash"]}),
]
forlabel, kwargsincases:
tool_called.clear()
client=CopilotClient(SubprocessConfig())
awaitclient.start()
try:
session=awaitclient.create_session(
on_permission_request=PermissionHandler.approve_all,
model="claude-haiku-4.5",
tools=[custom_tool],
**kwargs,
)
events= []
session.on(lambdaev: events.append(ev))
awaitsession.send("Call the fetch_test_data tool and tell me the value it returned.")
for_inrange(120):
awaitasyncio.sleep(0.5)
ifeventsandany("idle"instr(getattr(e, "type", "")) or"task_complete"instr(getattr(e, "type", "")) foreinevents[-5:]):
breakfinally:
awaitclient.stop()
print(f"[{label}] tool_called: {tool_called}")
asyncio.run(main())
The custom tool was registered, present in tools=, but the CLI hid it from the model whenever its name was missing from available_tools or present in excluded_tools — exactly as the merged-catalog reading predicts. Built-in-only filtering would leave the custom tool visible in all five cases.
Impact
This is silent and hard to debug:
No error or warning when a registered custom tool is omitted from available_tools or named in excluded_tools.
The model just doesn't have access to the tool; it falls back to other available tools or generic responses.
Looks identical to "the model decided not to use this tool."
Easy to overlook because client.py's docstring gives no hint that custom tools are subject to the filter.
There's a hidden landmine in the excluded_tools direction too: if a custom tool's name happens to collide with a built-in (bash, glob, view, etc.), denying the built-in silently denies the custom tool as well, with no warning.
We hit the available_tools direction trying to comply with an InfoSec recommendation that requires allowlist over blocklist. On a harness with ~40 in-process custom tools, naively setting available_tools=[<built-ins we want>] made every custom tool invisible. Trajectory score on our eval went from 1.0 (denylist) → 0.0 (allowlist with the same effective built-in catalog) on the same prompt because the model couldn't see its real tools.
Suggested fix
Either:
Make the docstrings match behavior. Update both client.py docstrings and session.py comments to make clear the filter applies to all tool names — built-in and custom registered — and that the application is responsible for explicitly enumerating its custom tool names in any allowlist.
Make the behavior match the (client.py) docstring. Custom tools registered via tools= should always be available regardless of available_tools/excluded_tools, since the application explicitly opted them in by passing them. Both fields would then filter only the CLI's built-in catalog. This is the more useful semantic — it lets applications build precise built-in policies without having to enumerate every custom tool name.
CopilotClient.create_session/resume_sessiondocumentavailable_toolsandexcluded_toolsas filtering built-in tools only:But empirically both fields filter the entire merged tool catalog including custom tools registered via
tools=, with no error or warning when a custom tool's name appears (or fails to appear, in the allowlist case). Custom tools just silently disappear from the model's tool catalog.The TypedDict in
session.pyis silent on the built-in-vs-custom distinction:The two docstrings disagree, and actual behavior matches the broader (
session.py) reading — all tool names are filtered, not just built-ins.Reproduction
Output:
The custom tool was registered, present in
tools=, but the CLI hid it from the model whenever its name was missing fromavailable_toolsor present inexcluded_tools— exactly as the merged-catalog reading predicts. Built-in-only filtering would leave the custom tool visible in all five cases.Impact
This is silent and hard to debug:
available_toolsor named inexcluded_tools.client.py's docstring gives no hint that custom tools are subject to the filter.There's a hidden landmine in the
excluded_toolsdirection too: if a custom tool's name happens to collide with a built-in (bash,glob,view, etc.), denying the built-in silently denies the custom tool as well, with no warning.We hit the
available_toolsdirection trying to comply with an InfoSec recommendation that requires allowlist over blocklist. On a harness with ~40 in-process custom tools, naively settingavailable_tools=[<built-ins we want>]made every custom tool invisible. Trajectory score on our eval went from 1.0 (denylist) → 0.0 (allowlist with the same effective built-in catalog) on the same prompt because the model couldn't see its real tools.Suggested fix
Either:
Make the docstrings match behavior. Update both
client.pydocstrings andsession.pycomments to make clear the filter applies to all tool names — built-in and custom registered — and that the application is responsible for explicitly enumerating its custom tool names in any allowlist.Make the behavior match the (
client.py) docstring. Custom tools registered viatools=should always be available regardless ofavailable_tools/excluded_tools, since the application explicitly opted them in by passing them. Both fields would then filter only the CLI's built-in catalog. This is the more useful semantic — it lets applications build precise built-in policies without having to enumerate every custom tool name.At minimum, emit a warning when
available_toolsis provided and a registered custom tool's name is not in the list, or whenexcluded_toolsnames a tool that's both a built-in and a registered custom tool. Similar to what Documentation missing: MCP tools are prefixed with <server-key>- in available_tools / custom_agents[].tools #869 suggests for MCP tool prefixing.Related
available_toolsfilters more than the docstring suggests, this time for MCP tools whose names need a<server-key>-prefix.CustomAgentConfig.toolswhitelist behavior for sub-agents.Versions
github-copilot-sdk0.3.0 with bundled CLI 1.0.36-0. Reproduced 2026-04-30.