Summary
Could XcodeBuildMCP add AX readiness polling/retry behavior when snapshot_ui / describe-ui returns the known empty hierarchy shape on iOS simulators?
This is a follow-up to #290 and the investigation in #312. I'm not asking to revive the proactive defaults write approach, since the maintainer investigation showed that early writes to com.apple.Accessibility can make the simulator AX state worse. This request is specifically for a readiness/recovery layer around the UI automation tools.
Background
In #290, the reported failure shape was:
- a bare
AXApplication node
- zero-size frame, e.g.
{{0, 0}, {0, 0}}
children: []
- tool result still looks like a successful capture, even though no usable UI hierarchy is available
The maintainer follow-up in #312 found that this is most likely an AX daemon / CoreSimulator readiness timing issue rather than the accessibility defaults being the direct gate. The comment also mentioned that XcodeBuildMCP may want to add a polling readiness probe later.
I hit a related practical variant in a downstream project workflow: the app had been launched while simulator AX introspection was unavailable, and snapshot_ui kept returning the empty hierarchy until the app process was relaunched. After relaunch, the same XcodeBuildMCP snapshot_ui call returned a populated hierarchy with the expected app frame and elements.
That makes the current behavior difficult for agent workflows: the tool reports SUCCEEDED, but the hierarchy is effectively unusable, so the agent has to infer whether the app is blank, accessibility metadata is missing, the simulator AX bus is not ready, or the app needs relaunch/reboot.
Proposed enhancement
When a UI hierarchy capture returns the known empty AX shape, XcodeBuildMCP could treat it as "AX not ready" rather than an ordinary successful empty app tree.
Possible behavior:
- Detect the suspicious shape:
- root application node only, or no meaningful descendants
- root frame is zero-size or otherwise invalid
children: []
- Poll/retry for a bounded period, e.g. 15-30 seconds, before returning the final result.
- If still empty, return an actionable diagnostic in the tool result, such as:
- AX hierarchy is unavailable / simulator AX service may not be ready
- try relaunching the app
- if persistent, try
simctl shutdown <UDID> && simctl boot <UDID>
- Optionally, for tools that already own launch/build-run flow, consider relaunching the target app once after the readiness timeout before failing/returning the diagnostic.
Why this would help
This keeps the fix aligned with the findings in #312:
- no proactive accessibility flag writes during the risky startup window
- no assumption that app-side accessibility metadata is broken
- clearer distinction between "the app has no accessible UI" and "the simulator AX pipeline is not ready"
- more robust behavior for fresh simulator boots and agent-driven UI verification
Related
Summary
Could XcodeBuildMCP add AX readiness polling/retry behavior when
snapshot_ui/describe-uireturns the known empty hierarchy shape on iOS simulators?This is a follow-up to #290 and the investigation in #312. I'm not asking to revive the proactive
defaults writeapproach, since the maintainer investigation showed that early writes tocom.apple.Accessibilitycan make the simulator AX state worse. This request is specifically for a readiness/recovery layer around the UI automation tools.Background
In #290, the reported failure shape was:
AXApplicationnode{{0, 0}, {0, 0}}children: []The maintainer follow-up in #312 found that this is most likely an AX daemon / CoreSimulator readiness timing issue rather than the accessibility defaults being the direct gate. The comment also mentioned that XcodeBuildMCP may want to add a polling readiness probe later.
I hit a related practical variant in a downstream project workflow: the app had been launched while simulator AX introspection was unavailable, and
snapshot_uikept returning the empty hierarchy until the app process was relaunched. After relaunch, the same XcodeBuildMCPsnapshot_uicall returned a populated hierarchy with the expected app frame and elements.That makes the current behavior difficult for agent workflows: the tool reports
SUCCEEDED, but the hierarchy is effectively unusable, so the agent has to infer whether the app is blank, accessibility metadata is missing, the simulator AX bus is not ready, or the app needs relaunch/reboot.Proposed enhancement
When a UI hierarchy capture returns the known empty AX shape, XcodeBuildMCP could treat it as "AX not ready" rather than an ordinary successful empty app tree.
Possible behavior:
children: []simctl shutdown <UDID> && simctl boot <UDID>Why this would help
This keeps the fix aligned with the findings in #312:
Related