feat: Agent deployment compatibility validation — server-side checks with auto-fix offers

[vybe]

Summary

When users deploy agents to Trinity that weren't created following Trinity best practices (no playbooks, missing YAML files, .claude/ excluded from .gitignore, no template.yaml), the agent may silently fail at runtime in ways that are hard to diagnose. Trinity should run server-side compatibility checks on deployment and surface actionable recommendations — without blocking the deployment.

Context

Raised in the 2026-05-05 dev catch-up after client experience with Paradigm Life: agents deployed without playbooks or required YAML files diverged from their local behavior on Trinity. Andrii proposed server-side checks; Eugene framed this as hard requirements (e.g., gitignore) vs soft recommendations (e.g., dashboard.yaml). The team agreed on a non-enforcing, recommendation-based approach with optional auto-fix for correctable issues.

Validation Spec

The full check list (100 checks, 11 categories) is defined in docs/agent-validation-spec.md — the canonical source of truth for what gets validated and how.

Spec updated since this issue was opened (2026-05-08): added file-structure checks F-011/F-012/F-013 (ARCHITECTURE.md / requirements.md / CHANGELOG.md) and a whole new category I — Composability (I-001…I-005) plus the data-exchange principle. Counts below reflect the current spec, not the original 56/10 framing.

Check categories:

• F — File & directory structure (13 checks)
• S — Security / secret hygiene (10 checks)
• T — template.yaml schema & content quality (17 checks)
• C — CLAUDE.md quality (12 checks)
• K — Credential configuration consistency (5 checks)
• G — Git configuration hygiene (5 checks)
• P — Skill & playbook construction (12 checks)
• A — Autonomy & scheduling design (5 checks)
• D — Dashboard & metrics configuration (8 checks)
• X — Cross-file consistency (8 checks)
• I — Composability / data-exchange contract (5 checks)

Checks marked [AI] are evaluated by an LLM reading file contents (e.g., "does this CLAUDE.md contain domain-specific instructions?", "does this autonomous skill contain approval gates?"). Checks marked [STATIC] use deterministic file/pattern analysis.

Acceptance Criteria

• On agent creation/deployment (GitHub template clone), Trinity runs a compatibility check against the agent workspace
• Checks are split into hard issues (will likely break Trinity) and soft recommendations (best practices), plus info suggestions — see spec for severity per check
• Results are surfaced in the Overview tab of the agent detail view (OverviewPanel.vue) — as a dedicated section reusing the existing "Needs attention" pattern (count + link, hidden when clean), expandable to the full checklist. Not the Info tab (see Technical Notes).
• For correctable hard issues (e.g., gitignore patterns), Trinity offers an auto-fix button — 10 checks are auto-fixable (all gitignore-related, listed in spec)
• Deployment is NOT blocked — agents with issues still start
• The check can be re-run on demand from the Overview tab

API

• GET /api/agents/{name}/compatibility → {hard: [...], soft: [...], info: [...]} check results
• POST /api/agents/{name}/compatibility/fix → {check_id} body, applies auto-fix for eligible checks

MCP Tool

• get_agent_compatibility_report(agent_name) → full report with check results and recommendations

Technical Notes

• Static checks run via docker exec into the agent container (same pattern as git_service.py and ssh_service.py)
• AI-evaluated checks call the Claude API with relevant file contents; results include confidence score and explanation
• Results are transient (no new DB table) — re-run on demand
• Frontend target moved (feat(ui): Agent Detail Overview dashboard as default tab + Info tab redesign #1107, refactor(ui): unify Health + Ops + Executions into a single Operations nav area #1109): the Info tab was redesigned into components/InfoPanel.vue and a new Overview tab (components/OverviewPanel.vue) is now the default landing tab on Agent Detail. The compatibility surface belongs on Overview — it already owns the "needs attention → link out" idiom (section 2 of the panel, hidden at zero). Wire the new section there; do not target the old inline AgentDetail.vue info tab.
• Related / prior art — bug: deploy_local_agent (no GitHub repo) silently creates empty agents — fallback path writes outside host-mapped volume #950 / fix(deploy-local): require usable CLAUDE.md + advisory MCP credential-gap warnings (#950) #982 (shipped 2026-06-02): fix(deploy-local): require usable CLAUDE.md + advisory MCP credential-gap warnings already implements a slice of this at deploy-local time (hard-fail on unusable CLAUDE.md, advisory MCP credential-gap warnings) in services/template_service.py. That overlaps spec checks C-001/C-002 and K-001/K-002 — the compatibility service should reuse/consolidate this logic, not re-implement it.
• Related: /trinity:onboard skill in abilityai/abilities has the canonical list of Trinity-compatible agent requirements

Alignment with Target Architecture

This feature is the advisory front-end for several agent-side contracts in docs/planning/TARGET_ARCHITECTURE.md:

• Governing Principle security: Fix token logging and add HTML reports to gitignore #7 — "Data exchange over conversation chains." The Composability category (I-001…I-005) operationalizes this principle at validation time: it checks that agents declare an output format/schema, write structured file-based output, and document their input/output interface rather than relying on conversation chaining. The validator nudges templates toward the composition pattern the target architecture treats as the default.
• ~/.trinity/post-check hook (Agent Runtime § "Post-execution hooks") is a planned platform primitive — companion to the existing pre-check. Spec check I-005 ("post-check exists when output contracts are declared") front-runs it; it reads as aspirational today only because the hook isn't built yet, not because the check is wrong.
• GuardAgent (Security & Trust §) will enforce output-schema validation for agents that declare an output contract in template.yaml. The compatibility validator is the advisory layer that checks the contract is declared (I-001/I-003/I-004); GuardAgent is the runtime layer that enforces it. Same contract, two enforcement points.
• Forward-looking checks to consider (not yet in spec): as the pull-coordination migration lands, the validator is the natural place to advise on (a) replica-safety declaration in template.yaml (Replica groups: horizontal scaling for single agents #927 — agents that mutate ~/.trinity/ mid-turn or hold in-memory state can't safely set replica_count > 1), and (b) an agent's side-effect profile (feat: effect-scoped idempotency keys for outbound side effects #1084 — read/analysis-only agents migrate to pull mode first; channel-/payment-bound agents last). Both are template-level facts a static check can surface. Track as spec follow-ups once those workstreams firm up.

[vybe]

/claim

[github-actions]

Claimed by @vybe. Label status-in-progress added.

Next steps:

1. Create branch: feature/668-<short-slug>
2. Implement the feature
3. Open a PR with Fixes #668

[vybe]

Cross-link: the new GitHub-repo import wizard (private abilityai/trinity-enterprise#15, under epic #13 there) is designed to consume this compatibility validation inline during import (fork/copy/clone). It explicitly reuses this service rather than reimplementing — flagging so the two stay aligned.