Enforce recorded ref and runner on skill verification - #6315
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6315 +/- ##
==========================================
+ Coverage 72.84% 72.86% +0.02%
==========================================
Files 742 742
Lines 77807 77876 +69
==========================================
+ Hits 56681 56747 +66
- Misses 17155 17158 +3
Partials 3971 3971 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
JAORMX
left a comment
There was a problem hiding this comment.
Panel review found three ref-enforcement bypasses and one public error-contract mismatch. Please address the inline findings before merge.
|
Thanks for the panel review — all four findings addressed in ede23e1. The three High findings were the same underlying problem from different angles: the automatic tag-to-tag ref rotation on upgrade had no mechanism tying a candidate's tag to the specific version being upgraded to, so it was bypassable three different ways (a public Given that, I removed the automatic allowance entirely rather than trying to patch it narrower: every ref or runner-class change now blocks exactly like a genuine signer-identity change, gated behind the same explicit For the Medium finding, added Full test suite, lint, and |
The certificate signing a skill artifact carries the git ref the signing workflow ran on and the runner class it executed in, but both were discarded: core's SAN policy deliberately matches any ref, so provenance recorded for "this workflow in this repository" was satisfied by the same workflow run from any branch, or on a self-hosted runner. Read both Fulcio extensions off the verified certificate, record them in the lock entry, and enforce the recorded values after the Sigstore policy has passed — on the OCI, git, and stored-bundle paths alike. An empty recorded value stays unconstrained, so entries written before the fields existed, and certificates that carry no such extension, keep verifying. Upgrade relaxes the ref alone and re-pins the observed one: a tag-based release workflow signs every version on a new ref, and rejecting that would demand --allow-signer-change on every routine upgrade, disabling the guard entirely. Install and sync keep the exact match, and a runner class change is blocked at every operation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every upgrade cleared the pinned ref unconditionally before re-verifying, so a candidate signed by the same identity, issuer, and runner from a different branch passed and silently replaced the locked ref — the exact substitution ref pinning exists to catch. guardSignerChange now validates the transition itself: a recorded tag ref may rotate to another tag (a release workflow signs each version on its own tag), anything else must stay identical. Separately, verifyKeylessBundles could lose a pinned-field mismatch to a later bundle's plain policy failure by simple iteration order, defeating classifyVerifyFailure's preference for the more specific diagnosis. The per-bundle verification step is now a pure function so the error-selection logic is directly testable without a live Sigstore deployment. Also documents both in the trust-model writeup, which previously described identity as fully enforced inside the Sigstore policy.
An earlier fix let a recorded tag ref rotate to any other tag ref automatically during upgrade, reasoning that a release workflow signs each version on its own tag. Panel review found this let a candidate signed from an attacker's own tag on the same repository (e.g. refs/tags/attacker-release) replace a pinned tag just as easily, since nothing tied the candidate's tag to the version actually being upgraded to. Binding it correctly would need the resolved release source's own tag, which the git resolver never surfaces (only the resolved commit hash), so a fix scoped to OCI would leave git-sourced skills with the identical hole. Every ref or runner-class change now blocks exactly like a genuine signer-identity change and needs the same explicit --allow-signer-change override — no more relaxation mechanism, no public InstallOptions field that could bypass it from another call site, and no window between a vetted planning probe and a differently verified install. Also distinguishes a pinned ref/runner mismatch from a genuine signer identity mismatch in error classification: pinnedFieldMismatch now wraps both ErrSignerMismatch (existing callers keep working) and a new ErrProvenanceFieldMismatch, so sync/upgrade/install no longer report "signer identity mismatch" for what is actually just a ref or runner change.
ede23e1 to
9fcfbac
Compare
JAORMX
left a comment
There was a problem hiding this comment.
Follow-up panel verified all four prior findings are resolved: the partial ref-relaxation path is gone, every recorded ref transition requires the explicit signer-change override, apply enforces the same full expectation, and provenance-field mismatches now have a distinct typed classification. No new findings.
Summary
.1.11), but ToolHive discarded both. Core's SAN policy deliberately matches(@.*)?$— any ref — so a lock entry pinning "this workflow in this repository" was equally satisfied by the same workflow run from an attacker's branch, or moved to a self-hosted runner. Record certificate ref and runner in lock provenance #6312 added the lock file fields; this PR populates and enforces them.Result.RepositoryRef/RunnerEnvironment→lockfile.Provenance), on the OCI, git, and adopted-bundle paths alike.checkPinnedCertificateFields. A mismatch disqualifies the bundle exactly like a policy failure and surfaces asErrSignerMismatch, naming which field differed and its observed value.RepositoryURIalready set.refs/tags/v0.1.0→refs/tags/v0.2.0); rejecting that would demand--allow-signer-changeon every routine upgrade, training users to bypass the guard entirely. Install and sync keep the exact match, and a runner-class change is blocked at every operation, including upgrade.Part of #6309, stacked on the merged #6312. This enforcement path works today for any keyless-signed skill — GitHub Actions certificates always carry these extensions, e.g. from the existing
skills-build-and-publishworkflow. #6307 (keyless push signing, separate stack) is what will make certificates carry them in bulk.Type of change
Test plan
task test)task lint-fix)New coverage:
pkg/skills/verifier: table-drivenTestCheckPinnedCertificateFields(match, ref mismatch, runner mismatch, unconstrained empty expected, certificate that stopped carrying the pinned ref);TestGitCertificateRefAndRunnersigns a commit with a synthetic Fulcio certificate carrying the DER-encoded extensions and asserts they reach the recorded provenance and are enforced, plus a non-CI certificate that carries neither;TestClassifyVerifyFailureKeepsPinnedFieldDiagnosis;ToLockProvenance/expectedIdentitycarry-through.pkg/skills/skillsvc:TestUpgrade_RepinsRepositoryRef(a release on a new ref upgrades without--allow-signer-change, the relaxed expectation still enforces the runner class and signer identity, and the lock entry re-records the new ref — verified non-vacuous by temporarily disabling the flag);TestInstallVerification_EnforcesPinnedRef(a reinstall on a different ref is rejected 403 and the entry is not re-pinned);TestRefRelaxedExpectation(install/sync enforce, upgrade relaxes, caller's entry not mutated);TestRunnerEnvironmentChanged.One pre-existing failure in
pkg/transport/proxy/streamableis unrelated: that test binds hardcoded port 8096, which a localthvprocess was holding.API Compatibility
v1beta1API, OR theapi-break-allowedlabel is applied and the migration guidance is described above.Does this introduce a user-facing change?
Yes. Skills installed from a keyless-signed artifact now record the signing workflow's git ref and runner class in
toolhive-skills.lock, and later installs and syncs reject an artifact whose certificate no longer matches them. Existing lock entries record neither and are unaffected until reinstalled. Upgrades re-pin the ref, which shows up in the lock diff next to the digest change.Special notes for reviewers
pkg/skills/verifier, comparing againstcertificate.Summarydirectly rather than extendingcoreverifier.Identity(which has only three fields) or binding the ref into the Sigstore policy — that would gate this on a toolhive-core release.refRelaxedExpectationclears onlyRepositoryRef, and only whenInstallOptions.AllowRefRepinis set, whichapplyUpgradealone sets. Sync also setsLockResolvedReference, so keying off the existing internal options would have silently relaxed sync too.classifyVerifyFailurenow returns a pinned-field error verbatim. Without that, the re-verify-without-constraint path would report the expected signer identity back as the observed one — printing "locked to X, but verifies as X" for what is really a ref change.guardSignerChangegained a runner-class comparison so a runner change is reported assigner-change-blockedat plan time (with preview parity) instead of failing mid-apply. The ref is deliberately not compared there.Generated with Claude Code