Skip to content

Enforce recorded ref and runner on skill verification - #6315

Merged
samuv merged 3 commits into
mainfrom
skills-refpin/02-extract-and-enforce
Aug 14, 2026
Merged

Enforce recorded ref and runner on skill verification#6315
samuv merged 3 commits into
mainfrom
skills-refpin/02-extract-and-enforce

Conversation

@samuv

@samuv samuv commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • A skill's signing certificate carries the git ref the signing workflow ran on (Fulcio OID 1.3.6.1.4.1.57264.1.14) and the runner class it executed in (.1.11), but ToolHive discarded both. Core's SAN policy deliberately matches (@.*)?$ — any ref — so a lock entry pinning "this workflow in this repository" was equally satisfied by the same workflow run from an attacker's branch, or moved to a self-hosted runner. Record certificate ref and runner in lock provenance #6312 added the lock file fields; this PR populates and enforces them.
  • Read both extensions off the certificate a successful verification produced and record them in the lock entry (Result.RepositoryRef / RunnerEnvironmentlockfile.Provenance), on the OCI, git, and adopted-bundle paths alike.
  • Enforce the recorded values post-hoc, after core's Sigstore policy has already accepted the certificate, via checkPinnedCertificateFields. A mismatch disqualifies the bundle exactly like a policy failure and surfaces as ErrSignerMismatch, naming which field differed and its observed value.
  • An empty recorded value is unconstrained, not a wildcard mismatch: every lock entry written before the fields existed has them empty, as do certificates from signers outside GitHub Actions (gitsign from a personal OIDC identity carries neither). Both keep verifying — the same precedent RepositoryURI already set.
  • Upgrade relaxes the ref alone and re-pins the observed one. A tag-based release workflow signs each version on a new ref (refs/tags/v0.1.0refs/tags/v0.2.0); rejecting that would demand --allow-signer-change on every routine upgrade, training users to bypass the guard entirely. Install and sync keep the exact match, and a runner-class change is blocked at every operation, including upgrade.

Part of #6309, stacked on the merged #6312. This enforcement path works today for any keyless-signed skill — GitHub Actions certificates always carry these extensions, e.g. from the existing skills-build-and-publish workflow. #6307 (keyless push signing, separate stack) is what will make certificates carry them in bulk.

Type of change

  • New feature

Test plan

  • Unit tests (task test)
  • Linting (task lint-fix)

New coverage:

  • pkg/skills/verifier: table-driven TestCheckPinnedCertificateFields (match, ref mismatch, runner mismatch, unconstrained empty expected, certificate that stopped carrying the pinned ref); TestGitCertificateRefAndRunner signs a commit with a synthetic Fulcio certificate carrying the DER-encoded extensions and asserts they reach the recorded provenance and are enforced, plus a non-CI certificate that carries neither; TestClassifyVerifyFailureKeepsPinnedFieldDiagnosis; ToLockProvenance / expectedIdentity carry-through.
  • pkg/skills/skillsvc: TestUpgrade_RepinsRepositoryRef (a release on a new ref upgrades without --allow-signer-change, the relaxed expectation still enforces the runner class and signer identity, and the lock entry re-records the new ref — verified non-vacuous by temporarily disabling the flag); TestInstallVerification_EnforcesPinnedRef (a reinstall on a different ref is rejected 403 and the entry is not re-pinned); TestRefRelaxedExpectation (install/sync enforce, upgrade relaxes, caller's entry not mutated); TestRunnerEnvironmentChanged.

One pre-existing failure in pkg/transport/proxy/streamable is unrelated: that test binds hardcoded port 8096, which a local thv process was holding.

API Compatibility

  • This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

Does this introduce a user-facing change?

Yes. Skills installed from a keyless-signed artifact now record the signing workflow's git ref and runner class in toolhive-skills.lock, and later installs and syncs reject an artifact whose certificate no longer matches them. Existing lock entries record neither and are unaffected until reinstalled. Upgrades re-pin the ref, which shows up in the lock diff next to the digest change.

Special notes for reviewers

  • No core dependency. Enforcement lives entirely in pkg/skills/verifier, comparing against certificate.Summary directly rather than extending coreverifier.Identity (which has only three fields) or binding the ref into the Sigstore policy — that would gate this on a toolhive-core release.
  • Why upgrade relaxes via a copied expectation rather than retrying on error: refRelaxedExpectation clears only RepositoryRef, and only when InstallOptions.AllowRefRepin is set, which applyUpgrade alone sets. Sync also sets LockResolvedReference, so keying off the existing internal options would have silently relaxed sync too.
  • classifyVerifyFailure now returns a pinned-field error verbatim. Without that, the re-verify-without-constraint path would report the expected signer identity back as the observed one — printing "locked to X, but verifies as X" for what is really a ref change.
  • guardSignerChange gained a runner-class comparison so a runner change is reported as signer-change-blocked at plan time (with preview parity) instead of failing mid-apply. The ref is deliberately not compared there.

Generated with Claude Code

@samuv
samuv requested a review from JAORMX as a code owner August 13, 2026 16:31
@github-actions github-actions Bot added the size/L Large PR: 600-999 lines changed label Aug 13, 2026
@samuv samuv self-assigned this Aug 13, 2026
@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 74.31193% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.86%. Comparing base (d057929) to head (9fcfbac).

Files with missing lines Patch % Lines
pkg/skills/verifier/offline.go 9.09% 10 Missing ⚠️
pkg/skills/verifier/git.go 18.18% 9 Missing ⚠️
pkg/skills/verifier/oci.go 72.00% 7 Missing ⚠️
pkg/skills/verifier/types.go 95.12% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6315      +/-   ##
==========================================
+ Coverage   72.84%   72.86%   +0.02%     
==========================================
  Files         742      742              
  Lines       77807    77876      +69     
==========================================
+ Hits        56681    56747      +66     
- Misses      17155    17158       +3     
  Partials     3971     3971              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Aug 13, 2026

@JAORMX JAORMX left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Panel review found three ref-enforcement bypasses and one public error-contract mismatch. Please address the inline findings before merge.

Comment thread pkg/skills/options.go Outdated
Comment thread pkg/skills/skillsvc/upgrade.go Outdated
Comment thread pkg/skills/skillsvc/upgrade.go Outdated
Comment thread pkg/skills/verifier/types.go Outdated
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Aug 14, 2026
@samuv

samuv commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the panel review — all four findings addressed in ede23e1.

The three High findings were the same underlying problem from different angles: the automatic tag-to-tag ref rotation on upgrade had no mechanism tying a candidate's tag to the specific version being upgraded to, so it was bypassable three different ways (a public InstallOptions field, a plan/apply TOCTOU gap, and the "any tag" acceptance itself). I looked into the suggested fix of binding the allowed ref to the resolved release source, but it only works for OCI — git's resolver never surfaces which ref/tag produced a resolved commit, only the commit hash — so a partial fix would leave git-sourced skills with the identical hole.

Given that, I removed the automatic allowance entirely rather than trying to patch it narrower: every ref or runner-class change now blocks exactly like a genuine signer-identity change, gated behind the same explicit --allow-signer-change. This is a real trade-off against the original goal of tag-based releases not needing that flag on every version bump — flagging it here rather than leaving it implicit. A future, more targeted fix would need the git resolver to surface its own resolved ref, which is a separate piece of work.

For the Medium finding, added ErrProvenanceFieldMismatch (wrapping alongside the existing ErrSignerMismatch, so nothing else breaks) and a matching FailureReasonProvenanceFieldMismatch, so a ref/runner-only mismatch is no longer reported as a signer-identity change.

Full test suite, lint, and task docs all clean; replied inline on each finding with specifics.

samuv and others added 3 commits August 14, 2026 10:59
The certificate signing a skill artifact carries the git ref the signing
workflow ran on and the runner class it executed in, but both were
discarded: core's SAN policy deliberately matches any ref, so provenance
recorded for "this workflow in this repository" was satisfied by the same
workflow run from any branch, or on a self-hosted runner.

Read both Fulcio extensions off the verified certificate, record them in
the lock entry, and enforce the recorded values after the Sigstore policy
has passed — on the OCI, git, and stored-bundle paths alike. An empty
recorded value stays unconstrained, so entries written before the fields
existed, and certificates that carry no such extension, keep verifying.

Upgrade relaxes the ref alone and re-pins the observed one: a tag-based
release workflow signs every version on a new ref, and rejecting that
would demand --allow-signer-change on every routine upgrade, disabling
the guard entirely. Install and sync keep the exact match, and a runner
class change is blocked at every operation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Every upgrade cleared the pinned ref unconditionally before
re-verifying, so a candidate signed by the same identity, issuer, and
runner from a different branch passed and silently replaced the
locked ref — the exact substitution ref pinning exists to catch.
guardSignerChange now validates the transition itself: a recorded tag
ref may rotate to another tag (a release workflow signs each version
on its own tag), anything else must stay identical.

Separately, verifyKeylessBundles could lose a pinned-field mismatch
to a later bundle's plain policy failure by simple iteration order,
defeating classifyVerifyFailure's preference for the more specific
diagnosis. The per-bundle verification step is now a pure function so
the error-selection logic is directly testable without a live
Sigstore deployment.

Also documents both in the trust-model writeup, which previously
described identity as fully enforced inside the Sigstore policy.
An earlier fix let a recorded tag ref rotate to any other tag ref
automatically during upgrade, reasoning that a release workflow signs
each version on its own tag. Panel review found this let a candidate
signed from an attacker's own tag on the same repository (e.g.
refs/tags/attacker-release) replace a pinned tag just as easily,
since nothing tied the candidate's tag to the version actually being
upgraded to. Binding it correctly would need the resolved release
source's own tag, which the git resolver never surfaces (only the
resolved commit hash), so a fix scoped to OCI would leave git-sourced
skills with the identical hole.

Every ref or runner-class change now blocks exactly like a genuine
signer-identity change and needs the same explicit
--allow-signer-change override — no more relaxation mechanism, no
public InstallOptions field that could bypass it from another call
site, and no window between a vetted planning probe and a differently
verified install.

Also distinguishes a pinned ref/runner mismatch from a genuine signer
identity mismatch in error classification: pinnedFieldMismatch now
wraps both ErrSignerMismatch (existing callers keep working) and a new
ErrProvenanceFieldMismatch, so sync/upgrade/install no longer report
"signer identity mismatch" for what is actually just a ref or runner
change.
@samuv
samuv force-pushed the skills-refpin/02-extract-and-enforce branch from ede23e1 to 9fcfbac Compare August 14, 2026 09:04
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Aug 14, 2026

@JAORMX JAORMX left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up panel verified all four prior findings are resolved: the partial ref-relaxation path is gone, every recorded ref transition requires the explicit signer-change override, apply enforces the same full expectation, and provenance-field mismatches now have a distinct typed classification. No new findings.

@samuv
samuv merged commit cfba580 into main Aug 14, 2026
57 of 58 checks passed
@samuv
samuv deleted the skills-refpin/02-extract-and-enforce branch August 14, 2026 12:43
@github-actions github-actions Bot mentioned this pull request Aug 14, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Large PR: 600-999 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants