Skip to content

fix(security): an unhashed audit chain no longer reports as verified (#1984) - #1985

Merged
vybe merged 2 commits into
devfrom
fix/1984-audit-verify-unverifiable
Aug 4, 2026
Merged

fix(security): an unhashed audit chain no longer reports as verified (#1984)#1985
vybe merged 2 commits into
devfrom
fix/1984-audit-verify-unverifiable

Conversation

@dolho

@dolho dolho commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Closes #1984 — a bug I found by probing the live instance for siblings of the #1966#1971 batch.

Problem, as observed on real data

$ psql -c "SELECT count(*), count(entry_hash) FROM audit_log;"
 1162 | 0

$ curl -X POST ".../api/audit-log/verify?start_id=1&end_id=1162"
{"valid":true,"checked":0,"first_invalid_id":null}

And in the UI: a green "✓ Valid · 0 entries" badge — the tick sitting directly beside the zero that contradicts it.

valid: true was returned for three materially different states:

State Old answer
Chain verified intact valid: true
Range empty valid: true
Nothing hashed at all valid: true

The third is the default for every install that never enabled hashing — so the verdict was effectively independent of the input. That is the class learnings.md already names from #1941: "when a check reports the SAME verdict for every input, that is the signal that it is broken."

Skipping an unhashed entry is correct on its own — a chain enabled midway legitimately has an unhashed prefix. The defect was the aggregate verdict when every entry was skipped.

Fix

valid becomes tri-state: True verified · False mismatch · None unverifiable.

None and not False, deliberately. False claims tampering — an equally wrong and considerably louder lie that would page someone about a breach that did not happen. And a caller doing a plain truthiness test degrades to "not verified", which is the safe direction.

status carries the precise verdict (verified / verified_partial / tampered / unverifiable / empty_range), and skipped_unhashed makes a late-enabled chain's permanent unhashed prefix visible rather than silently averaged over — a partially-hashed range can no longer pass as fully verified.

The response now also reports hash_chain_enabled, which is precisely what made checked: 0 uninterpretable.

Trinity already solves this correctly one endpoint over. GET /api/monitoring/status faces the identical situation — a summary produced by a loop that is default-OFF (#1121) — and returns enabled: false beside a stale last_check_at so the reader can interpret the payload. This is that pattern applied where it was missing.

The frontend had to change too

This is the part a backend-only fix would have got wrong. The store did:

this.verifyState = data.valid ? 'valid' : 'invalid'

So valid: null would have rendered as "✗ Tamper detected" — trading a false green for a false red, which is worse. It now branches on the tri-state and renders an amber "Unverifiable · no hashes on N entries" badge.

The store also carried its own copy of the bug: an empty entry list set verifyState='valid', checked:0 without calling the API at all. Also gone.

Verification

tests/unit/test_1984_audit_verify_unverifiable.py16 checks, 15 of which fail against the pre-fix tree. 74 green across all audit-related suites.

Scope of what I actually verified, stated precisely:

  • Live pre-fix behaviour: confirmed over HTTP against the running instance (above).
  • Post-fix behaviour: unit tests, using the live shape (rows present, zero hashed) as the fixture.
  • I did not redeploy your instance to test the fixed code in place.

Two of my own mistakes are recorded in the test file rather than quietly fixed:

Acceptance criteria

  • A range containing rows but no hashes does NOT return valid: true
  • The response distinguishes "verified intact", "nothing hashed", and "empty range"
  • A genuinely intact hashed chain still returns valid: true
  • A test asserts the unhashed case — the current default state of every install that never enabled the chain

🤖 Generated with Claude Code

dolho and others added 2 commits August 4, 2026 14:46
…1984)

`POST /api/audit-log/verify` answered `valid: true, checked: 0` for an audit
log in which no entry carried a hash. Found by probing a live instance:
1,162 real audit entries, zero hashes, and a green "✓ Valid · 0 entries"
badge in the UI — the tick sitting directly beside the zero that contradicts
it.

`valid: true` was returned for three materially different states — chain
verified, range empty, and *no integrity data exists at all*. The last is
the DEFAULT for every install that never enabled hashing, so the endpoint's
verdict was effectively independent of its input: the class
`docs/memory/learnings.md` already names from #1941.

Skipping an unhashed entry is correct on its own — a chain enabled midway
legitimately has an unhashed prefix. The defect was the aggregate verdict
when EVERY entry was skipped.

`valid` is now tri-state: True verified, False mismatch, **None
unverifiable**. `None` rather than `False` deliberately — `False` claims
tampering, which is an equally wrong and considerably louder lie, and a
caller doing a plain truthiness test degrades to "not verified", the safe
direction. `status` carries the precise verdict (`verified`,
`verified_partial`, `tampered`, `unverifiable`, `empty_range`) and
`skipped_unhashed` makes a late-enabled chain's permanent unhashed prefix
visible instead of silently averaged over.

The response also reports `hash_chain_enabled`, which is what made
`checked: 0` uninterpretable. Trinity already does this correctly one
endpoint over: `/api/monitoring/status` returns `enabled: false` beside a
stale summary so the reader can interpret it.

**The frontend had to change too.** The store did
`data.valid ? 'valid' : 'invalid'`, so a backend-only fix would have turned
a false green into a false RED — a tamper alarm for a log that is merely
unhashed. It now branches on the tri-state and renders an amber
"Unverifiable" badge. The store also had its own vacuous path, asserting
`verifyState='valid'` for an empty list without calling the API at all;
that is gone.

tests/unit/test_1984_audit_verify_unverifiable.py — 16 checks, 15 of which
fail against the pre-fix tree.

Closes #1984
Resolve tests/registry.json (rebuild: dev entries + this PR's entry).
Also sync the two stale surfaces the review found:
- tests/test_audit_log_unit.py::test_verify_chain_empty_range still asserted
  the old vacuous valid=True empty-range contract this PR removes — it is not
  collected by PR CI (unit/ only) but breaks tests/run-core.sh on dev.
- docs/memory/feature-flows/audit-trail.md verify example updated to the
  tri-state response shape.

@vybe vybe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validated via /validate-pr — the fix itself is exemplary (honest tri-state semantics, 16-check named regression test across model/service/store/view; verified an all-unhashed range can never report valid:true). The one blocker from validation — tests/test_audit_log_unit.py::test_verify_chain_empty_range still asserting the old vacuous empty-range contract (invisible to PR CI which runs unit/ only, but breaks tests/run-core.sh on dev) — I fixed on the branch in the merge commit, plus the stale audit-trail.md response example. Full CI green post-rebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants