bug(audit): enabling the hash chain does not survive a backend restart — the integrity control silently turns itself off

[dolho]

Summary

Enabling the audit-log hash chain is in-memory only. POST /api/audit-log/hash-chain/enable sets an instance attribute and writes nothing; nothing restores it at boot. So the integrity control silently switches itself back off on every backend restart, and nothing tells the operator.

Found by /edge-cases analysis of #1985 (the PR that made verify_chain honest about unhashed ranges). #1985 fixed the reporting; this is the reason ranges keep going unhashed in the first place.

Mechanism

src/backend/services/platform_audit_service.py:

def __init__(self) -> None:
    self._last_hash: Optional[str] = None
    self._hash_chain_enabled = False          # Phase 4 toggle

def enable_hash_chain(self, enabled: bool = True) -> None:
    """Toggle hash chain computation for new entries (Phase 4)."""
    self._hash_chain_enabled = enabled        # …and that is the whole write path

routers/audit_log.py:125-131 hands the flag straight to that setter — no system_settings row, no env var. grep -rn "hash_chain" src/backend/main.py → nothing, so there is no boot restore either.

Verified: getattr(platform_audit_service, "_hash_chain_enabled", False) in audit_retention_service.py:92 does read the real singleton (it imports the instance, not the module), so that consumer is fine — the gap is purely persistence.

Why it matters

• Restarts are routine and expected — CLAUDE.md documents "Re-login after restart: when the backend restarts, users need to re-login". Every deploy, every config change, every container recreate silently disables audit hashing.
• The operator gets no signal. The next verification over a range spanning the restart returns status: "verified_partial" with valid: true — a range that is mostly unverifiable reads as verified to any truthiness check. That is a weaker form of exactly the failure bug(security): audit-log verify returns valid:true with checked:0 — an unhashed chain reports as intact #1984 set out to eliminate, moved one step along.
• A range consisting only of post-restart rows returns unverifiable, which is correct — but by then the entries are already unhashed and unhashable after the fact.

Note _last_hash is instance state for the same reason: if the backend is ever run with more than one uvicorn worker (several subsystems already guard for that — the #1464 monitoring leader lock, #1632 operator-queue leader, the #506 clamp-on-use rationale), each worker keeps its own chain head and writes rows whose previous_hash points into a different worker's sequence. verify_chain's link check then reports valid: false — "tampered" on an untampered log. The shipped CMD has no --workers, so that half is conditional; the restart half is not.

Reproduction

tests/unit/test_audit_chain_edges.py::TestHashChainLifecycle::test_enabling_the_hash_chain_survives_a_restart (currently xfail(strict=True)) models a restart as the next process constructing the service:

svc_a = mod.PlatformAuditService()
svc_a.enable_hash_chain(True)
svc_b = mod.PlatformAuditService()      # the process after a restart
assert svc_b._hash_chain_enabled is True   # fails

test_the_enable_route_persists_nothing pins the mechanism so the finding survives a refactor of the service.

Suggested fix

Persist the flag in system_settings (the pattern monitoring_config already uses for exactly this problem — #1121 made the monitoring loop lifespan-resumed from a persisted flag for the same reason) and resolve it at startup. Whatever the mechanism, the requirement is that a control an operator switched on is still on after a restart, or that they are told it isn't.

Also worth deciding: whether an install that has hashing enabled should surface "hashing was off for rows N..M" anywhere, since that gap is permanent once written.

Acceptance criteria

• Hash-chain enablement survives a backend restart
• The state is readable (so verify_chain's hash_chain_enabled reflects the install, not the process)
• _last_hash is derived from the DB rather than per-process instance state, so a multi-worker deployment cannot produce a false tampered verdict
• Regression test: enable → new process → still enabled (the xfail above flips to passing)

Related

• bug(security): audit-log verify returns valid:true with checked:0 — an unhashed chain reports as intact #1984 / fix(security): an unhashed audit chain no longer reports as verified (#1984) #1985 — the tri-state verdict this sits under
• bug(monitoring): fleet health loop not wired to app lifespan — dies on every backend restart, never resumes #1121 — the persisted-flag + lifespan-resume pattern for monitoring_config

[dolho]

/claim

[github-actions]

Claimed by @dolho. Label status-in-progress added.

Next steps:

1. Create branch: feature/2015-<short-slug>
2. Implement the feature
3. Open a PR with Fixes #2015