Skip to content

Harden daemon wire, durability, and fragment-ingest boundaries - #7

Merged
taylor009 merged 1 commit into
mainfrom
harden-daemon-boundaries
Jul 15, 2026
Merged

Harden daemon wire, durability, and fragment-ingest boundaries#7
taylor009 merged 1 commit into
mainfrom
harden-daemon-boundaries

Conversation

@taylor009

Copy link
Copy Markdown
Owner

Why

Four verified latent defects sit on the daemon's externally reachable trust boundary and durability path, plus the host-contract gap they expose. None break the happy path (64/64 tests were green before this change) — each is a real bug under adversarial or failure conditions:

Defect Location Failure
Unvalidated wire length daemon_server.cpp read_frame A hostile/corrupt 4-byte header (0xFFFFFFFF) forces a ~4 GB allocation before any body is read
No socket timeouts both serve loops A client stalling mid-frame wedges the single serve thread: queries, watcher poll, persistence, and idle shutdown all freeze
status() data race daemon_ops.cpp status op Enrichment counters and the unextracted map were read while written under a different lock — races constantly because the enrichment drainer is status-gated and polls
Destructive persist fallback daemon_lifecycle.cpp persist_graph_snapshot On rename failure it deleted the last-known-good graph.json and retried; double failure = no graph at all
Shape-only fragment validation semantic_ingest.cpp A schema-valid fragment with dangling edge endpoints merged silently, breaking the host contract's 'malformed -> rejected, graph unchanged'

What changed

  • Shared kMaxFrameBodyBytes cap (64 MiB) in protocol.hpp, enforced on both send (encode_frame) and receive (daemon read_frame) — oversized declared lengths rejected before allocation.
  • SO_RCVTIMEO/SO_SNDTIMEO (5 s) on every accepted connection in both serve loops; stalled peers are dropped, the loop continues.
  • Dedicated enrichment_mutex in DaemonState taken by every writer (enrichment refresh, drop ingest, rescan/incremental update, EnrichmentRunningScope) and by the status reader, which snapshots under the lock.
  • persist_graph_snapshot is atomic-rename-only: failure leaves the prior graph.json untouched, cleans up the temp, surfaces the error.
  • Semantic ingest rejects a fragment atomically when any edge endpoint exists in neither the fragment (keyed exactly as merge keys nodes, via the now-exported graph_builder::node_key — linkage change only) nor the live graph. merge_fragment/merge_fragments semantics unchanged (Graphify parity is a hard contract).

Verification

  • 64/64 tests passing (ctest --preset default, macOS).
  • End-to-end raw-socket coverage in daemon_server_test against a live daemon: a 0xFFFFFFFF length header is rejected without allocation and the daemon still answers status; a stalled half-frame reader is timed out and the serve loop recovers.
  • Persist durability regression test: a failed rename leaves the prior destination intact and the temp cleaned up.
  • Referential integrity: dangling-endpoint fragment rejected with graph byte-identical; control fragment with an edge into an existing graph node still merges.

OpenSpec change: openspec/changes/harden-daemon-boundaries/ (spec deltas for graph-daemon-client and semantic-fragment-ingest).

🤖 Generated with Claude Code

Four verified latent defects on the daemon's trust and durability
boundaries, plus the host-contract gap they expose:

- Wire DoS: read_frame allocated a buffer straight from the uint32
  length read off the socket (a 0xFFFFFFFF header forced a ~4 GB
  allocation). A shared kMaxFrameBodyBytes cap (64 MiB) in protocol.hpp
  is now enforced by both encode_frame and the daemon receive path;
  oversized frames are rejected before allocation.
- Hang: no socket timeouts existed, so a client stalling mid-frame froze
  the single-threaded serve loop (queries, watcher, persistence, idle
  shutdown). SO_RCVTIMEO/SO_SNDTIMEO (5 s) now apply to every accepted
  connection; stalled peers are dropped and the loop continues.
- status() data race: the Status op read enrichment counters and
  iterated the unextracted map while enrichment refresh, drop ingest,
  and rescans wrote them under a different lock. A dedicated
  enrichment_mutex in DaemonState is now taken by every reader and
  writer; status snapshots under the lock.
- Destructive persist fallback: on rename failure persist_graph_snapshot
  deleted the last-known-good graph.json and retried, so a double
  failure destroyed the graph. Now atomic-rename-only: failure leaves
  the prior file untouched, removes the orphan temp, surfaces the error.
- Fragment referential integrity: semantic ingest now rejects a fragment
  atomically when an edge endpoint exists in neither the fragment (keyed
  via the now-exported graph_builder node_key) nor the live graph,
  keeping the host contract's 'malformed -> rejected, graph unchanged'
  true for referential breakage. merge_fragment semantics unchanged
  (Graphify parity).

Tests: 64/64 passing, including raw-socket hostile-frame coverage
(oversized header, stalled half-frame) against a live daemon.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@taylor009
taylor009 merged commit 2cf4718 into main Jul 15, 2026
11 checks passed
taylor009 pushed a commit that referenced this pull request Jul 15, 2026
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants