Skip to content

docs(contract-interface): bound the delta to an up-to-date peer - #90

Merged
sanity merged 1 commit into
mainfrom
docs/empty-delta-requirement
Jul 30, 2026
Merged

docs(contract-interface): bound the delta to an up-to-date peer#90
sanity merged 1 commit into
mainfrom
docs/empty-delta-requirement

Conversation

@sanity

@sanity sanity commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Problem

ContractInterface's rustdoc says nothing about what get_state_delta owes a peer that is already up to date, and there is a real defect in the wild that it should rule out: one live contract ignores the summary argument entirely and returns its whole state as the delta, 25,403 bytes against a 24,832-byte state. Every reconciliation re-ships data the peer already holds, it never converges with anyone, and it is currently 55.6% of all broadcast sends on the network (freenet/freenet-core#5056).

Core decides whether a neighbour is up to date by running get_state_delta against that neighbour's summary and reading an empty result as "converged" (broadcast_queue.rs::fanout_send_needed via peer_summary_has_pending_state), so this is load-bearing rather than a matter of efficiency.

Approach

Doc comments only, no code or behavior change.

The obvious phrasing, "the delta to an up-to-date peer must be zero bytes", is wrong as a hard rule, and an earlier revision of this PR made that mistake. A contract that hand-rolls get_state_delta around a plain (non-Option) delta struct serializes an all-empty struct and returns roughly 20 bytes. Atlas does exactly this and is behaving correctly. A flat zero-byte rule would mark it noncompliant, which is the same false-positive shape as freenet/freenet-core#4295.

So the rustdoc states it in three tiers:

  • MUST NOT return a delta containing the state, or approaching its size. This is the actual defect, and the only tier that carries the existing "may result in the contract being deprioritized or removed" consequence.
  • SHOULD return a literally empty delta, StateDelta::from(vec![]), since zero bytes is the unambiguous converged signal.
  • Acceptable: a small fixed amount of encoding framing from an all-empty delta struct. About a byte per field with bincode; tens of bytes with CBOR, because ciborium serializes structs as maps and writes the field names (ciborium-0.2.2/src/ser/mod.rs:288).

The discriminator is delta size relative to state size, not an absolute byte count: ~20 bytes against a 500 KB state versus a state-sized delta is five orders of magnitude, and no encoding choice moves a contract across that gap.

Also documented, since it is the actionable difference between two live apps: freenet-scaffold's #[composable] derive collapses an all-None delta struct to None and so gets zero bytes for free (River), while a hand-rolled get_state_delta has to add that collapse itself (Atlas).

summarize_state gains the companion rule: the summary must be much smaller than the state, and a summary that is a copy of the state is always a bug. The offending contract fails this too, with summary == state == 24 KB.

Testing

cargo fmt --check clean. cargo doc --no-deps --features contract produces no warnings from trait_def.rs and the new intra-doc links resolve; the 5 remaining warnings are pre-existing and in other files.

[AI-assisted - Claude]

@sanity

sanity commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Please do not merge as written — a live contract violates this rule

I requested this change, and I specified the rule wrongly. Flagging before it merges, because the trait-level sentence attaches "may result in the contract being deprioritized or removed" to it.

Atlas violates the rule as written, and is behaving correctly.

atlas/main/contracts/index-contract/src/lib.rs:139:

into_writer(&st.delta(&summ), &mut out)?;
Ok(StateDelta::from(out))     // unconditional — no None arm, no empty collapse

and IndexState::delta returns a plain struct, never an Option (atlas/main/common/src/state.rs:193):

IndexDelta { key_auth, records }

Against its own summary, key_auth is None and records filters to empty, so it returns IndexDelta { key_auth: None, records: [] } — CBOR-encoding to roughly 3-4 bytes, not zero. Atlas is one of the two main live applications on the network. It correctly computed "nothing to send"; it just says so in a few bytes rather than none.

River passes only because freenet-scaffold's #[composable] macro collapses an all-None composite to None (freenet-scaffold-macro-0.2.2/src/lib.rs:187-197), which the room contract maps to StateDelta::from(vec![]). Atlas hand-rolls get_state_delta and has no such collapse. That difference is the genuinely useful thing for a contract author, and it is what the docs should lead with.

The corrected rule, in three tiers

  • MUST NOT return a delta containing state, or one approaching the state's size. That is the actual defect: the contract motivating this returns 25,403 bytes against a 24,832-byte state, larger than the state itself.
  • SHOULD return literally zero bytes. Unambiguous, free via #[composable], one match arm otherwise.
  • Acceptable: a few bytes of encoding framing from a serialized all-empty struct. Not ideal, not a bug, not something the network will penalise.

The discriminator has to be size-relative: Atlas is ~4 bytes against a 500 KB state; the offender is state-sized. Five orders of magnitude apart, and no encoding choice moves a contract across that gap.

What needs changing

  1. Reframe the get_state_delta section as SHOULD-zero-bytes plus MUST-NOT-state-sized, and say plainly that a few bytes of framing is fine.
  2. Scope the "deprioritized or removed" consequence to the MUST NOT tier only. As written it threatens Atlas.
  3. Keep the all-None trap explanation — it is good guidance. Just present it as "here is why you may see ~12 bytes and why it is tolerable but worth avoiding" rather than as a defect.

The summarize_state size rule is unaffected and correct as written.

Apologies for the churn — the error is mine, not the author's. They implemented the brief I gave them, and the brief predated my checking Atlas.

[AI-assisted - Claude]

Document what `get_state_delta` owes a peer that is already up to date, in three
tiers, because a flat "must be zero bytes" rule flags contracts that are behaving
correctly:

- MUST NOT return a delta containing the state, or approaching its size. This is
  the actual defect and the only tier that carries the deprioritized-or-removed
  consequence.
- SHOULD return a literally empty delta. Zero bytes is the unambiguous
  "converged" signal that lets peers skip the broadcast.
- Acceptable: a small fixed amount of encoding framing from serializing an
  all-empty delta struct, about a byte per field with bincode and tens of bytes
  with CBOR since ciborium writes field names.

The discriminator is delta size relative to state size. A contract that hand-rolls
`get_state_delta` around a plain (non-`Option`) delta struct returns ~20 bytes to
an up-to-date peer, which is fine; freenet/freenet-core#5056 returns 25,403 bytes
against a 24,832-byte state, which is not. Those are five orders of magnitude
apart and no encoding choice moves a contract across the gap.

Also notes the concrete difference this makes to an author: `freenet-scaffold`'s
`#[composable]` derive collapses an all-`None` delta to `None` and gets zero bytes
for free, while a hand-rolled implementation has to add that collapse itself.

`summarize_state` gains the companion rule: the summary must be much smaller than
the state, and a summary that is a copy of the state is always a bug.

Doc comments only, no code or behavior change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeAw3G4bFhTT242xoEeMxN
@sanity
sanity force-pushed the docs/empty-delta-requirement branch from 0929d98 to bc7eb26 Compare July 30, 2026 19:31
@sanity sanity changed the title docs(contract-interface): require an empty delta for an up-to-date peer docs(contract-interface): bound the delta to an up-to-date peer Jul 30, 2026
@sanity

sanity commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Force-pushed a correction (0929d98 -> bc7eb26). The first revision of this PR stated the rule as a flat "MUST return zero bytes" and attached the existing "may result in the contract being deprioritized or removed" consequence to it. That was wrong, and it would have flagged a correct app.

Atlas's index contract hand-rolls get_state_delta around a plain, non-Option delta struct:

// atlas/contracts/index-contract/src/lib.rs:139
into_writer(&st.delta(&summ), &mut out)?;
Ok(StateDelta::from(out))     // unconditional, no empty collapse

Against its own summary that produces IndexDelta { key_auth: None, records: [] }, which is about 20 bytes of CBOR rather than zero (ciborium serializes structs as maps and writes the field names). Atlas is behaving correctly and does not use freenet-scaffold, so it has no all-None collapse; River passes the zero-byte bar only because the #[composable] macro gives it one.

The rule is now stated in three tiers, and only the MUST NOT tier (a delta containing the state, or approaching its size) carries the deprioritize/remove consequence. The discriminator is delta size relative to state size: ~20 bytes against a 500 KB state versus the 25,403-byte delta against a 24,832-byte state in freenet/freenet-core#5056.

[AI-assisted - Claude]

sanity added a commit to freenet/freenet-agent-skills that referenced this pull request Jul 30, 2026
1.10.1 stated it as a flat "get_state_delta MUST return zero bytes to an
up-to-date peer". That is wrong as a hard rule and marks a correct live app
noncompliant.

Atlas's index contract hand-rolls get_state_delta around a plain, non-`Option`
delta struct and serializes it unconditionally, so against its own summary it
returns `IndexDelta { key_auth: None, records: [] }`, about 20 bytes of CBOR
rather than zero, since ciborium serializes structs as maps and writes the field
names. Atlas is behaving correctly; it has no all-`None` collapse because it does
not use freenet-scaffold, and River clears the zero-byte bar only because the
`#[composable]` macro gives it one.

The rule is now three tiers: MUST NOT return a delta containing the state or
approaching its size (the actual defect, and the only tier with a consequence);
SHOULD return a literally empty StateDelta, the only result that passes core's
converged check; acceptable to return a few tens of bytes of encoding framing
from an all-empty struct. The discriminator is delta size relative to state size.
Roughly 20 bytes against a 500 KB state versus freenet/freenet-core#5056's 25,403
bytes against a 24,832-byte state is five orders of magnitude, and no encoding
choice moves a contract across that gap.

- `references/contract-patterns.md`: retitled and restated in tiers. The
  all-`None` block is now "why you may see a few tens of bytes and why that is
  tolerable" rather than a defect, with the byte figure corrected to 28 measured
  for the three-field CBOR example. Adds the actionable difference for an author,
  `#[composable]` gives zero bytes for free while a hand-rolled implementation
  must add the collapse. Describes core's converged test as it actually works,
  byte-identical summaries first and the get_state_delta probe only when summary
  bytes differ. The test asserts the size bound rather than `== 0`.
- `SKILL.md`: same reframing in the sync section and Phase 1 step 4.

The companion stdlib change carries the same correction, scoping the
deprioritize-or-remove consequence to the MUST NOT tier (freenet/freenet-stdlib#90).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SeAw3G4bFhTT242xoEeMxN
@sanity

sanity commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

All three items are applied in bc7eb26 (the force-push above crossed with this comment). Mapping them to the current text:

  1. Reframed as SHOULD-zero / MUST-NOT-state-sized. get_state_delta now carries a "The delta to an up-to-date peer must be negligible" section with the three tiers, including "Acceptable: a small fixed amount of encoding framing ... not a defect and carries no penalty", and the size-relative discriminator stated as "Twenty bytes against a 500 KB state is fine. A state-sized delta is not."
  2. The consequence is scoped to MUST NOT only. The trait-level sentence now reads "failing to obey the commutativity rule or returning a state-sized delta to a peer that is already up to date", and the MUST NOT bullet is the only tier that mentions deprioritization.
  3. The all-None explanation is kept, as the reason you may see framing bytes rather than as a defect, and the #[composable]-vs-hand-rolled difference is stated explicitly as the last paragraph of the section.

Two corrections to the numbers in your comment, neither of which changes the argument:

  • The Atlas delta is ~20 bytes, not 3-4. ciborium serializes structs as CBOR maps and writes the field names (ciborium-0.2.2/src/ser/mod.rs:288), so {key_auth: null, records: []} encodes to 20 bytes. My original "10-15 bytes" was too low for the same reason. The docs now say "about a byte per field with bincode, and tens of bytes with CBOR".
  • Core's converged test is strictly byte-emptiness (let has_change = !d.as_ref().is_empty();, crates/core/src/ring/interest.rs:1944), so framing bytes do fail that check. They rarely cost anything, because plan_fanout_send skips byte-identical summaries before the probe runs, so a converged pair skips on the summary comparison first. That is why the framing tier is tolerable, and why zero is still worth having.

[AI-assisted - Claude]

@sanity

sanity commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Block resolved — I re-read the diff rather than taking the summary on trust, and all three items are addressed in the text.

  • The deprioritize/remove consequence now sits on the MUST NOT tier only ("or returning a state-sized delta to a peer that is already up to date").
  • The Acceptable tier says outright that framing bytes are "not a defect and carries no penalty", which is what keeps live Atlas out of scope.
  • The size-relative framing is explicit: "Twenty bytes against a 500 KB state is fine. A state-sized delta is not." That is the discriminator that matters, and the earlier flat zero-bytes rule lacked it.

Since I filed the block, two more data points landed that support this wording. Delta's site-contract returns 39 bytes for the same reason Atlas returns 20 — ciborium writing field names for an all-empty struct — and in both cases the contract had already computed the correct "nothing changed" answer and discarded it at serialization. Two independent teams hit the identical shape. A flat rule would have flagged both as violations when neither is doing anything wrong.

The #[composable]-vs-hand-rolled closing line is the actionable part: it names why one group of contracts gets this free and the other has to add the check.

Merging.

[AI-assisted - Claude]

@sanity
sanity merged commit 99ee584 into main Jul 30, 2026
8 checks passed
@sanity
sanity deleted the docs/empty-delta-requirement branch July 30, 2026 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant