Skip to content

Commit fb52d87

Browse files
tartakovskytartakovsky
committed
Plan exact list difference unification
1 parent 599e086 commit fb52d87

2 files changed

Lines changed: 323 additions & 0 deletions

File tree

‎.plans/2026-07-15-211236-exact-list-differences.md‎

Lines changed: 298 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,298 @@
1+
# Exact List Difference Unification
2+
3+
## Status
4+
`IMPLEMENTATION-READY`
5+
6+
## Goal
7+
- Keep `ListRequirements::{contains, excludes, exact}` as the complete list requirement vocabulary.
8+
- Make exact-list findings identify missing and unexpected members with selectors.
9+
- Keep a selectorless finding only when the list has the correct multiset in the wrong order.
10+
- Reuse one core difference calculation in JSON, TOML, and YAML engines.
11+
- Let a standard waiver suppress one exact-list member without suppressing unrelated members.
12+
- Preserve exact-list merge, desired-byte, and create-missing-list behavior.
13+
14+
This is the AQC prerequisite for the Shakts CSpell plan. CSpell needs exact-empty suppression lists whose intentional exceptions remain individually waivable.
15+
16+
Excluded:
17+
- no new assertion verb or compatibility alias;
18+
- no `allowed`, `closed`, or format-specific list requirement;
19+
- no change to scalar, item, map, or forbidden-glob semantics;
20+
- no file paths, IO, policy meaning, adapter meaning, or Shackles dependency in AQC;
21+
- no publication in the implementation round unless separately requested.
22+
23+
## Evidence
24+
### Current Core
25+
`ListRequirements` already expresses:
26+
```rust
27+
pub struct ListRequirements {
28+
pub contains: BTreeMap<String, String>,
29+
pub excludes: BTreeMap<String, String>,
30+
pub exact: Option<(Vec<String>, String)>,
31+
}
32+
```
33+
34+
Core resolution already:
35+
- merges equal exact lists with provenance;
36+
- conflicts unequal exact lists;
37+
- conflicts exact lists with incompatible contains/excludes;
38+
- exposes `ResolvedExactList` with merged values and attribution.
39+
40+
The requirement model is sufficient. The defect is reconciliation reporting.
41+
42+
### Current Reconciliation Duplication
43+
- `aqc-toml-engine-core` emits one selectorless mismatch for any exact-list difference.
44+
- `aqc-json-file-engine` independently emits the same whole-list mismatch.
45+
- `aqc-pnpm-workspace-yaml-engine` independently emits one collection mismatch.
46+
- all three compute desired exact bytes separately after reporting.
47+
48+
The whole-list finding makes a waiver overbroad: waiving one intentional extra member suppresses every other difference in that list.
49+
50+
### Comparable Member Findings
51+
Contains, excludes, and forbidden-glob requirements already report one member identity at a time. Standard Shackles waivers already consume finding selectors. Exact-list differences should use the same identity boundary where membership, rather than order, is wrong.
52+
53+
## Required Behavior
54+
### D1: Difference Calculation
55+
Add one universal core result and function:
56+
```rust
57+
#[derive(Debug, Clone, PartialEq, Eq)]
58+
pub struct ExactListDifference {
59+
missing: BTreeMap<String, usize>,
60+
unexpected: BTreeMap<String, usize>,
61+
order_mismatch: bool,
62+
}
63+
64+
pub fn exact_list_difference(
65+
current: &[String],
66+
expected: &[String],
67+
) -> ExactListDifference;
68+
69+
impl ExactListDifference {
70+
pub const fn missing(&self) -> &BTreeMap<String, usize>;
71+
pub const fn unexpected(&self) -> &BTreeMap<String, usize>;
72+
pub const fn order_mismatch(&self) -> bool;
73+
pub fn is_empty(&self) -> bool;
74+
}
75+
```
76+
77+
Rules:
78+
- counts are multiset differences, not set differences;
79+
- `missing[value]` is expected count minus current count when positive;
80+
- `unexpected[value]` is current count minus expected count when positive;
81+
- `order_mismatch` is true only when both count maps are empty and sequences differ;
82+
- equal lists return empty maps and `false`;
83+
- result ordering is lexical through `BTreeMap`;
84+
- every stored count is positive;
85+
- `order_mismatch` cannot coexist with missing or unexpected members;
86+
- empty strings and duplicate values are valid identities and retain correct counts;
87+
- the function performs no rendering and creates no findings.
88+
89+
Examples:
90+
```text
91+
current [b], expected [a]
92+
missing {a:1}, unexpected {b:1}, order false
93+
94+
current [a,a], expected [a]
95+
missing {}, unexpected {a:1}, order false
96+
97+
current [b,a], expected [a,b]
98+
missing {}, unexpected {}, order true
99+
100+
current [], expected []
101+
missing {}, unexpected {}, order false
102+
```
103+
104+
### D2: Finding Contract
105+
Every list reconciler uses the shared difference result.
106+
107+
When the list field itself is absent:
108+
- emit the existing selectorless missing/exact-list finding;
109+
- do not also emit member differences;
110+
- initialize the complete expected list, including exact empty;
111+
- a field-level waiver remains distinct from a member waiver.
112+
113+
For each distinct missing value:
114+
- finding selector is that value;
115+
- current says absent or includes the lower count;
116+
- expected says present or includes the required count;
117+
- message and attribution come from the resolved exact assertion.
118+
119+
For each distinct unexpected value:
120+
- finding selector is that value;
121+
- current says present or includes the extra count;
122+
- expected says absent or includes the allowed count;
123+
- message and attribution come from the resolved exact assertion.
124+
125+
When only order differs:
126+
- emit one selectorless exact-order finding;
127+
- render complete current and expected lists;
128+
- use exact assertion message and attribution.
129+
130+
Do not emit a second selectorless exact mismatch when member findings already explain the difference.
131+
132+
Compatible exact and member assertions remain separate findings:
133+
- exact plus `contains` may emit two missing-member findings for one identity;
134+
- exact plus `excludes` may emit two unexpected-member findings for one identity;
135+
- each finding keeps its own message and attribution;
136+
- both use the same format-specific key and selector identity;
137+
- one standard key/selector waiver suppresses both findings for that identity without suppressing siblings.
138+
139+
Format engines retain ownership of:
140+
- finding key syntax;
141+
- current/expected rendering;
142+
- severity;
143+
- collection mutation;
144+
- format-specific list shape errors.
145+
146+
### D3: Waiver Consequences
147+
- A selector waiver for one unexpected member suppresses only that member finding.
148+
- A sibling unexpected or missing member remains visible.
149+
- A selector waiver does not suppress an order-only finding.
150+
- A selectorless waiver may suppress the order-only finding under existing waiver semantics.
151+
- Waivers do not change expected bytes or merge semantics.
152+
153+
### D4: Reconciliation Consequences
154+
- Desired bytes remain the resolved exact list after findings are created.
155+
- Missing exact-empty lists remain constructive and initialize as empty arrays/sequences.
156+
- Wrong list value kinds keep the existing shape finding; no member difference is emitted when members cannot be read.
157+
- Contains/excludes findings remain unchanged. Compatible overlap with exact remains separately attributed as defined in D2; contradictory combinations still fail merge.
158+
- Forbidden-glob findings remain member-specific and may coexist with compatible exact findings only where current bytes violate both requirements; each requirement retains its own attribution.
159+
160+
## Package Scope
161+
### Core Runtime/API
162+
Change `aqc-file-engine-core`:
163+
- add `ExactListDifference` beside list requirement/resolution types;
164+
- add `exact_list_difference` in list merge/support code;
165+
- re-export both from the facade;
166+
- add unit tests for equality, missing, unexpected, replacement, duplicates, empty strings, order only, and deterministic ordering.
167+
168+
Version: `0.7.2`. The helper is additive and the changed diagnostics correct overbroad exact-list findings without changing requirement resolution or expected bytes.
169+
170+
### TOML
171+
Change `aqc-toml-engine-core`:
172+
- replace local whole-list comparison reporting with the core difference result;
173+
- preserve `ListFieldKeyStyle` and TOML rendering;
174+
- add presence-aware entry points:
175+
```rust
176+
pub fn reconcile_optional_list_field(
177+
display_key: String,
178+
current: Option<Vec<String>>,
179+
requirements: &ResolvedListRequirements,
180+
key_style: ListFieldKeyStyle,
181+
findings: &mut Vec<Finding>,
182+
) -> Option<Vec<String>>;
183+
184+
pub fn reconcile_optional_table_list_field(
185+
display_key: String,
186+
current: Option<Vec<String>>,
187+
requirements: &ResolvedListRequirements,
188+
findings: &mut Vec<Finding>,
189+
) -> Option<Vec<String>>;
190+
```
191+
- retain `reconcile_list_field` and `reconcile_table_list_field` only as lower-level known-present APIs;
192+
- make optional entry points handle absence once, then delegate member/order reconciliation to the known-present primitive;
193+
- migrate every concrete caller that reads an optional TOML field without erasing `None` to an empty vector;
194+
- add contract tests for member selectors, duplicate counts, order-only mismatch, exact-empty initialization, attribution, and no duplicate whole finding.
195+
196+
Runtime callers change in Cargo TOML, deny TOML, rust-toolchain TOML, and rustfmt TOML. Clippy does not reconcile `ListRequirements` exact values and needs no runtime change. Add regressions wherever an engine transforms list findings, keys, canonical order, or missing-field writes.
197+
198+
### JSON
199+
Change `aqc-json-file-engine`:
200+
- replace `push_list_findings` exact whole-list branch with the core difference result;
201+
- keep RFC 6901 path keys and put member identity in `selector`;
202+
- add exact-empty, sibling-waiver-ready, duplicate, empty-string, and order tests.
203+
204+
Package JSON and TSConfig do not use this generic list reconciliation but move to the coherent core dependency generation.
205+
206+
### YAML
207+
Change `aqc-pnpm-workspace-yaml-engine`:
208+
- replace its exact collection mismatch with the core difference result;
209+
- keep YAML shape/render/write behavior;
210+
- add member, duplicate, order-only, attribution, and desired-output tests.
211+
212+
`aqc-yaml-engine-core` moves to the coherent core dependency generation; no YAML-core runtime behavior changes.
213+
214+
### Dependency Generation
215+
The API addition is backward-compatible and the behavior change fixes overbroad diagnostics without changing resolved requirements or expected bytes. Use one patch generation:
216+
- `aqc-file-engine-core 0.7.2`;
217+
- `aqc-json-file-engine 0.1.1`;
218+
- `aqc-toml-engine-core 0.8.1`;
219+
- `aqc-pnpm-workspace-yaml-engine 0.7.2`;
220+
- Cargo, deny, rust-toolchain, and rustfmt TOML engines at `0.7.2`.
221+
222+
Each changed crate requires `aqc-file-engine-core >=0.7.2, <0.8.0` through normal Cargo `0.7.2` dependency syntax. Existing published consumers requiring `0.7.1` accept `0.7.2`, so Cargo resolves one core generation. Existing TOML engines requiring `aqc-toml-engine-core 0.8.0` accept `0.8.1`; the existing pnpm adapter requiring YAML engine `0.7.1` accepts `0.7.2`; existing Prettier consumers requiring JSON file engine `0.1.0` accept `0.1.1`.
223+
224+
Existing adapters require the affected engine `0.7` lines and accept these patch releases, so no Shackles dependency migration is needed. Do not bump or republish dependency-only JSON, text, Clippy TOML, YAML-core, or Shackles packages. Their declared ranges already consume the fixes without duplicate core versions. Regenerate locks only in changed workspaces and verification consumers that must prove the selected generation.
225+
226+
Publication order when requested:
227+
1. `aqc-file-engine-core 0.7.2`;
228+
2. `aqc-toml-engine-core 0.8.1`;
229+
3. JSON file, pnpm YAML, Cargo TOML, deny TOML, rust-toolchain TOML, and rustfmt TOML patch releases;
230+
4. downstream Shackles adapters, policies, and CLIs.
231+
232+
Local source replacement is verification-only. Committed manifests and registry locks contain no path dependencies.
233+
234+
## Verification
235+
### Core Tests
236+
- every difference example and duplicate count;
237+
- lexical output order independent of input insertion;
238+
- no format-specific contract leaks into the API;
239+
- existing list resolution tests unchanged except imports.
240+
241+
### Engine Tests
242+
For JSON, TOML, and YAML:
243+
- one unexpected value gives one selector finding;
244+
- two unexpected values give two independently suppressible selectors;
245+
- one missing and one unexpected value both remain visible;
246+
- compatible exact-plus-contains and exact-plus-excludes retain two separately attributed findings with the same waiver identity;
247+
- duplicate count mismatch is reported once with count data;
248+
- order-only mismatch is selectorless;
249+
- exact-empty missing field initializes successfully;
250+
- a missing exact list emits one selectorless field finding and no member findings;
251+
- wrong shape emits only shape finding;
252+
- output bytes equal the exact list;
253+
- attribution includes every exact contributor in deterministic order.
254+
255+
### Fixture3
256+
Add AQC probe fixtures for serialized findings and expected bytes in all three formats. Shackles waiver behavior is proved in the downstream CSpell fixture because AQC does not own waiver application.
257+
258+
### Specular
259+
The AQC spec must prove:
260+
- exact changed-file scope;
261+
- core API definitions and exports;
262+
- all direct dependency migrations and crate versions;
263+
- no `allowed`/`closed` vocabulary or aliases;
264+
- all three reconcilers call the core helper;
265+
- every optional TOML list caller preserves absence until the presence-aware core entry point;
266+
- no remaining local exact-list multiset/order implementation;
267+
- required test and fixture families;
268+
- no Shackles dependency or product vocabulary.
269+
270+
## Decisions
271+
Accepted:
272+
- improve exact-list diagnostics rather than add an allowed-item assertion;
273+
- require exact-empty suppression lists downstream;
274+
- keep finding construction in format engines;
275+
- add one presence-aware TOML layer over the existing known-present reconciler;
276+
- preserve separately attributed compatible member assertions.
277+
278+
Rejected:
279+
- `allowed` item requirements: unnecessary new algebra when exact-empty lists express the product state;
280+
- generic finding construction in core: finding keys and rendering remain format-owned;
281+
- JSON-only exact-list behavior: repeats a universal operation and leaves format behavior inconsistent;
282+
- whole-list waiver: permits unrelated list drift;
283+
- compatibility fields, aliases, and dual core versions.
284+
285+
## Implementation Stops
286+
Return to architecture if implementation requires:
287+
- a new assertion verb;
288+
- format or product names in file-engine core;
289+
- finding construction in core;
290+
- different difference semantics by format;
291+
- a committed path dependency;
292+
- an engine that cannot preserve its current exact desired bytes;
293+
- a caller left on core 0.7 in the coordinated downstream graph.
294+
295+
## Review Result
296+
- Review found that exact-empty TOML fields lost absence information and that compatible exact/member assertions can produce separately attributed findings.
297+
- Corrections add presence-aware TOML entry points over the known-present reconciler, migrate every optional caller, preserve same-identity compatible findings, and define patch releases for every runtime caller.
298+
- Confirmation review found no remaining architectural blocker.

‎.worklogs/2026-07-15-212556-exact-list-differences-plan.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Exact List Differences Plan
2+
3+
## Summary
4+
Designed the AQC prerequisite for member-specific exact-list findings across JSON, TOML, and YAML. The plan keeps the established list algebra and adds one universal read-only difference result.
5+
6+
## Decisions Made
7+
- Keep only `contains`, `excludes`, and `exact`; do not add allowed or closed assertions.
8+
- Compute multiset membership differences and order-only mismatches once in file-engine core.
9+
- Keep finding keys, rendering, mutation, and shape errors in format engines.
10+
- Add presence-aware TOML entry points that delegate to the existing known-present reconciler.
11+
- Preserve compatible exact-plus-contains/excludes findings separately so each policy retains attribution under one waiver identity.
12+
- Use patch releases accepted by existing downstream ranges; do not force unrelated package migrations.
13+
14+
## Key Files For Context
15+
- `.plans/2026-07-15-211236-exact-list-differences.md`
16+
- `packages/aqc-file-engine-core/src/merge/model.rs`
17+
- `packages/aqc-file-engine-core/src/merge/lists.rs`
18+
- `packages/file-types/json/aqc-json-file-engine/src/runtime/reconcile.rs`
19+
- `packages/file-types/toml/aqc-toml-engine-core/src/lists.rs`
20+
- `packages/file-types/yaml/aqc-pnpm-workspace-yaml-engine/src/runtime/reconcile/apply.rs`
21+
22+
## Next Steps
23+
- Extract an AQC Specular spec from the frozen plan.
24+
- Implement core, JSON, TOML, and YAML changes in dependency order.
25+
- Prove all three format families before releasing the patch chain.

0 commit comments

Comments
 (0)