fix(ce-doc-review): ask only where a real choice exists, batch the rest - #1373
Conversation
ce-doc-review asks the user to adjudicate individual findings rather than the decisions that resolve them. Two sampled runs collapsed 23 findings to 9 decisions and 27 to 8, and every duplicate pair was cross-persona -- invisible to the current section+title fingerprint. The plan gates the structural change behind an evidence pass that can conclude no new stage is needed, deletes the two narrow grouping rules it would supersede rather than stacking a third, and requires a paired cross-host eval judged against a measured variance band before it ships. No numeric cap on findings: that mechanism was evaluated and rejected in this repo already. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…t size Three instructions told reviewers to emit more as documents grew: the security lens produced a finding per uncovered attack-surface element, and the adversarial persona capped Quick at three findings, scaled Standard to "decision density", and ran multiple passes at Deep. Depth now selects which techniques run and how far they trace, never how many findings come out. Also drops the synthesis claim that the routing menu handles volume -- the menu is a dismissal path and implements no volume control, so the sentence licensed broad surfacing against a promise nothing downstream keeps. The anchor rubric is what holds the actionable tier. The same paragraph cited an external review threshold by name and number. An agent reading this file cannot resolve that reference, so it carried no information; replaced with the actual reason the floor sits where it does. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
The three seeded fixtures carried 57 inline annotations naming each planted defect's expected classification -- (Seeded gated_auto: ...), (Seeded manual: ...) -- directly in body prose, plus a header comment announcing the file as a test fixture. Any reviewer run against them was told what to conclude, so the fixtures could not measure reviewer judgment at all. Expectations now live in per-fixture .expectations.md sidecars the fixture body never references. The planted defects themselves are untouched: stale cross-references, drifted counts, terminology drift, and unresolved design tensions all survive, minus the labels describing them. Frontmatter now states provenance explicitly instead of omitting it. Absent provenance read as greenfield and force-activated the adversarial reviewer on all three, making activation an accident of omission; two fixtures now suppress premise-level techniques and one stays deliberately greenfield so both paths are covered. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
… fixtures The corpus had no control and no way to diagnose the empty advisory tier. Two fixtures close both gaps. negative-control-plan.md carries four mechanical defects that each resolve through a different question, with no premise-level challenge and no shared resolving question anywhere. Grouping logic has nothing to act on, which is the point: a change in grouping cannot move this fixture's result, so movement means the change did something unintended. Its sidecar deliberately predicts no band -- the pre-change spread is measured at the eval's trial count and recorded before any post-change arm runs. seeded-advisory-plan.md discriminates two causes of the same symptom. Reviewers emit almost nothing at the advisory level, which happens either because they round advisory observations up into the actionable tier or because they suppress them entirely under the false-positive catalog -- opposite fixes. It plants four genuinely advisory items, three catalog-suppressible ones, and two actionable ones as a floor, so a run that produces nothing is distinguishable from one that correctly suppressed. The sidecar fixes the reading in advance. Both sidecars record where the advisory boundary is thin, so a runner sees the caveat before the result rather than after. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Sampling returned its verdict and it inverts the plan's priorities. Both observed cross-persona duplicates defeated the syntactic matcher -- one shared an identical section and differed only in title wording -- and both originated as persona territory bleed, on an issue where a third persona found the same thing and correctly suppressed it by citing its own suppress list. So the cheap layers were never tested before the expensive one was planned. The plan now ships in evidence order: replace the section+title fingerprint with a one-fix reasoning test, scope the two persona boundaries sampling exposed, and route entailed findings to their owning unit. Decision clustering and the presentation-unit change become U12, built only if measurement shows the load is still too high, with the envelope preserved until then so no caller changes. Also records what the sampling cannot support: both sampled plans were already implemented and already reviewed, and ran four personas against the motivating runs' five to seven, so the ~13% duplication rate is a floor. The causal finding is the load-bearing result; the rate is not. U-IDs are preserved where the concept survived; U12 and U13 are new. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…osed Both cross-persona duplicates found while sampling real reviews were one reviewer working in another's declared lens, and both suppress lists already looked complete. The gaps were narrower than that. Adversarial ceded "technical feasibility or architecture conflicts" to feasibility, but whether a document still matches the codebase is currency, not feasibility -- a different question the list never named. It now cedes stale baselines, line references that no longer point at what they describe, and work the plan proposes that has already shipped. Scope-guardian ceded missing requirements to coherence but never ceded internal contradictions, so a unit whose tests contradicted its own stated scope read as scope territory. It is two passages disagreeing, which is coherence's. Scope-guardian judges whether the scope is right, not whether the document is self-consistent about it. No boundary was added where both personas have a legitimate claim. Those go to the matcher; suppressing one there would silence a real angle, which is worse than a surviving duplicate. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…quality Step 3.3 fingerprinted findings on normalize(section) + normalize(title), so two reviewers describing one problem in different words produced two decisions. Sampling caught it twice, and the sharper case shared a section exactly -- coherence and scope-guardian raised the same contradiction at the same unit, quoting the same three evidence lines, and survived as separate findings purely on title wording. The test is now whether one fix would resolve both, judged by reading title, section, why_it_matters, evidence, and suggested_fix. It fails closed: unclear means do not merge, because a surviving duplicate costs one line while a wrong merge hides a real concern where nothing signals its absence. Merges stay non-destructive -- round-to-round memory keys on a single finding's section, title, and evidence, so collapsing the constituents would make every settled finding re-raise next round. Deletes both narrower mechanisms this supersedes: same-persona premise collapse, whose three-variant floor is what let the observed pairs through, and premise-dependency chain linking, which could only model a rejection cascade from framing-level roots. With them go the chain-rendering rules, the dependents sub-block, the chains footnote, and the walk-through's cascade machinery. The withdrawal-precedence rule inside that machinery survived and was rewritten -- it governs how an explicit user decision outranks an inferred withdrawal, which is not cascade behavior. Net 114 lines removed. The output envelope is unchanged, so no caller moves. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…on queue In a 27-finding review, 8 of the 16 distinct issues needed no judgment at all -- contradictions the document resolved elsewhere, a missing owner for behavior it already required, a callsite its own decision implied. Every one was presented as a decision. A finding is now an obligation when the question that resolves it is already answered elsewhere in the document. Obligations group under the implementation unit they affect and are confirmed together, so the user makes one decision about settled work instead of N. The group renders in full first -- a batch confirmation with nothing above it is a rubber stamp, not a decision. Concreteness is explicitly not authority: a fix introducing a new user-visible state, limit, failure policy, retention rule, or operational commitment stays a decision no matter how concrete it is. That boundary is what keeps this from becoming a quiet auto-apply path. Obligations count as proposed fixes in the exported totals rather than getting their own bucket. A review whose findings are all obligations must still report actionable items, or a caller gating on that sum would hide the confirmation step and the user would never see work the review found. The envelope shape is unchanged, so no caller moves. The obligation block is specified once in the rendering floor and mapped by each surface, matching how the floor already handles the Open Questions entry: token policy and consequence-first phrasing, but no recommendation field, because there is no decision to recommend. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…there is no choice A real run on a sibling worktree's plan reported "No decisions requiring judgment" in its own summary line and then surfaced 11 items for confirmation. Judged against whether the author would realistically decline, about four had a genuine fork. The rest were corrections the document had already decided and not caught up with, including one the plan itself cited as a known measured failure. The contradiction is already inside the skill. The reviewer contract says a finding whose only alternatives are strawmen is gated_auto rather than manual, and defines manual as genuinely multiple valid approaches. So gated_auto already means no genuine alternative exists -- and the pipeline then routes it into a per-item Apply/Defer/Skip prompt, asking the author to choose between options the rubric just asserted are not there. Eleven such prompts per review teach acceptance without reading, which is what destroys the confirmations that do matter. U14 routes on decision entropy instead of the edit's blast radius: settled problem with no alternative applies and is reported, settled problem with real options asks which remedy rather than whether, and an arguable problem stays a decision. It also fixes the misclassification the same rubric forbids -- the observed P0 had two defensible resolutions and still arrived as gated_auto because a concrete fix was attached. Sequenced after U9 so it does not move the baseline mid-measurement, and because it may cut the load enough that U12 is unnecessary. U12's cancel-or-build verdict now reads the re-measurement after U14. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Routing decides which findings need the user; rendering has to signal which is which. Fixing only the first would ship a correct queue that still reads ambiguously. Today "Proposed fixes" and "Decisions" render identically -- both open with Recommendation: <Apply | Defer | Skip>, same fields, same order, distinguished only by a bucket header. Scanning the output, a reader cannot separate "this is what I am doing" from "this is what I need from you" without tracking which header they are under. The observed run states both at once in a single summary line: "11 proposed fixes remain" beside "No decisions requiring judgment". That is the ambiguity surfacing as a self-contradiction -- if nothing needs judgment, nothing remains for the user. U14 now gives each speech act its own grammar in the rendering floor, which every surface already maps onto. A report is settled tense with no recommendation field and no offered actions. A question carries its options and names what differs between them; a question with one option is a report wearing a question mark. The summary line counts changes made and choices requested separately. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Authored by an independent Codex session working from the U8/U9 handoff, which held the generated manifests and the distinct-pair truth outside the session that wrote the matcher. Committed here unmodified. The gate that mattered passed cleanly. Merge precision was 100% in every host/model cell across 24 accepted attempts, with zero wrong merges and zero merges against the pre-registered zero-duplicate control. Recall rose from 66.7% to 100% on Opus and from 26.7% to 100% on Codex Luna. The reasoning matcher is safe on a corpus its author never saw. The load-reduction gate did not pass, and the record says so: no host/model's mean reduction exceeded its own within-arm spread. The cancellation rests on reliable new-arm behaviour and on reading the residual failures as weaker-model instruction-following, not on a demonstrated load win. Procedurally it held the lines that would have silently invalidated the result: the installed skill was never invoked, an independent scorer produced the aggregates without returning the key, all five rejected attempts are preserved, and N=3 expanded to 7 only on wide cells. Two things this commit does not yet resolve. The plan edits reference U14 nowhere, so the cancellation trail cites U9 where the current sequencing routes that verdict through the post-U14 re-measurement -- same conclusion, wrong path. And Haiku and Sol showed unstable recall and empty output, a portability defect no unit currently owns. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Two gaps the eval commit left open. The cancellation of U12 and U7 was decided from a handoff written before U14 existed, so its reasoning cites U9's load number where this plan routes that verdict through the post-U14 re-measurement. The verdict is kept and the trail corrected: U9 already bounded load at 5-8 choices on the reliable cells and U14 only reduces it further, so no result U14 could produce would revive the case for clustering. Stage E no longer holds U12; the sequence ends at guards and learnings. U15 takes the finding nothing owned. Claude Haiku and Codex Sol showed unstable recall across a 0-100% range and produced runs with zero decision load -- output implausible enough to be instruction-following failure rather than an empty review. This skill ships to five harnesses, so that is a portability defect. It is scoped as reliability, not safety, because merge precision held at 100% on those same unstable cells with zero wrong merges. That is the fail-closed design working: a model that cannot judge whether one fix resolves both findings declines to merge instead of guessing. Weaker models under-perform; they do not corrupt. So shipping is not blocked on U15, and no fix inside it may trade precision for recall. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…e is no choice A real run reported "No decisions requiring judgment" in its own summary line and then surfaced 11 items for confirmation. Judged against whether the author would realistically decline, about four had a genuine fork. The contradiction was already inside the skill. The reviewer contract classifies manual precisely when genuinely different approaches exist, and sends a finding to gated_auto when its only alternatives are strawmen. So gated_auto already asserts no real alternative exists -- and routing then asked the author to choose between options the classification had just ruled out. Eleven such prompts per review teach acceptance without reading, which is what destroys the confirmations that matter. Routing now reads the two claims a finding carries. The anchor scores the problem-claim, autofix_class the remedy-claim. Confirmed problem with no alternative applies and is reported; well-evidenced problem with no alternative joins one grouped confirmation shown in full first; a real fork asks which remedy rather than whether to proceed. That last question shape did not exist -- Apply meant accepting the framing and the remedy together, with no way to say the problem is real but choose differently. It lands as a sub-question so the four-option menu stays the closed set it declares itself to be. Two safeguards needed rewiring because gated_auto now applies. The cross-model peer rule worked by demoting peer-only safe_auto to gated_auto, which no longer protects anything, so it targets the route directly: a peer-only finding never routes to Apply at any anchor or class. And the emission rubric now states that a concrete suggested_fix never outranks a real alternative, since misclassifying a fork as gated_auto is what would put a scope change onto an unattended path. The rendering floor gains the grammar to tell these apart: a report is settled tense with no offered actions, a question carries its options and names what differs, and a question with one option is a report wearing a question mark. The summary line counts changes and choices separately. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…Sol check Recall parity on weak models is no longer a goal. Most users do not run this on a Haiku-class model, and the one-fix test works because it is a genuine judgment that fails closed. A formulation a weak model could apply confidently would likely be more confidently wrong on every model -- trading 100% precision, which the blind eval actually proved, for recall on a tier that barely matters. Haiku's 69.8% against Opus's 100% is real, and the right response is to run a capable model for a judgment task. One thing still deserves a look. Haiku degrades like a capability curve. Codex Sol does not: its old arm returned 0% recall flat across all seven runs, and its new arm produced runs with zero decision load. A flat zero is the shape of something structurally wrong with the route -- a truncated payload, a schema mismatch, an output contract silently unmet -- not of a weaker reader doing worse. If that is a defect in how the route is driven, it reaches everyone on Codex at any tier. So the unit is now two runs and a paragraph: inspect Sol's raw returns rather than its scores, distinguish a non-conforming return from a poor judgment, fix it only if it is a route defect, and otherwise record the floor. The answer may simply be a dispatch note -- this skill already selects models per persona, so "synthesis runs on a capable tier" could be one line instead of a workstream. The floor gets written down either way. Deciding not to chase something is only a decision if it is recorded; otherwise it is rediscovered. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…split The sibling-worktree run has a second half. After surfacing 11 items under a summary line reading "No decisions requiring judgment", the agent was challenged and re-read its own findings: "9 of the 11 have no defensible 'no' -- they're corrections, not choices. Two carry real product judgment." My earlier hand-count guessed four; the agent that wrote the findings says two. That replaces an estimate with a measurement, and it sharpens why this is a routing defect rather than a judgment one: nothing new was gathered between surfacing the 11 and concluding 9 were not choices. The agent could already tell. The routing simply never asked it to. It also shows the same run failing in both directions. It asked about nine items with no defensible refusal, then -- once pushed -- decided the two that carried real product judgment, changing an acceptance example and combining two reviewers' asks into something neither requested. It disclosed both rather than burying them, which is honest, but it made the calls. Current routing interrupts where there is nothing to choose and commits where there is. U14 now carries a concrete acceptance target instead of a rough one: that document should surface two which-remedy questions and apply the other nine as reported changes. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
A blind eval on U14 returned "does not hold" and named the cause: the new routing table in 3.7 was added without removing the rules it replaced, so the skill carried two contradictory sets of instructions and models followed different ones on different runs. Three conflicts, all mine: Phase 4's apply step still said "apply only safe_auto at anchor 100" and described anchor-75 safe_auto as demoted into the walk-through -- the old 3.7 behaviour, contradicting the new table where anchor-100 gated_auto applies and anchor-75 joins a grouped confirmation. Phase 4's routing step still split the remainder by autofix_class, sending every actionable gated_auto and manual finding into the routing question. Under the new table gated_auto at anchor 100 has already applied and manual is the only decision class. The rendering floor required an unmistakable recommendation on every actionable finding while the new report grammar forbids one on a reported change, with no scope separating them. Phase 4 now routes by the route 3.7 assigned rather than re-deriving from the class, and the field-order rule is scoped to findings the reader is actually asked about, with the report grammar winning where they appear to conflict. The eval's other results stand: zero false applies in every U14 trial, so the routing design is safe -- the instability came from the contradiction, not the concept. False asks did drop where the new rule won (10 -> 6), and the 2/9 target was never reached because the outcome depended on which instruction the model happened to follow. Re-measurement is required before U14 can be called verified; this commit does not claim that. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Re-measurement after the Phase 4 reconciliation showed the contradiction was fixed -- no run reverted to per-item confirmations, false apply stayed zero across seven trials -- but only 2 to 4 of 9 corrections reached the change list. The rest were still being asked about. The cause is in the routing table. Apply was gated on anchor 100, and a real review returns almost everything at 75: the finding set under test is 1 finding at 100, 23 at 75, 7 at 50. So the Apply path was nearly empty by construction and everything with no genuine alternative fell into the grouped confirmation instead. That gate was inherited from the old safe_auto rule, where it was right -- silent application of a mechanical edit deserves the strongest evidence. It is wrong on the new axis. The anchor scores the problem-claim; Apply turns on the remedy-claim. Anchor 75 means the reviewer double-checked and the issue will be hit in practice, which is a settled problem, not a doubtful one. Requiring 100 mixed the two axes back together -- the exact confusion this step exists to undo. Anchors 75 and 100 now route identically. What separates Apply from a decision is whether an alternative exists, and nothing else. The grouped confirmation keeps obligations and peer-only findings, which are there for reasons unrelated to evidence strength. The rationale is written beside the table so it does not get "corrected" back. Not yet re-measured; this does not claim U14 holds. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Two failed evaluations traced to the same cause: a routing rule changed in one place while seven downstream surfaces went on describing the old behaviour. The skill carried two answers and models picked either one, which is why results came back unstable rather than uniformly wrong. This sweeps the rest. SKILL.md still told every run that gated_auto findings are returned as text rather than applied -- it is the always-loaded entry point, so that alone would have kept the contradiction in every review. The findings schema still defined gated_auto as "warrants user confirmation". The output template's worked example showed two gated_auto findings sitting in P1/P2 awaiting a walk-through, which the new taxonomy makes impossible; the example, its summary line, and its Coverage counts are rebalanced so it is internally consistent again. Bulk-preview scopes, the walk-through's intake and skip conditions, and the persona tier rubric all follow. Bucket names, exported counts, the Findings-equals-sum invariant, and the Review complete signal are unchanged, so no caller moves. One change has behavioural reach beyond renaming. Three safeguards used "downgrade to gated_auto" as the mechanism for making the user look at something -- the persona strawman rule, the oversized-suggested_fix gate, and synthesis 3.6's strawman downgrade. That downgrade is now a no-op because both tiers apply. All three retarget to manual, which is what the rubric already implies: a finding with a real alternative is a decision. Findings that previously landed in the confirmation bucket via those safeguards now reach the decision surface, which is the intended direction. Two documentation gaps were found and deliberately not invented around: the ordering of the grouped confirmation relative to the routing question, and how Coverage attributes a peer-only finding. Both predate this change; both are now more visible. Not yet re-measured. This does not claim U14 holds. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…IDs at emission Two changes, both aimed at the failure the third evaluation exposed rather than at plumbing. Uncertainty now has its own route. Measurement showed a clean pendulum: gating Apply at anchor 100 gave zero false applies but reported only 2-4 of 9 corrections; opening it to anchor 75 reported 6-9 of 9 but applied a real product fork in six of seven runs. Each trial's forks-asked plus false-applies came to two, which is the whole story -- routing works, and the model cannot reliably tell which findings carry a genuine choice. A binary apply-or-ask forces a guess under that uncertainty, and both wrong answers are expensive: applying decides for the author, asking spends a prompt on a foregone conclusion. The grouped confirmation costs one question for the batch, so a finding that lands there wrongly costs a glance. Uncertainty now goes there. Same fail-closed reasoning the merge test already uses, applied to the routing axis. Identifiers now get their handle from the reviewer that raised them. The floor already banned bare identifiers, but assigned the lookup to rendering -- which runs after a long dispatch has filled context with reviewer returns, and was being asked to re-derive fifteen handles from a document it may no longer hold. So bare IDs survived into output with the rule in force: "U1 claims R12, R13, R18" is four anchors in a sentence capped at two. The reviewer has the document open and knows what R12 means at no cost. It now writes the handle into the finding, and rendering renders what it is given. The schema's section description was asking for a bare section name and now asks for the same thing the template does. This is not only legibility. Synthesis decides whether two findings describe one problem by comparing what they say, so two reviewers who both write "R15" instead of naming the rule may not read as describing one issue -- opaque identifiers plausibly cost merge recall as well as comprehension. Not yet re-measured. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…onfirmation Four evaluation rounds said U14 does not hold, and the reason was the design rather than the tuning. Letting gated_auto apply unattended reported far more corrections -- 7 to 9 of 9 against 2 to 4 -- but applied a genuine product fork in most runs, because the model cannot reliably tell which findings carry a real choice. Offering it a route for its own uncertainty changed nothing: it never used that route, since it does not experience the uncertainty as uncertainty. It decides, and is sometimes wrong. So the two problems are separated instead of being solved by one rule. Only safe_auto at anchor 100 applies unattended. Everything else with a concrete fix goes to the grouped confirmation -- one question over the batch, rendered in full first. Volume is fixed by batching; authority is fixed by keeping it attended. A wrong classification now costs the reader a glance instead of an unrequested change to their document. The dead uncertainty route is deleted rather than qualified. It also contradicted the misclassification guard on the same close call, which is the shape that produced unstable results in the first place. The guard survives, reframed: with nothing auto-applying, the residual risk is a real fork buried in a batch the reader skims, so a close call between gated_auto and manual goes to manual. The remaining "do not prompt where there is nothing to choose" rule was literally false once foreseeable findings started landing in a confirmation. Rewritten to say what it now means: do not spend a separate question on each of them, because eleven foregone questions teach acceptance without reading. Batching is the remedy, not silence. Downstream surfaces swept for the fourth time: the persona tier rubric, the schema description, the envelope, the Tier column, and the worked example, whose applied-fixes list had two gated_auto findings in it. The example's counts were rebalanced and verified -- every persona row sums to its Findings value and the totals match the summary line. Not measured. This is a retreat to a posture whose safety property held across seven trials in an earlier round, with the volume win relocated to batching. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…tually showed The plan described intentions that no longer match what shipped, which would give anyone reading it cold a wrong picture. U14 is marked shipped-narrowed with its target unmet, and carries what each round established: rounds 1-2 failed on reconciliation rather than design, and their signature was instability rather than uniform failure; round 3 exposed a clean pendulum between safety and usefulness; round 4 showed that routing a model's own uncertainty somewhere safe does nothing because it does not experience the uncertainty. Two changes -- identifier glossing and the report-versus-question grammar -- were never measured, because a frozen finding set freezes reviewer behaviour along with reviewer variance and cannot see an emission-layer change at all. U12's cancellation is flagged as reopenable. It read a decision-load number off the generated corpus, and that corpus returned 100% merge recall where the one organic pair returned 20-36%. If duplicates survive at that rate on real documents, the number the cancellation rested on does not describe real reviews. U15's rescope is flagged the same way: its weak-cell numbers came from the same inputs, so they may describe the corpus rather than the model. The Codex Sol check survives -- a flat zero across seven runs is not a shape easy inputs explain. And the corpus finding is recorded in its own right, because it outlives this plan. A corpus you author to test a mechanism tends to contain that mechanism's happy path. The blind-manifest discipline here guarded against a biased scorer and did nothing about easy inputs, so four tuning rounds ran against material that could not fail us. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…ect what they disproved Two new learnings, both about evaluation rather than about doc review, and both general enough to bite any skill measured the same way. The first is that a corpus you author to test a mechanism tends to contain that mechanism's happy path. The duplicate matcher scored 100% recall on generated plans and 26% on one pair taken from a real review, with no change to the matcher in between. Blinding the scorer was done carefully and protected against the wrong failure: withholding the manifest stops a biased grader, and says nothing about whether the inputs are representative. Sampling real documents instead has its own version of the same bias -- the plans sampled had already been reviewed, so their defects were already gone, and the duplication rate measured off them was a floor rather than the rate. The second is that a frozen finding set cannot see a change to how findings are produced. Freezing removes reviewer variance by removing reviewer execution, which are the same act, so the harness is structurally blind upstream of the freeze point. It reports normally while doing this: an unmeasurable change and an ineffective change produce the same null result. Three existing docs described behaviour that no longer exists. `confidence-anchored-scoring` claimed the routing menu absorbs volume and that dismissing a finding is cheap -- that assumption is what produced a 34-finding review, since the cost is not per-keystroke but the reader holding thirty open questions at once. The `>= 50` threshold survives; the reasoning under it is corrected, and its port criteria now require batching. `ce-doc-review-calibration-patterns` documented the deleted 3.5c chain-linking machinery as though implementable; it now says the mechanism is gone and why, keeping the peer-versus-nested reasoning for anyone who revisits the problem. `paired-old-vs-new- injection-skill-evals` cited a fixture answer-key leak that has since been closed by moving the keys into sidecars, and gains the two new leak classes. The rendering-floor doc gains the behaviour-rule case. Its own thesis -- a contract duplicated across surfaces is a latent divergence -- turned out to apply harder to routing than to presentation, which drifted three times in a row across eight restatements with no single source. A stale presentation rule produces ugly output; a stale routing rule produces wrong action, and the always-loaded file is the most dangerous copy because it is the least likely to be reread. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…r-finding decisions The user-facing page described three-tier routing with gated_auto and manual both landing as "user decision," a four-option section keyed on findings landing in those tiers, and a worked example whose summary line called three proposed fixes "3 decisions." None of that matches what ships: routing now reads confidence and fix class together, only a full-confidence mechanical correction applies unattended, anything that touches meaning arrives batched, and a question is reserved for a real fork between remedies. The example's summary line was the sharpest case, since the rendering floor exists partly to prevent exactly it -- counting changes made and choices requested as one number, and describing items as awaiting the reader when none is. This is the drift class the rendering-floor doc now documents, and it was found by checking rather than by anything failing, which is the point: the page is not covered by a parity test, so nothing would have caught it. U11 is marked complete with what it actually delivered, including that its step 3 is dropped -- it was conditional on U2's verdict and U2 was never run. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
PR SummaryCursor Bugbot is generating a summary for commit a884c90. Configure here. |
…finish the 3.3 sweep Review found three defects, all the same class this PR set out to fix: a rule changed in one place and its restatements left describing the old behavior. The serious one is that the grouped confirmation had no execution path. Synthesis routes most meaning-touching fixes to it and five files describe it, but no surface ever fired it -- the walk-through's entry says those findings have no per-finding entry, and the routing question only covers what remains. So a reader who picked "review each finding one by one" saw the batch presented as awaiting one answer and then never got asked, leaving every obligation, gated_auto, and anchor-75 safe_auto unapplied. The central mechanism of this change did nothing in interactive mode. It now fires as its own step between the applied changes and the routing question: batch rendered in full first, then one question that applies it, with an escape hatch to step through and an option to apply none. The routing question is scoped to the decision surface, and the skip rule, bulk-preview scope, and Phase 4 route list follow that split. The second is that 3.4 still matched cross-model peers with the section-plus-title fingerprint and its twin evidence-overlap exception, both deleted when 3.3 became a reasoning test. Merging is now decided once in 3.3 and read by 3.4, and the same stale vocabulary is cleared from the cross-model review contract, its eval scenarios, and the whole-doc persona. Round-to-round memory keeps its own fingerprint -- that one is live, and now says why it does not share 3.3's. The third is that the walk-through told the renderer to look identifier handles up in the document while the floor says the handle arrives with the finding and rendering must not rebuild it. The walk-through described the floor's defect path as the normal one. One test moved with the code. It pinned the literal string "cannot use the twin fingerprint exception" -- prose describing a mechanism this PR had already deleted, which survived only because the stale sentence survived. It now pins the rule that outlives the mechanism: an unverified peer cannot trigger anchor promotion. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…e decision surface Batching the confirmation moved the volume without moving the comprehension. The batch still rendered as N independent entries, which costs a reader exactly what N questions cost -- one answer at the bottom does not help if understanding it still takes eleven reads. The floor now carries a "Presenting a batch" rule: lead with what the batch does as a whole, head each group with the consequence its members share, keep every member visible, and do not manufacture a theme where none exists. The axis that helps is the change itself -- one root cause, one kind of edit, one section. Severity and reviewer sort a list without helping anyone decide. The rule states the goal and leaves the judgment to the agent, which is the layer that can actually do it: synthesis holds every finding at once and the reader holds none of them. Finding the two or three real themes in a batch of eleven is what this step is for. Review also caught that the confirmation pointed at a template section that did not exist. The interactive template filed grouped-confirmation members into the same P0-P3 tables as the decision surface, so "apply all of them" could have swept a genuine fork into a batch answer -- the exact failure this work set out to prevent -- or missed batch members buried in those tables. The non-interactive envelope already separated the two; the interactive template never did. It now renders Proposed fixes as its own section beside Implementation obligations, those two sections are the batch and nothing else is, and the P-level sections carry decisions only. A gated_auto row appearing in a P-level table now means routing went wrong rather than rendering being loose. The floor owns the rule; the walk-through, the template, and the envelope reference it instead of restating it. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a884c90840
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ut units Obligations route to the implementation unit they affect, which assumes every reviewed document has units. Requirements-shaped documents do not, and an obligation is defined by the document already entailing the fix -- an entailed contradiction or an unowned requirement arises there just as readily. Such a finding had nowhere to go, so it would be dropped or filed under a unit that does not exist. They now route to the part of the document they affect: the implementation unit where the document has units, the owning section where it does not. The section is titled "Entailed corrections" in that case, since "Implementation obligations" names something a requirements document does not contain, and the walk-through refers to the obligations section by role so it holds under either title. The floor owns the destination rule; synthesis, the template, and the envelope follow it. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 216b6850eb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Review found four defects in the grouped-confirmation work itself. Stepping through the batch individually dropped the decision surface. Option B enters the per-finding loop, and that loop's terminal contract emits the completion report and moves to Phase 5 -- so a reader who chose to review the batch one at a time ended the run with their decisions never asked. The same disappearance the confirmation step was added to fix, one layer down. It now returns to the routing question when the batch is exhausted. The which-remedy question could not fire in the case the routing table promised it for. The table said a manual finding means real alternatives exist, so ask which remedy -- but the reviewer contract commits suggested_fix to one recommendation and forbids alternative menus outright, so an ordinary manual finding arrives with one fix or none and has no menu to offer. The question is not dead, it was mis-scoped: 3.5 turns a contradiction between personas into one finding holding both perspectives, and that is a genuine fork. It now fires there, the table no longer promises it everywhere, and the note says plainly not to invent a second option to make the fork appear. The non-interactive envelope claimed confirmation it never obtained. That mode asks nothing and returns the batch unapplied, yet the envelope told the caller the obligations were "confirmed as a group" and the proposed fixes "confirmed with" them -- inviting a caller to treat unapproved changes as accepted. Both headers now say awaiting. And the obligations fix that shipped an hour ago was half applied: only the first group placeholder became "unit or section name", the second still said "unit name", and the retitle instruction sat inside the fence that is itself the emitted output. A requirements review could still invent a unit heading, and a bracketed instruction could leak into caller- parsed text. The instruction now lives in prose above the fence, where the rest of the template's guidance already is. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b72a35df7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…g 50 actionable Three more from review, and the first one is the interesting one. The batch-only pass still had an exit that ended the run. The previous fix covered normal exhaustion of the loop; "Auto-resolve with best judgment on the rest" leaves it through bulk-preview, which emits the completion report, so the decision surface disappeared again by a different door. Patching that door would have invited a third. The rule is now stated once at the loop: in batch context the walk-through is a subroutine with exactly one exit, back to the routing question, and it never emits the completion report -- whatever the exit is called, and including ones added later. Cross-section contradictions had nowhere to land. 3.3 applies its one-fix test across sections and hands opposing recommendations to 3.5 rather than merging them, but 3.5 only combined disagreements on the same section. A cut-versus-keep pair split across two sections therefore survived as two independent decisions, and the best-judgment route would execute both -- applying mutually incompatible fixes. 3.5 now resolves a disagreement wherever it was noticed, because the section is where a contradiction is spotted and the decision is what makes it one. Anchor 50 was called the actionable floor two lines below a bullet saying it enters nothing. It is the surfacing floor; 75 is the actionable one. While correcting that paragraph: it still argued that surfacing low is safe because the routing menu makes dismissal cheap. That is the exact reasoning documented as the cause of the thirty-four finding review, and it had survived in the skill while the learning that disproves it sat in docs. The real argument is narrower and still holds -- a 50 costs one line in an observational subsection and never becomes a question -- and it holds only because 50 stays out of the pipeline. The same stale premise is corrected in confidence-anchored-scoring's port criteria, which asserted it one section below its own correction. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 31ac8fbdf6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
bulk-preview said two contradictory things about the batch, one from each of my last two commits. The header claimed pending always means the decision surface and the batch is never in scope here; the auto-resolve path then said that in batch context "the rest" is the rest of the batch. A model following the header previews nothing, applies nothing, and hands back a batch the reader was just told was about to land. The header was written when the preview served one pass. It now serves two, so scope is stated per pass: serving routing, pending is the decision surface and the batch is already answered; serving the grouped confirmation's batch pass, pending is the rest of that batch and the decision surface has not been routed yet. Site 3 reads against whichever pass is running, since it is the site that can appear in both. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
… and always report Two from review, both cases of a rule stated more broadly than its reason supports. The cross-model peer safeguard sent every peer-only finding to the grouped confirmation, at any class. Its reason is narrower: a peer cannot authorize an edit the reader never sees. Withholding apply authority is that reason; demoting a fork into the batch is not. A peer-only manual finding is still a genuine choice, and may carry no suggested_fix at all, so the batch would either sweep a scope decision under "apply all" or attempt an edit with no payload. It now diverts to the batch only where the table would have applied it, and a peer-only manual finding stays a decision. Four downstream restatements said the unconditional version and were reconciled with it. A batch-only run emitted no completion report. With a non-empty batch and an empty decision surface, routing is skipped -- and skipping routing jumped to the Phase 5 terminal question, past the report. The reader answered a confirmation, findings were applied or skipped, and nothing recorded which. Skipping routing now never skips the report, and the collapsed degenerate form is explicitly reserved for the run where nothing was ever put to the reader. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ea10dee01
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ions bouncing Five from review, every one a consequence of the previous two commits that I did not carry through. The completion-report instruction landed on Phase 4's shared remaining- findings path with no mode gate. An empty decision surface is the ordinary non-interactive case, so a chained run could have printed the interactive report beside or instead of the envelope its caller parses, or said "complete" before the confirmation was answered. It is now interactive- only and explicitly after the confirmation settles. The terminal-path list still named the skipped-routing case as the one-line collapse, and that list is read before the degenerate-case section, so a batch-only run could still collapse and drop the record the previous commit added. A no-fix obligation had no stable route. The obligation rule reclassified it to gated_auto; the routing table requires suggested_fix for that class and demotes it back to manual. It bounced, and either lost the obligation route or joined an apply-all batch with no payload. An obligation now takes gated_auto only when a fix was written; without one it stays a decision, because entailing a correction is not the same as having written it and the reader supplies the missing text. The obligation rendering rules in the template and the floor say so too -- a line whose form is consequence-plus-change cannot render a finding with no change. The presentation before the routing question was the pre-confirmation one. Answering the batch ends a turn, so that render happens after the batch is settled, yet it still listed every member under Proposed fixes and summarized them as awaiting one confirmation. It now renders current state: applied members in the applied list, skipped ones reported skipped, neither awaiting anything. And cross-model-eval still told the behavioral eval to assert that a peer-only manual finding caps at gated_auto -- rewarding the regression the last commit fixed. I swept four restatements of that rule and missed the one that grades it. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9a166392d7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…or reclaiming no-fix findings Two contradictions I introduced when rewriting the obligation rule last commit. The reclassification was widened from "manual becomes gated_auto" to "anything carrying a suggested_fix becomes gated_auto", which swept up safe_auto obligations -- demoting a mechanical correction that the same step, two paragraphs below, says still applies unattended at anchor 100. It now only ever moves manual up, and never touches safe_auto. And the no-fix exception was undone immediately after by the routing paragraph, which still sent every obligation to the grouped list. It now routes only the obligations that survived both rules, and says plainly that a no-fix one is not among them because it stayed a decision. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Every round of this PR's review targeted text an earlier round added, and every round I qualified it instead of deleting it. AGENTS.md says the opposite: prefer deleting the addition, and a section that has absorbed three rounds is itself the defect. The grouped confirmation had absorbed four, the obligation rule three, the anchor-50 rationale two. Collapsed, with each shortened rule re-checked against every path it served: - The grouped-confirmation section drops from ~450 words to ~250. Gone are "the failure this step exists to prevent", the paragraph justifying why A is recommended, the history of how the decision surface went missing, and a cross-reference to another section's invariant. The subroutine rule keeps "a bug whatever its name", which forecloses the next exit; that is a constraint, not rationale. - The obligation rule states three cases in one paragraph instead of three paragraphs arguing with each other. Collapsing it exposed that safe_auto was then covered twice, so the standing rule below keeps it and the forward reference is gone. - The anchor-50 paragraph loses its meta-commentary about the claim it replaced. The correction stays, because the disproven reasoning is a known tendency; the essay about it does not. - The peer safeguard, the bulk-preview scope header, the post-confirmation render, the skip-routing rule, and the degenerate-report case each lose a trailing sentence that restated the directive above it. Net effect: about 480 words removed from the accreted sections while synthesis lands +71 against main rather than +323. Nine invariants verified still named after the collapse, since a condensed rule that drops a path is a new defect rather than a simplification. The rendering floor's growth is deliberate and stays -- "Presenting a batch" is new capability, not accretion. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…the 75/safe_auto row Two tightenings of existing text, not new rules. The peer cap kept a peer-only manual finding out of safe_auto but still promoted it to gated_auto, and 3.7 batches gated_auto -- so the finding reached Apply-all through promotion after the 3.7 safeguard had been fixed to keep it off that surface. The cap now declines to promote a peer-only manual finding at all. And the 75/safe_auto row sent findings to the grouped confirmation with no suggested_fix guard, while all three neighbouring rows demote on a missing fix. suggested_fix is optional in the schema, so that row could put a finding with no edit into a batch whose Apply-all expects one. It now carries the same clause as its siblings. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 372e44af75
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…rning to routing The subroutine rule created a silent data-loss path. The walk-through accumulates Apply decisions in memory and dispatches them in a single pass at its terminal path -- which is exactly the path the batch exit skips. So a reader who stepped through the batch and approved fixes returned to routing with those edits still staged, and choosing report- only or append-to-Open-Questions there ended the run without ever dispatching them. Approved fixes silently did not land. Flushing is now part of the exit rather than of the report: run the accumulated Apply set, then return. bulk-preview says the same for the auto-resolve path, whose findings join that set. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 18643ea. Configure here.
Last commit made the batch exit write its accumulated Apply set but never empty it, so the decision pass's terminal dispatch would write the same members a second time. Flushing and clearing are one operation; the exit now does both, which leaves the terminal dispatch correct as written rather than needing a second rule about what to skip. The bulk-preview auto-resolve bullet had accumulated the round-5 and round-11 statements of the same routing rule side by side. Merged into one, thirty words shorter. Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
* fix(ce-commit-push-pr): root PR stacks on the parent PR the user named (EveryInc#1365) * fix(ce-babysit-pr): decode gh output as UTF-8 on Windows (EveryInc#1368) * fix(ce-prototype): cover decisions settled by seeing, not just driving (EveryInc#1369) * perf(tests): cut suite wall time by splitting the largest test file (EveryInc#1370) * fix(tests): stop the cross-model routes test reading the working tree (EveryInc#1371) * fix(ce-doc-review): ask only where a real choice exists, batch the rest (EveryInc#1373) * chore(orca): re-pin upstream provenance baseline to 421a337 Co-authored-by: Cursor <cursoragent@cursor.com> * fix(ce-doc-review): align Orca ownership wording with Apply routing Co-authored-by: Cursor <cursoragent@cursor.com> * feat(ce-prototype): add a seeing-mode craft floor and durable storage (EveryInc#1374) * fix(ce-pov): stop the panel guessing the cross-model host argument (EveryInc#1375) * chore(cross-model): pin the Grok peer to 4.6 (EveryInc#1376) * docs(skills): rewrite user skill pages for accuracy and clearer use (EveryInc#1377) * fix(commit): append known plan unit ids to commit subjects (EveryInc#1379) * fix(ce-work): stop sandboxed workers committing in linked worktrees (EveryInc#1382) * fix(ce-doc-review): edit HTML plans in native format (EveryInc#1381) * fix(ce-code-review): cover adversarial after quota or auth no-review (EveryInc#1380) * fix(skills): correct a rejected dispatch instead of spending the fallback (EveryInc#1383) * fix(ce-compound): find Claude sessions started outside the repo root (EveryInc#1378) * ci(windows-native): retry peer-job-runner smoke on ctypes flake (EveryInc#1384) * fix(ce-debug): stop asking at the handoff, stop shipping unoffered work (EveryInc#1385) * docs(solutions): record why skill gates state conditions, not git commands (EveryInc#1386) * fix(skills): drop the residual-findings record file for real sinks (EveryInc#1387) * fix(ce-doc-review): run the cross-model pass when CROSS_MODEL_PEERS is unset (EveryInc#1389) * fix(ce-proof): sync with current Proof v3 contract (EveryInc#1390) * fix(skill-authoring): make goal-first the default when authoring and reviewing skills (EveryInc#1391) * fix(cross-model): let reviews run on Fable and pin model/effort from CE config (EveryInc#1392) * docs(cross-model): point superseded peer benchmarks at the luna/xhigh decision (EveryInc#1393) * fix(cross-model): discover the Codex.app-bundled codex CLI and name the peer-CLI requirement (EveryInc#1395) * feat(cross-model): add cross_model_review_mode checkout egress gate (EveryInc#1396) * fix(ce-compound-refresh): compare knowledge-track learnings against guidance they name (EveryInc#1399) * docs(solutions): capture the named-guidance contradiction-check learning (EveryInc#1400) * fix(ce-compound): prefer the repo's own frontmatter vocabulary over the Rails-era enums (EveryInc#1394) * fix(ce-work): stop asking about branches before starting work (EveryInc#1397) * fix(review): answer covered cases on skill prose with the condition, not a patch (EveryInc#1401) * fix(scratch): fall back to $TMPDIR when /tmp cannot host the scratch root (EveryInc#1398) * feat(ce-skill-work): repo-local skill for authoring, editing, reviewing, and responding to review on skills (EveryInc#1402) * fix(ce-pov): reject non-final peer positions instead of folding them in (EveryInc#1403) * feat(manifest): add Agent Plugins v1.0.0 manifest support (EveryInc#1345) * chore: release main (EveryInc#1354) * fix(ce-work): run cross-model verification on warm checkouts (EveryInc#1404) * fix(orca): reconcile upstream skill contracts * fix(orca): preserve additive Codex session roots --------- Co-authored-by: Trevin Chow <trevin@trevinchow.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Summary
A real
ce-doc-reviewrun produced 34 findings and handed the reader all of them. Reviewing that output showed the volume was the smaller half of the problem. The skill was asking about things whose answers were foregone, presenting changes it had already decided to make as if they were approvals, and naming findings by identifiers (U18,R6) that mean nothing to someone who does not have the document open.Those are three different defects with one shape in common: the skill could not tell the difference between telling you something and asking you something.
Routing now reads confidence and fix class together rather than mapping a tier straight to a prompt. Only a mechanical correction the reviewer is fully confident about applies unattended. Everything that touches meaning still gets applied — but batched into one grouped confirmation the reader sees in full before answering, rather than arriving as its own question. A question is reserved for a genuine fork, and it asks which remedy, never whether to proceed.
That separation is the fix. Batching addresses volume; keeping the batch attended preserves authority. Earlier rounds tried to solve both with one dial and produced a pendulum — loosen it and the agent silently rewrites meaning, tighten it and thirty questions come back.
The batch is presented by theme, not as a list. One answer over eleven entries only helps if the reader can also understand it once, so the confirmation leads with what the batch does as a whole, groups members by what they share, and keeps every member visible. A flat list of eleven costs the same reading as eleven questions did.
Two supporting changes:
U1was for free; the renderer, four thousand tokens later, does not. Asking the layer that lost the information to reconstruct it is why bare identifiers survived the previous rule.Around 114 net lines of cascade machinery came out — synthesis steps 3.3b and 3.5c built a dependency graph over findings before the model could reliably classify which findings carried a choice at all.
What the evaluation showed, including where it failed
The headline target was not met, and I am shipping anyway. Four rounds of blind cross-model evaluation ran against a corpus built for the purpose. It returned 100% merge recall in every cell. The same matcher, unchanged, scored 26% against a single duplicate pair taken from a real review.
The corpus was the weak link. A corpus you author to test a mechanism tends to contain that mechanism's happy path, and the blind-manifest discipline used here guarded the scorer while doing nothing about the inputs. Four tuning rounds ran against material that could not fail us.
Identifier glossing and the report-versus-question grammar are not measured at all, and could not have been: the harness freezes reviewer output to remove variance, which removes reviewer execution with it, so a change to what reviewers emit is invisible to it by construction.
What review changed
Twelve rounds of automated review (Cursor Bugbot and Codex) found 27 defects, all fixed. Two were user-visible data bugs rather than wording, and both were introduced by this PR's own repairs:
The rest were reconciliation drift — a rule changed in one place, its restatements left describing the old behaviour. That class accounts for most of the 27, across roughly eight restatement sites per rule.
A test moved with the code.
pipeline-review-contract.test.tspinned the literal stringcannot use the twin fingerprint exception— prose describing a mechanism this PR deleted, which passed only because the stale sentence survived. It now pins the rule that outlives the mechanism.One round was spent deleting rather than adding. Per this repo's accretion rule, a section that has absorbed three rounds of additions is itself the defect; the grouped-confirmation section had absorbed four. About 480 words came out of the accreted blocks, with nine invariants re-verified as still named afterward.
Also in here
.expectations.mdsidecars, closing a leak where an injected fixture told the agent what to find. Added a negative-control fixture and an advisory-discrimination fixture.confidence-anchored-scoring, whose claim that "the routing menu handles volume" is the assumption that produced the 34-finding review.Validation
bun run test— 3058 pass, 0 fail (122 files).bun run release:validate— metadata in sync, 33 skills._ctypesfailing to load on the Windows image), both passing on rerun with no code change.No behavioural eval is a gate, per the repo's CI-versus-behavioural split. Eval evidence and its limits live in
docs/solutions/skill-design/dedup-and-obligation-eval-results.md.Residual
U12(decision clustering) andU2(the empty advisory tier) are deliberately excluded — both build on measurements the corpus finding undermined.The routing and confirmation flow is now a small state machine — batch pass, decision pass, four exits, one deferred write — specified only in prose across three files. Five of the 27 defects were my own repairs perturbing it. It wants a mechanical guard asserting the Apply set is dispatched exactly once per path; that is the recommended follow-up.
Security Disclosure
No security-relevant changes. This PR touches skill prose, reference files, test fixtures, and documentation. No shell or exec paths, no path or input handling, no converter or writer output, no credential handling, no permission changes, and no dependency changes.
Agent Disclosure
Claude Code · claude-opus-5[1m]