Skip to content

fix(ce-doc-review): ask only where a real choice exists, batch the rest - #1373

Merged
tmchow merged 36 commits into
mainfrom
tmchow/fix-ce-doc-review
Aug 14, 2026
Merged

fix(ce-doc-review): ask only where a real choice exists, batch the rest#1373
tmchow merged 36 commits into
mainfrom
tmchow/fix-ce-doc-review

Conversation

@tmchow

@tmchow tmchow commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

A real ce-doc-review run produced 34 findings and handed the reader all of them. Reviewing that output showed the volume was the smaller half of the problem. The skill was asking about things whose answers were foregone, presenting changes it had already decided to make as if they were approvals, and naming findings by identifiers (U18, R6) that mean nothing to someone who does not have the document open.

Those are three different defects with one shape in common: the skill could not tell the difference between telling you something and asking you something.

Routing now reads confidence and fix class together rather than mapping a tier straight to a prompt. Only a mechanical correction the reviewer is fully confident about applies unattended. Everything that touches meaning still gets applied — but batched into one grouped confirmation the reader sees in full before answering, rather than arriving as its own question. A question is reserved for a genuine fork, and it asks which remedy, never whether to proceed.

That separation is the fix. Batching addresses volume; keeping the batch attended preserves authority. Earlier rounds tried to solve both with one dial and produced a pendulum — loosen it and the agent silently rewrites meaning, tighten it and thirty questions come back.

The batch is presented by theme, not as a list. One answer over eleven entries only helps if the reader can also understand it once, so the confirmation leads with what the batch does as a whole, groups members by what they share, and keeps every member visible. A flat list of eleven costs the same reading as eleven questions did.

Two supporting changes:

  • Duplicate detection was a string fingerprint and is now a reasoning test — two findings are duplicates when one fix would resolve both. It fails closed and merges non-destructively.
  • The handle arrives with the finding. The reviewer that raised it knew what U1 was for free; the renderer, four thousand tokens later, does not. Asking the layer that lost the information to reconstruct it is why bare identifiers survived the previous rule.

Around 114 net lines of cascade machinery came out — synthesis steps 3.3b and 3.5c built a dependency graph over findings before the model could reliably classify which findings carried a choice at all.

What the evaluation showed, including where it failed

The headline target was not met, and I am shipping anyway. Four rounds of blind cross-model evaluation ran against a corpus built for the purpose. It returned 100% merge recall in every cell. The same matcher, unchanged, scored 26% against a single duplicate pair taken from a real review.

The corpus was the weak link. A corpus you author to test a mechanism tends to contain that mechanism's happy path, and the blind-manifest discipline used here guarded the scorer while doing nothing about the inputs. Four tuning rounds ran against material that could not fail us.

Result Confidence
Merge precision 100% — no false merges on the generated set Holds; precision is the silent-failure direction and it was verified blind
Merge recall Distrust the number. 100% generated vs 26% organic
Persona territory gaps (2 closed) Holds — causal, not rate-based
Decision-load numbers that cancelled the clustering unit Suspect; the cancellation is marked reopenable

Identifier glossing and the report-versus-question grammar are not measured at all, and could not have been: the harness freezes reviewer output to remove variance, which removes reviewer execution with it, so a change to what reviewers emit is invisible to it by construction.

What review changed

Twelve rounds of automated review (Cursor Bugbot and Codex) found 27 defects, all fixed. Two were user-visible data bugs rather than wording, and both were introduced by this PR's own repairs:

  • The grouped confirmation had no apply path. Synthesis routed findings to it and five files described it, but no surface ever fired it — so a reader who chose per-finding review saw the batch presented as awaiting one answer, was never asked, and ended with every proposed fix unapplied.
  • The batch pass silently discarded approved edits. The walk-through defers Apply to a single pass at its terminal path; the batch exit skips that path. Fixes the reader explicitly approved stayed staged and never landed. The repair then had to clear the set too, or they would have landed twice.

The rest were reconciliation drift — a rule changed in one place, its restatements left describing the old behaviour. That class accounts for most of the 27, across roughly eight restatement sites per rule.

A test moved with the code. pipeline-review-contract.test.ts pinned the literal string cannot use the twin fingerprint exception — prose describing a mechanism this PR deleted, which passed only because the stale sentence survived. It now pins the rule that outlives the mechanism.

One round was spent deleting rather than adding. Per this repo's accretion rule, a section that has absorbed three rounds of additions is itself the defect; the grouped-confirmation section had absorbed four. About 480 words came out of the accreted blocks, with nine invariants re-verified as still named afterward.

Also in here

  • Fixture answer keys moved out of the reviewed text into .expectations.md sidecars, closing a leak where an injected fixture told the agent what to find. Added a negative-control fixture and an advisory-discrimination fixture.
  • Four documented learnings, two general: the authored-corpus lesson, and why a frozen-input harness cannot measure a change to the layer it froze. Corrected three existing docs — including confidence-anchored-scoring, whose claim that "the routing menu handles volume" is the assumption that produced the 34-finding review.

Validation

  • bun run test3058 pass, 0 fail (122 files).
  • bun run release:validate — metadata in sync, 33 skills.
  • CI green on the head; two failures during the run were runner infrastructure (a subprocess-reaping race, and _ctypes failing to load on the Windows image), both passing on rerun with no code change.

No behavioural eval is a gate, per the repo's CI-versus-behavioural split. Eval evidence and its limits live in docs/solutions/skill-design/dedup-and-obligation-eval-results.md.

Residual

U12 (decision clustering) and U2 (the empty advisory tier) are deliberately excluded — both build on measurements the corpus finding undermined.

The routing and confirmation flow is now a small state machine — batch pass, decision pass, four exits, one deferred write — specified only in prose across three files. Five of the 27 defects were my own repairs perturbing it. It wants a mechanical guard asserting the Apply set is dispatched exactly once per path; that is the recommended follow-up.

Security Disclosure

No security-relevant changes. This PR touches skill prose, reference files, test fixtures, and documentation. No shell or exec paths, no path or input handling, no converter or writer output, no credential handling, no permission changes, and no dependency changes.

Agent Disclosure

  • Model: Claude Code · claude-opus-5[1m]

tmchow added 23 commits August 13, 2026 18:05
ce-doc-review asks the user to adjudicate individual findings rather
than the decisions that resolve them. Two sampled runs collapsed 23
findings to 9 decisions and 27 to 8, and every duplicate pair was
cross-persona -- invisible to the current section+title fingerprint.

The plan gates the structural change behind an evidence pass that can
conclude no new stage is needed, deletes the two narrow grouping rules
it would supersede rather than stacking a third, and requires a paired
cross-host eval judged against a measured variance band before it
ships. No numeric cap on findings: that mechanism was evaluated and
rejected in this repo already.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…t size

Three instructions told reviewers to emit more as documents grew: the
security lens produced a finding per uncovered attack-surface element,
and the adversarial persona capped Quick at three findings, scaled
Standard to "decision density", and ran multiple passes at Deep. Depth
now selects which techniques run and how far they trace, never how many
findings come out.

Also drops the synthesis claim that the routing menu handles volume --
the menu is a dismissal path and implements no volume control, so the
sentence licensed broad surfacing against a promise nothing downstream
keeps. The anchor rubric is what holds the actionable tier.

The same paragraph cited an external review threshold by name and
number. An agent reading this file cannot resolve that reference, so it
carried no information; replaced with the actual reason the floor sits
where it does.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
The three seeded fixtures carried 57 inline annotations naming each
planted defect's expected classification -- (Seeded gated_auto: ...),
(Seeded manual: ...) -- directly in body prose, plus a header comment
announcing the file as a test fixture. Any reviewer run against them was
told what to conclude, so the fixtures could not measure reviewer
judgment at all.

Expectations now live in per-fixture .expectations.md sidecars the
fixture body never references. The planted defects themselves are
untouched: stale cross-references, drifted counts, terminology drift,
and unresolved design tensions all survive, minus the labels describing
them.

Frontmatter now states provenance explicitly instead of omitting it.
Absent provenance read as greenfield and force-activated the adversarial
reviewer on all three, making activation an accident of omission; two
fixtures now suppress premise-level techniques and one stays
deliberately greenfield so both paths are covered.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
… fixtures

The corpus had no control and no way to diagnose the empty advisory
tier. Two fixtures close both gaps.

negative-control-plan.md carries four mechanical defects that each
resolve through a different question, with no premise-level challenge
and no shared resolving question anywhere. Grouping logic has nothing to
act on, which is the point: a change in grouping cannot move this
fixture's result, so movement means the change did something
unintended. Its sidecar deliberately predicts no band -- the pre-change
spread is measured at the eval's trial count and recorded before any
post-change arm runs.

seeded-advisory-plan.md discriminates two causes of the same symptom.
Reviewers emit almost nothing at the advisory level, which happens
either because they round advisory observations up into the actionable
tier or because they suppress them entirely under the false-positive
catalog -- opposite fixes. It plants four genuinely advisory items,
three catalog-suppressible ones, and two actionable ones as a floor, so
a run that produces nothing is distinguishable from one that correctly
suppressed. The sidecar fixes the reading in advance.

Both sidecars record where the advisory boundary is thin, so a runner
sees the caveat before the result rather than after.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Sampling returned its verdict and it inverts the plan's priorities. Both
observed cross-persona duplicates defeated the syntactic matcher -- one
shared an identical section and differed only in title wording -- and
both originated as persona territory bleed, on an issue where a third
persona found the same thing and correctly suppressed it by citing its
own suppress list.

So the cheap layers were never tested before the expensive one was
planned. The plan now ships in evidence order: replace the
section+title fingerprint with a one-fix reasoning test, scope the two
persona boundaries sampling exposed, and route entailed findings to
their owning unit. Decision clustering and the presentation-unit change
become U12, built only if measurement shows the load is still too high,
with the envelope preserved until then so no caller changes.

Also records what the sampling cannot support: both sampled plans were
already implemented and already reviewed, and ran four personas against
the motivating runs' five to seven, so the ~13% duplication rate is a
floor. The causal finding is the load-bearing result; the rate is not.

U-IDs are preserved where the concept survived; U12 and U13 are new.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…osed

Both cross-persona duplicates found while sampling real reviews were one
reviewer working in another's declared lens, and both suppress lists
already looked complete. The gaps were narrower than that.

Adversarial ceded "technical feasibility or architecture conflicts" to
feasibility, but whether a document still matches the codebase is
currency, not feasibility -- a different question the list never named.
It now cedes stale baselines, line references that no longer point at
what they describe, and work the plan proposes that has already shipped.

Scope-guardian ceded missing requirements to coherence but never ceded
internal contradictions, so a unit whose tests contradicted its own
stated scope read as scope territory. It is two passages disagreeing,
which is coherence's. Scope-guardian judges whether the scope is right,
not whether the document is self-consistent about it.

No boundary was added where both personas have a legitimate claim.
Those go to the matcher; suppressing one there would silence a real
angle, which is worse than a surviving duplicate.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…quality

Step 3.3 fingerprinted findings on normalize(section) + normalize(title),
so two reviewers describing one problem in different words produced two
decisions. Sampling caught it twice, and the sharper case shared a
section exactly -- coherence and scope-guardian raised the same
contradiction at the same unit, quoting the same three evidence lines,
and survived as separate findings purely on title wording.

The test is now whether one fix would resolve both, judged by reading
title, section, why_it_matters, evidence, and suggested_fix. It fails
closed: unclear means do not merge, because a surviving duplicate costs
one line while a wrong merge hides a real concern where nothing signals
its absence. Merges stay non-destructive -- round-to-round memory keys
on a single finding's section, title, and evidence, so collapsing the
constituents would make every settled finding re-raise next round.

Deletes both narrower mechanisms this supersedes: same-persona premise
collapse, whose three-variant floor is what let the observed pairs
through, and premise-dependency chain linking, which could only model a
rejection cascade from framing-level roots. With them go the
chain-rendering rules, the dependents sub-block, the chains footnote,
and the walk-through's cascade machinery. The withdrawal-precedence rule
inside that machinery survived and was rewritten -- it governs how an
explicit user decision outranks an inferred withdrawal, which is not
cascade behavior.

Net 114 lines removed. The output envelope is unchanged, so no caller
moves.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…on queue

In a 27-finding review, 8 of the 16 distinct issues needed no judgment
at all -- contradictions the document resolved elsewhere, a missing
owner for behavior it already required, a callsite its own decision
implied. Every one was presented as a decision.

A finding is now an obligation when the question that resolves it is
already answered elsewhere in the document. Obligations group under the
implementation unit they affect and are confirmed together, so the user
makes one decision about settled work instead of N. The group renders in
full first -- a batch confirmation with nothing above it is a rubber
stamp, not a decision.

Concreteness is explicitly not authority: a fix introducing a new
user-visible state, limit, failure policy, retention rule, or
operational commitment stays a decision no matter how concrete it is.
That boundary is what keeps this from becoming a quiet auto-apply path.

Obligations count as proposed fixes in the exported totals rather than
getting their own bucket. A review whose findings are all obligations
must still report actionable items, or a caller gating on that sum would
hide the confirmation step and the user would never see work the review
found. The envelope shape is unchanged, so no caller moves.

The obligation block is specified once in the rendering floor and mapped
by each surface, matching how the floor already handles the Open
Questions entry: token policy and consequence-first phrasing, but no
recommendation field, because there is no decision to recommend.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…there is no choice

A real run on a sibling worktree's plan reported "No decisions requiring
judgment" in its own summary line and then surfaced 11 items for
confirmation. Judged against whether the author would realistically
decline, about four had a genuine fork. The rest were corrections the
document had already decided and not caught up with, including one the
plan itself cited as a known measured failure.

The contradiction is already inside the skill. The reviewer contract
says a finding whose only alternatives are strawmen is gated_auto rather
than manual, and defines manual as genuinely multiple valid approaches.
So gated_auto already means no genuine alternative exists -- and the
pipeline then routes it into a per-item Apply/Defer/Skip prompt, asking
the author to choose between options the rubric just asserted are not
there. Eleven such prompts per review teach acceptance without reading,
which is what destroys the confirmations that do matter.

U14 routes on decision entropy instead of the edit's blast radius:
settled problem with no alternative applies and is reported, settled
problem with real options asks which remedy rather than whether, and an
arguable problem stays a decision. It also fixes the misclassification
the same rubric forbids -- the observed P0 had two defensible
resolutions and still arrived as gated_auto because a concrete fix was
attached.

Sequenced after U9 so it does not move the baseline mid-measurement, and
because it may cut the load enough that U12 is unnecessary. U12's
cancel-or-build verdict now reads the re-measurement after U14.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Routing decides which findings need the user; rendering has to signal
which is which. Fixing only the first would ship a correct queue that
still reads ambiguously.

Today "Proposed fixes" and "Decisions" render identically -- both open
with Recommendation: <Apply | Defer | Skip>, same fields, same order,
distinguished only by a bucket header. Scanning the output, a reader
cannot separate "this is what I am doing" from "this is what I need
from you" without tracking which header they are under.

The observed run states both at once in a single summary line: "11
proposed fixes remain" beside "No decisions requiring judgment". That
is the ambiguity surfacing as a self-contradiction -- if nothing needs
judgment, nothing remains for the user.

U14 now gives each speech act its own grammar in the rendering floor,
which every surface already maps onto. A report is settled tense with
no recommendation field and no offered actions. A question carries its
options and names what differs between them; a question with one option
is a report wearing a question mark. The summary line counts changes
made and choices requested separately.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Authored by an independent Codex session working from the U8/U9 handoff,
which held the generated manifests and the distinct-pair truth outside
the session that wrote the matcher. Committed here unmodified.

The gate that mattered passed cleanly. Merge precision was 100% in every
host/model cell across 24 accepted attempts, with zero wrong merges and
zero merges against the pre-registered zero-duplicate control. Recall
rose from 66.7% to 100% on Opus and from 26.7% to 100% on Codex Luna.
The reasoning matcher is safe on a corpus its author never saw.

The load-reduction gate did not pass, and the record says so: no
host/model's mean reduction exceeded its own within-arm spread. The
cancellation rests on reliable new-arm behaviour and on reading the
residual failures as weaker-model instruction-following, not on a
demonstrated load win.

Procedurally it held the lines that would have silently invalidated the
result: the installed skill was never invoked, an independent scorer
produced the aggregates without returning the key, all five rejected
attempts are preserved, and N=3 expanded to 7 only on wide cells.

Two things this commit does not yet resolve. The plan edits reference
U14 nowhere, so the cancellation trail cites U9 where the current
sequencing routes that verdict through the post-U14 re-measurement --
same conclusion, wrong path. And Haiku and Sol showed unstable recall
and empty output, a portability defect no unit currently owns.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Two gaps the eval commit left open.

The cancellation of U12 and U7 was decided from a handoff written before
U14 existed, so its reasoning cites U9's load number where this plan
routes that verdict through the post-U14 re-measurement. The verdict is
kept and the trail corrected: U9 already bounded load at 5-8 choices on
the reliable cells and U14 only reduces it further, so no result U14
could produce would revive the case for clustering. Stage E no longer
holds U12; the sequence ends at guards and learnings.

U15 takes the finding nothing owned. Claude Haiku and Codex Sol showed
unstable recall across a 0-100% range and produced runs with zero
decision load -- output implausible enough to be instruction-following
failure rather than an empty review. This skill ships to five harnesses,
so that is a portability defect.

It is scoped as reliability, not safety, because merge precision held at
100% on those same unstable cells with zero wrong merges. That is the
fail-closed design working: a model that cannot judge whether one fix
resolves both findings declines to merge instead of guessing. Weaker
models under-perform; they do not corrupt. So shipping is not blocked on
U15, and no fix inside it may trade precision for recall.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…e is no choice

A real run reported "No decisions requiring judgment" in its own summary
line and then surfaced 11 items for confirmation. Judged against whether
the author would realistically decline, about four had a genuine fork.

The contradiction was already inside the skill. The reviewer contract
classifies manual precisely when genuinely different approaches exist,
and sends a finding to gated_auto when its only alternatives are
strawmen. So gated_auto already asserts no real alternative exists --
and routing then asked the author to choose between options the
classification had just ruled out. Eleven such prompts per review teach
acceptance without reading, which is what destroys the confirmations
that matter.

Routing now reads the two claims a finding carries. The anchor scores
the problem-claim, autofix_class the remedy-claim. Confirmed problem
with no alternative applies and is reported; well-evidenced problem with
no alternative joins one grouped confirmation shown in full first; a
real fork asks which remedy rather than whether to proceed. That last
question shape did not exist -- Apply meant accepting the framing and
the remedy together, with no way to say the problem is real but choose
differently. It lands as a sub-question so the four-option menu stays
the closed set it declares itself to be.

Two safeguards needed rewiring because gated_auto now applies. The
cross-model peer rule worked by demoting peer-only safe_auto to
gated_auto, which no longer protects anything, so it targets the route
directly: a peer-only finding never routes to Apply at any anchor or
class. And the emission rubric now states that a concrete suggested_fix
never outranks a real alternative, since misclassifying a fork as
gated_auto is what would put a scope change onto an unattended path.

The rendering floor gains the grammar to tell these apart: a report is
settled tense with no offered actions, a question carries its options
and names what differs, and a question with one option is a report
wearing a question mark. The summary line counts changes and choices
separately.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…Sol check

Recall parity on weak models is no longer a goal. Most users do not run
this on a Haiku-class model, and the one-fix test works because it is a
genuine judgment that fails closed. A formulation a weak model could
apply confidently would likely be more confidently wrong on every model
-- trading 100% precision, which the blind eval actually proved, for
recall on a tier that barely matters. Haiku's 69.8% against Opus's 100%
is real, and the right response is to run a capable model for a
judgment task.

One thing still deserves a look. Haiku degrades like a capability curve.
Codex Sol does not: its old arm returned 0% recall flat across all seven
runs, and its new arm produced runs with zero decision load. A flat zero
is the shape of something structurally wrong with the route -- a
truncated payload, a schema mismatch, an output contract silently unmet
-- not of a weaker reader doing worse. If that is a defect in how the
route is driven, it reaches everyone on Codex at any tier.

So the unit is now two runs and a paragraph: inspect Sol's raw returns
rather than its scores, distinguish a non-conforming return from a poor
judgment, fix it only if it is a route defect, and otherwise record the
floor. The answer may simply be a dispatch note -- this skill already
selects models per persona, so "synthesis runs on a capable tier" could
be one line instead of a workstream.

The floor gets written down either way. Deciding not to chase something
is only a decision if it is recorded; otherwise it is rediscovered.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…split

The sibling-worktree run has a second half. After surfacing 11 items
under a summary line reading "No decisions requiring judgment", the
agent was challenged and re-read its own findings: "9 of the 11 have no
defensible 'no' -- they're corrections, not choices. Two carry real
product judgment." My earlier hand-count guessed four; the agent that
wrote the findings says two.

That replaces an estimate with a measurement, and it sharpens why this
is a routing defect rather than a judgment one: nothing new was gathered
between surfacing the 11 and concluding 9 were not choices. The agent
could already tell. The routing simply never asked it to.

It also shows the same run failing in both directions. It asked about
nine items with no defensible refusal, then -- once pushed -- decided
the two that carried real product judgment, changing an acceptance
example and combining two reviewers' asks into something neither
requested. It disclosed both rather than burying them, which is honest,
but it made the calls. Current routing interrupts where there is nothing
to choose and commits where there is.

U14 now carries a concrete acceptance target instead of a rough one:
that document should surface two which-remedy questions and apply the
other nine as reported changes.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
A blind eval on U14 returned "does not hold" and named the cause: the new
routing table in 3.7 was added without removing the rules it replaced, so
the skill carried two contradictory sets of instructions and models
followed different ones on different runs.

Three conflicts, all mine:

Phase 4's apply step still said "apply only safe_auto at anchor 100" and
described anchor-75 safe_auto as demoted into the walk-through -- the old
3.7 behaviour, contradicting the new table where anchor-100 gated_auto
applies and anchor-75 joins a grouped confirmation.

Phase 4's routing step still split the remainder by autofix_class,
sending every actionable gated_auto and manual finding into the routing
question. Under the new table gated_auto at anchor 100 has already
applied and manual is the only decision class.

The rendering floor required an unmistakable recommendation on every
actionable finding while the new report grammar forbids one on a reported
change, with no scope separating them.

Phase 4 now routes by the route 3.7 assigned rather than re-deriving from
the class, and the field-order rule is scoped to findings the reader is
actually asked about, with the report grammar winning where they appear
to conflict.

The eval's other results stand: zero false applies in every U14 trial, so
the routing design is safe -- the instability came from the contradiction,
not the concept. False asks did drop where the new rule won (10 -> 6), and
the 2/9 target was never reached because the outcome depended on which
instruction the model happened to follow. Re-measurement is required
before U14 can be called verified; this commit does not claim that.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Re-measurement after the Phase 4 reconciliation showed the contradiction
was fixed -- no run reverted to per-item confirmations, false apply stayed
zero across seven trials -- but only 2 to 4 of 9 corrections reached the
change list. The rest were still being asked about.

The cause is in the routing table. Apply was gated on anchor 100, and a
real review returns almost everything at 75: the finding set under test is
1 finding at 100, 23 at 75, 7 at 50. So the Apply path was nearly empty by
construction and everything with no genuine alternative fell into the
grouped confirmation instead.

That gate was inherited from the old safe_auto rule, where it was right --
silent application of a mechanical edit deserves the strongest evidence.
It is wrong on the new axis. The anchor scores the problem-claim; Apply
turns on the remedy-claim. Anchor 75 means the reviewer double-checked and
the issue will be hit in practice, which is a settled problem, not a
doubtful one. Requiring 100 mixed the two axes back together -- the exact
confusion this step exists to undo.

Anchors 75 and 100 now route identically. What separates Apply from a
decision is whether an alternative exists, and nothing else. The grouped
confirmation keeps obligations and peer-only findings, which are there for
reasons unrelated to evidence strength.

The rationale is written beside the table so it does not get "corrected"
back. Not yet re-measured; this does not claim U14 holds.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Two failed evaluations traced to the same cause: a routing rule changed
in one place while seven downstream surfaces went on describing the old
behaviour. The skill carried two answers and models picked either one,
which is why results came back unstable rather than uniformly wrong.

This sweeps the rest. SKILL.md still told every run that gated_auto
findings are returned as text rather than applied -- it is the
always-loaded entry point, so that alone would have kept the
contradiction in every review. The findings schema still defined
gated_auto as "warrants user confirmation". The output template's worked
example showed two gated_auto findings sitting in P1/P2 awaiting a
walk-through, which the new taxonomy makes impossible; the example, its
summary line, and its Coverage counts are rebalanced so it is internally
consistent again. Bulk-preview scopes, the walk-through's intake and skip
conditions, and the persona tier rubric all follow.

Bucket names, exported counts, the Findings-equals-sum invariant, and the
Review complete signal are unchanged, so no caller moves.

One change has behavioural reach beyond renaming. Three safeguards used
"downgrade to gated_auto" as the mechanism for making the user look at
something -- the persona strawman rule, the oversized-suggested_fix gate,
and synthesis 3.6's strawman downgrade. That downgrade is now a no-op
because both tiers apply. All three retarget to manual, which is what the
rubric already implies: a finding with a real alternative is a decision.
Findings that previously landed in the confirmation bucket via those
safeguards now reach the decision surface, which is the intended
direction.

Two documentation gaps were found and deliberately not invented around:
the ordering of the grouped confirmation relative to the routing
question, and how Coverage attributes a peer-only finding. Both predate
this change; both are now more visible.

Not yet re-measured. This does not claim U14 holds.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…IDs at emission

Two changes, both aimed at the failure the third evaluation exposed
rather than at plumbing.

Uncertainty now has its own route. Measurement showed a clean pendulum:
gating Apply at anchor 100 gave zero false applies but reported only 2-4
of 9 corrections; opening it to anchor 75 reported 6-9 of 9 but applied a
real product fork in six of seven runs. Each trial's forks-asked plus
false-applies came to two, which is the whole story -- routing works, and
the model cannot reliably tell which findings carry a genuine choice.

A binary apply-or-ask forces a guess under that uncertainty, and both
wrong answers are expensive: applying decides for the author, asking
spends a prompt on a foregone conclusion. The grouped confirmation costs
one question for the batch, so a finding that lands there wrongly costs a
glance. Uncertainty now goes there. Same fail-closed reasoning the merge
test already uses, applied to the routing axis.

Identifiers now get their handle from the reviewer that raised them. The
floor already banned bare identifiers, but assigned the lookup to
rendering -- which runs after a long dispatch has filled context with
reviewer returns, and was being asked to re-derive fifteen handles from a
document it may no longer hold. So bare IDs survived into output with the
rule in force: "U1 claims R12, R13, R18" is four anchors in a sentence
capped at two.

The reviewer has the document open and knows what R12 means at no cost.
It now writes the handle into the finding, and rendering renders what it
is given. The schema's section description was asking for a bare section
name and now asks for the same thing the template does.

This is not only legibility. Synthesis decides whether two findings
describe one problem by comparing what they say, so two reviewers who
both write "R15" instead of naming the rule may not read as describing
one issue -- opaque identifiers plausibly cost merge recall as well as
comprehension.

Not yet re-measured.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…onfirmation

Four evaluation rounds said U14 does not hold, and the reason was the
design rather than the tuning. Letting gated_auto apply unattended
reported far more corrections -- 7 to 9 of 9 against 2 to 4 -- but
applied a genuine product fork in most runs, because the model cannot
reliably tell which findings carry a real choice. Offering it a route for
its own uncertainty changed nothing: it never used that route, since it
does not experience the uncertainty as uncertainty. It decides, and is
sometimes wrong.

So the two problems are separated instead of being solved by one rule.
Only safe_auto at anchor 100 applies unattended. Everything else with a
concrete fix goes to the grouped confirmation -- one question over the
batch, rendered in full first. Volume is fixed by batching; authority is
fixed by keeping it attended. A wrong classification now costs the reader
a glance instead of an unrequested change to their document.

The dead uncertainty route is deleted rather than qualified. It also
contradicted the misclassification guard on the same close call, which is
the shape that produced unstable results in the first place. The guard
survives, reframed: with nothing auto-applying, the residual risk is a
real fork buried in a batch the reader skims, so a close call between
gated_auto and manual goes to manual.

The remaining "do not prompt where there is nothing to choose" rule was
literally false once foreseeable findings started landing in a
confirmation. Rewritten to say what it now means: do not spend a separate
question on each of them, because eleven foregone questions teach
acceptance without reading. Batching is the remedy, not silence.

Downstream surfaces swept for the fourth time: the persona tier rubric,
the schema description, the envelope, the Tier column, and the worked
example, whose applied-fixes list had two gated_auto findings in it. The
example's counts were rebalanced and verified -- every persona row sums
to its Findings value and the totals match the summary line.

Not measured. This is a retreat to a posture whose safety property held
across seven trials in an earlier round, with the volume win relocated to
batching.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…tually showed

The plan described intentions that no longer match what shipped, which
would give anyone reading it cold a wrong picture.

U14 is marked shipped-narrowed with its target unmet, and carries what
each round established: rounds 1-2 failed on reconciliation rather than
design, and their signature was instability rather than uniform failure;
round 3 exposed a clean pendulum between safety and usefulness; round 4
showed that routing a model's own uncertainty somewhere safe does nothing
because it does not experience the uncertainty. Two changes -- identifier
glossing and the report-versus-question grammar -- were never measured,
because a frozen finding set freezes reviewer behaviour along with
reviewer variance and cannot see an emission-layer change at all.

U12's cancellation is flagged as reopenable. It read a decision-load
number off the generated corpus, and that corpus returned 100% merge
recall where the one organic pair returned 20-36%. If duplicates survive
at that rate on real documents, the number the cancellation rested on
does not describe real reviews.

U15's rescope is flagged the same way: its weak-cell numbers came from
the same inputs, so they may describe the corpus rather than the model.
The Codex Sol check survives -- a flat zero across seven runs is not a
shape easy inputs explain.

And the corpus finding is recorded in its own right, because it outlives
this plan. A corpus you author to test a mechanism tends to contain that
mechanism's happy path. The blind-manifest discipline here guarded
against a biased scorer and did nothing about easy inputs, so four
tuning rounds ran against material that could not fail us.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…ect what they disproved

Two new learnings, both about evaluation rather than about doc review,
and both general enough to bite any skill measured the same way.

The first is that a corpus you author to test a mechanism tends to
contain that mechanism's happy path. The duplicate matcher scored 100%
recall on generated plans and 26% on one pair taken from a real review,
with no change to the matcher in between. Blinding the scorer was done
carefully and protected against the wrong failure: withholding the
manifest stops a biased grader, and says nothing about whether the
inputs are representative. Sampling real documents instead has its own
version of the same bias -- the plans sampled had already been reviewed,
so their defects were already gone, and the duplication rate measured
off them was a floor rather than the rate.

The second is that a frozen finding set cannot see a change to how
findings are produced. Freezing removes reviewer variance by removing
reviewer execution, which are the same act, so the harness is
structurally blind upstream of the freeze point. It reports normally
while doing this: an unmeasurable change and an ineffective change
produce the same null result.

Three existing docs described behaviour that no longer exists.
`confidence-anchored-scoring` claimed the routing menu absorbs volume
and that dismissing a finding is cheap -- that assumption is what
produced a 34-finding review, since the cost is not per-keystroke but
the reader holding thirty open questions at once. The `>= 50` threshold
survives; the reasoning under it is corrected, and its port criteria now
require batching. `ce-doc-review-calibration-patterns` documented the
deleted 3.5c chain-linking machinery as though implementable; it now
says the mechanism is gone and why, keeping the peer-versus-nested
reasoning for anyone who revisits the problem. `paired-old-vs-new-
injection-skill-evals` cited a fixture answer-key leak that has since
been closed by moving the keys into sidecars, and gains the two new
leak classes.

The rendering-floor doc gains the behaviour-rule case. Its own thesis --
a contract duplicated across surfaces is a latent divergence -- turned
out to apply harder to routing than to presentation, which drifted three
times in a row across eight restatements with no single source. A stale
presentation rule produces ugly output; a stale routing rule produces
wrong action, and the always-loaded file is the most dangerous copy
because it is the least likely to be reread.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…r-finding decisions

The user-facing page described three-tier routing with gated_auto and
manual both landing as "user decision," a four-option section keyed on
findings landing in those tiers, and a worked example whose summary line
called three proposed fixes "3 decisions." None of that matches what
ships: routing now reads confidence and fix class together, only a
full-confidence mechanical correction applies unattended, anything that
touches meaning arrives batched, and a question is reserved for a real
fork between remedies.

The example's summary line was the sharpest case, since the rendering
floor exists partly to prevent exactly it -- counting changes made and
choices requested as one number, and describing items as awaiting the
reader when none is.

This is the drift class the rendering-floor doc now documents, and it
was found by checking rather than by anything failing, which is the
point: the page is not covered by a parity test, so nothing would have
caught it.

U11 is marked complete with what it actually delivered, including that
its step 3 is dropped -- it was conditional on U2's verdict and U2 was
never run.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
@cursor

cursor Bot commented Aug 14, 2026

Copy link
Copy Markdown

PR Summary

Cursor Bugbot is generating a summary for commit a884c90. Configure here.

Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
Comment thread skills/ce-doc-review/references/walkthrough.md
…finish the 3.3 sweep

Review found three defects, all the same class this PR set out to fix: a
rule changed in one place and its restatements left describing the old
behavior.

The serious one is that the grouped confirmation had no execution path.
Synthesis routes most meaning-touching fixes to it and five files
describe it, but no surface ever fired it -- the walk-through's entry
says those findings have no per-finding entry, and the routing question
only covers what remains. So a reader who picked "review each finding
one by one" saw the batch presented as awaiting one answer and then
never got asked, leaving every obligation, gated_auto, and anchor-75
safe_auto unapplied. The central mechanism of this change did nothing in
interactive mode.

It now fires as its own step between the applied changes and the routing
question: batch rendered in full first, then one question that applies
it, with an escape hatch to step through and an option to apply none.
The routing question is scoped to the decision surface, and the skip
rule, bulk-preview scope, and Phase 4 route list follow that split.

The second is that 3.4 still matched cross-model peers with the
section-plus-title fingerprint and its twin evidence-overlap exception,
both deleted when 3.3 became a reasoning test. Merging is now decided
once in 3.3 and read by 3.4, and the same stale vocabulary is cleared
from the cross-model review contract, its eval scenarios, and the
whole-doc persona. Round-to-round memory keeps its own fingerprint --
that one is live, and now says why it does not share 3.3's.

The third is that the walk-through told the renderer to look identifier
handles up in the document while the floor says the handle arrives with
the finding and rendering must not rebuild it. The walk-through
described the floor's defect path as the normal one.

One test moved with the code. It pinned the literal string "cannot use
the twin fingerprint exception" -- prose describing a mechanism this PR
had already deleted, which survived only because the stale sentence
survived. It now pins the rule that outlives the mechanism: an
unverified peer cannot trigger anchor promotion.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
…e decision surface

Batching the confirmation moved the volume without moving the
comprehension. The batch still rendered as N independent entries, which
costs a reader exactly what N questions cost -- one answer at the bottom
does not help if understanding it still takes eleven reads. The floor now
carries a "Presenting a batch" rule: lead with what the batch does as a
whole, head each group with the consequence its members share, keep every
member visible, and do not manufacture a theme where none exists. The
axis that helps is the change itself -- one root cause, one kind of edit,
one section. Severity and reviewer sort a list without helping anyone
decide.

The rule states the goal and leaves the judgment to the agent, which is
the layer that can actually do it: synthesis holds every finding at once
and the reader holds none of them. Finding the two or three real themes
in a batch of eleven is what this step is for.

Review also caught that the confirmation pointed at a template section
that did not exist. The interactive template filed grouped-confirmation
members into the same P0-P3 tables as the decision surface, so "apply all
of them" could have swept a genuine fork into a batch answer -- the exact
failure this work set out to prevent -- or missed batch members buried in
those tables. The non-interactive envelope already separated the two; the
interactive template never did. It now renders Proposed fixes as its own
section beside Implementation obligations, those two sections are the
batch and nothing else is, and the P-level sections carry decisions only.
A gated_auto row appearing in a P-level table now means routing went
wrong rather than rendering being loose.

The floor owns the rule; the walk-through, the template, and the envelope
reference it instead of restating it.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a884c90840

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
…ut units

Obligations route to the implementation unit they affect, which assumes
every reviewed document has units. Requirements-shaped documents do not,
and an obligation is defined by the document already entailing the fix --
an entailed contradiction or an unowned requirement arises there just as
readily. Such a finding had nowhere to go, so it would be dropped or filed
under a unit that does not exist.

They now route to the part of the document they affect: the implementation
unit where the document has units, the owning section where it does not.
The section is titled "Entailed corrections" in that case, since
"Implementation obligations" names something a requirements document does
not contain, and the walk-through refers to the obligations section by
role so it holds under either title.

The floor owns the destination rule; synthesis, the template, and the
envelope follow it.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 216b6850eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md
Review found four defects in the grouped-confirmation work itself.

Stepping through the batch individually dropped the decision surface.
Option B enters the per-finding loop, and that loop's terminal contract
emits the completion report and moves to Phase 5 -- so a reader who chose
to review the batch one at a time ended the run with their decisions
never asked. The same disappearance the confirmation step was added to
fix, one layer down. It now returns to the routing question when the
batch is exhausted.

The which-remedy question could not fire in the case the routing table
promised it for. The table said a manual finding means real alternatives
exist, so ask which remedy -- but the reviewer contract commits
suggested_fix to one recommendation and forbids alternative menus
outright, so an ordinary manual finding arrives with one fix or none and
has no menu to offer. The question is not dead, it was mis-scoped: 3.5
turns a contradiction between personas into one finding holding both
perspectives, and that is a genuine fork. It now fires there, the table
no longer promises it everywhere, and the note says plainly not to invent
a second option to make the fork appear.

The non-interactive envelope claimed confirmation it never obtained. That
mode asks nothing and returns the batch unapplied, yet the envelope told
the caller the obligations were "confirmed as a group" and the proposed
fixes "confirmed with" them -- inviting a caller to treat unapproved
changes as accepted. Both headers now say awaiting.

And the obligations fix that shipped an hour ago was half applied: only
the first group placeholder became "unit or section name", the second
still said "unit name", and the retitle instruction sat inside the fence
that is itself the emitted output. A requirements review could still
invent a unit heading, and a bracketed instruction could leak into caller-
parsed text. The instruction now lives in prose above the fence, where
the rest of the template's guidance already is.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b72a35df7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
…g 50 actionable

Three more from review, and the first one is the interesting one.

The batch-only pass still had an exit that ended the run. The previous
fix covered normal exhaustion of the loop; "Auto-resolve with best
judgment on the rest" leaves it through bulk-preview, which emits the
completion report, so the decision surface disappeared again by a
different door. Patching that door would have invited a third. The rule
is now stated once at the loop: in batch context the walk-through is a
subroutine with exactly one exit, back to the routing question, and it
never emits the completion report -- whatever the exit is called, and
including ones added later.

Cross-section contradictions had nowhere to land. 3.3 applies its one-fix
test across sections and hands opposing recommendations to 3.5 rather
than merging them, but 3.5 only combined disagreements on the same
section. A cut-versus-keep pair split across two sections therefore
survived as two independent decisions, and the best-judgment route would
execute both -- applying mutually incompatible fixes. 3.5 now resolves a
disagreement wherever it was noticed, because the section is where a
contradiction is spotted and the decision is what makes it one.

Anchor 50 was called the actionable floor two lines below a bullet saying
it enters nothing. It is the surfacing floor; 75 is the actionable one.

While correcting that paragraph: it still argued that surfacing low is
safe because the routing menu makes dismissal cheap. That is the exact
reasoning documented as the cause of the thirty-four finding review, and
it had survived in the skill while the learning that disproves it sat in
docs. The real argument is narrower and still holds -- a 50 costs one
line in an observational subsection and never becomes a question -- and
it holds only because 50 stays out of the pipeline. The same stale
premise is corrected in confidence-anchored-scoring's port criteria,
which asserted it one section below its own correction.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Comment thread skills/ce-doc-review/references/bulk-preview.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 31ac8fbdf6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
tmchow added 2 commits August 13, 2026 19:22
bulk-preview said two contradictory things about the batch, one from each
of my last two commits. The header claimed pending always means the
decision surface and the batch is never in scope here; the auto-resolve
path then said that in batch context "the rest" is the rest of the batch.
A model following the header previews nothing, applies nothing, and hands
back a batch the reader was just told was about to land.

The header was written when the preview served one pass. It now serves
two, so scope is stated per pass: serving routing, pending is the
decision surface and the batch is already answered; serving the grouped
confirmation's batch pass, pending is the rest of that batch and the
decision surface has not been routed yet. Site 3 reads against whichever
pass is running, since it is the site that can appear in both.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
… and always report

Two from review, both cases of a rule stated more broadly than its reason
supports.

The cross-model peer safeguard sent every peer-only finding to the
grouped confirmation, at any class. Its reason is narrower: a peer cannot
authorize an edit the reader never sees. Withholding apply authority is
that reason; demoting a fork into the batch is not. A peer-only manual
finding is still a genuine choice, and may carry no suggested_fix at all,
so the batch would either sweep a scope decision under "apply all" or
attempt an edit with no payload. It now diverts to the batch only where
the table would have applied it, and a peer-only manual finding stays a
decision. Four downstream restatements said the unconditional version and
were reconciled with it.

A batch-only run emitted no completion report. With a non-empty batch and
an empty decision surface, routing is skipped -- and skipping routing
jumped to the Phase 5 terminal question, past the report. The reader
answered a confirmation, findings were applied or skipped, and nothing
recorded which. Skipping routing now never skips the report, and the
collapsed degenerate form is explicitly reserved for the run where
nothing was ever put to the reader.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
Comment thread skills/ce-doc-review/references/walkthrough.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6ea10dee01

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
Comment thread skills/ce-doc-review/references/walkthrough.md
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
…ions bouncing

Five from review, every one a consequence of the previous two commits
that I did not carry through.

The completion-report instruction landed on Phase 4's shared remaining-
findings path with no mode gate. An empty decision surface is the ordinary
non-interactive case, so a chained run could have printed the interactive
report beside or instead of the envelope its caller parses, or said
"complete" before the confirmation was answered. It is now interactive-
only and explicitly after the confirmation settles.

The terminal-path list still named the skipped-routing case as the
one-line collapse, and that list is read before the degenerate-case
section, so a batch-only run could still collapse and drop the record the
previous commit added.

A no-fix obligation had no stable route. The obligation rule reclassified
it to gated_auto; the routing table requires suggested_fix for that class
and demotes it back to manual. It bounced, and either lost the obligation
route or joined an apply-all batch with no payload. An obligation now
takes gated_auto only when a fix was written; without one it stays a
decision, because entailing a correction is not the same as having
written it and the reader supplies the missing text. The obligation
rendering rules in the template and the floor say so too -- a line whose
form is consequence-plus-change cannot render a finding with no change.

The presentation before the routing question was the pre-confirmation
one. Answering the batch ends a turn, so that render happens after the
batch is settled, yet it still listed every member under Proposed fixes
and summarized them as awaiting one confirmation. It now renders current
state: applied members in the applied list, skipped ones reported skipped,
neither awaiting anything.

And cross-model-eval still told the behavioral eval to assert that a
peer-only manual finding caps at gated_auto -- rewarding the regression
the last commit fixed. I swept four restatements of that rule and missed
the one that grades it.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9a166392d7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
Comment thread skills/ce-doc-review/references/synthesis-and-presentation.md Outdated
tmchow added 3 commits August 13, 2026 19:39
…or reclaiming no-fix findings

Two contradictions I introduced when rewriting the obligation rule last
commit.

The reclassification was widened from "manual becomes gated_auto" to
"anything carrying a suggested_fix becomes gated_auto", which swept up
safe_auto obligations -- demoting a mechanical correction that the same
step, two paragraphs below, says still applies unattended at anchor 100.
It now only ever moves manual up, and never touches safe_auto.

And the no-fix exception was undone immediately after by the routing
paragraph, which still sent every obligation to the grouped list. It now
routes only the obligations that survived both rules, and says plainly
that a no-fix one is not among them because it stayed a decision.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
Every round of this PR's review targeted text an earlier round added, and
every round I qualified it instead of deleting it. AGENTS.md says the
opposite: prefer deleting the addition, and a section that has absorbed
three rounds is itself the defect. The grouped confirmation had absorbed
four, the obligation rule three, the anchor-50 rationale two.

Collapsed, with each shortened rule re-checked against every path it
served:

- The grouped-confirmation section drops from ~450 words to ~250. Gone are
  "the failure this step exists to prevent", the paragraph justifying why
  A is recommended, the history of how the decision surface went missing,
  and a cross-reference to another section's invariant. The subroutine
  rule keeps "a bug whatever its name", which forecloses the next exit;
  that is a constraint, not rationale.
- The obligation rule states three cases in one paragraph instead of
  three paragraphs arguing with each other. Collapsing it exposed that
  safe_auto was then covered twice, so the standing rule below keeps it
  and the forward reference is gone.
- The anchor-50 paragraph loses its meta-commentary about the claim it
  replaced. The correction stays, because the disproven reasoning is a
  known tendency; the essay about it does not.
- The peer safeguard, the bulk-preview scope header, the post-confirmation
  render, the skip-routing rule, and the degenerate-report case each lose
  a trailing sentence that restated the directive above it.

Net effect: about 480 words removed from the accreted sections while
synthesis lands +71 against main rather than +323. Nine invariants
verified still named after the collapse, since a condensed rule that
drops a path is a new defect rather than a simplification.

The rendering floor's growth is deliberate and stays -- "Presenting a
batch" is new capability, not accretion.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
…the 75/safe_auto row

Two tightenings of existing text, not new rules.

The peer cap kept a peer-only manual finding out of safe_auto but still
promoted it to gated_auto, and 3.7 batches gated_auto -- so the finding
reached Apply-all through promotion after the 3.7 safeguard had been
fixed to keep it off that surface. The cap now declines to promote a
peer-only manual finding at all.

And the 75/safe_auto row sent findings to the grouped confirmation with
no suggested_fix guard, while all three neighbouring rows demote on a
missing fix. suggested_fix is optional in the schema, so that row could
put a finding with no edit into a batch whose Apply-all expects one. It
now carries the same clause as its siblings.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 372e44af75

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
…rning to routing

The subroutine rule created a silent data-loss path. The walk-through
accumulates Apply decisions in memory and dispatches them in a single
pass at its terminal path -- which is exactly the path the batch exit
skips. So a reader who stepped through the batch and approved fixes
returned to routing with those edits still staged, and choosing report-
only or append-to-Open-Questions there ended the run without ever
dispatching them. Approved fixes silently did not land.

Flushing is now part of the exit rather than of the report: run the
accumulated Apply set, then return. bulk-preview says the same for the
auto-resolve path, whose findings join that set.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 18643ea. Configure here.

Comment thread skills/ce-doc-review/references/walkthrough.md Outdated
Last commit made the batch exit write its accumulated Apply set but never
empty it, so the decision pass's terminal dispatch would write the same
members a second time. Flushing and clearing are one operation; the exit
now does both, which leaves the terminal dispatch correct as written
rather than needing a second rule about what to skip.

The bulk-preview auto-resolve bullet had accumulated the round-5 and
round-11 statements of the same routing rule side by side. Merged into
one, thirty words shorter.

Claude-Session: https://claude.ai/code/session_018wUcBhPSqwTAMWUSL9ZJvw
@tmchow
tmchow merged commit 421a337 into main Aug 14, 2026
5 checks passed
@github-actions github-actions Bot mentioned this pull request Aug 14, 2026
ethras added a commit to ethras/compound-engineering-orca that referenced this pull request Aug 16, 2026
* fix(ce-commit-push-pr): root PR stacks on the parent PR the user named (EveryInc#1365)

* fix(ce-babysit-pr): decode gh output as UTF-8 on Windows (EveryInc#1368)

* fix(ce-prototype): cover decisions settled by seeing, not just driving (EveryInc#1369)

* perf(tests): cut suite wall time by splitting the largest test file (EveryInc#1370)

* fix(tests): stop the cross-model routes test reading the working tree (EveryInc#1371)

* fix(ce-doc-review): ask only where a real choice exists, batch the rest (EveryInc#1373)

* chore(orca): re-pin upstream provenance baseline to 421a337

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(ce-doc-review): align Orca ownership wording with Apply routing

Co-authored-by: Cursor <cursoragent@cursor.com>

* feat(ce-prototype): add a seeing-mode craft floor and durable storage (EveryInc#1374)

* fix(ce-pov): stop the panel guessing the cross-model host argument (EveryInc#1375)

* chore(cross-model): pin the Grok peer to 4.6 (EveryInc#1376)

* docs(skills): rewrite user skill pages for accuracy and clearer use (EveryInc#1377)

* fix(commit): append known plan unit ids to commit subjects (EveryInc#1379)

* fix(ce-work): stop sandboxed workers committing in linked worktrees (EveryInc#1382)

* fix(ce-doc-review): edit HTML plans in native format (EveryInc#1381)

* fix(ce-code-review): cover adversarial after quota or auth no-review (EveryInc#1380)

* fix(skills): correct a rejected dispatch instead of spending the fallback (EveryInc#1383)

* fix(ce-compound): find Claude sessions started outside the repo root (EveryInc#1378)

* ci(windows-native): retry peer-job-runner smoke on ctypes flake (EveryInc#1384)

* fix(ce-debug): stop asking at the handoff, stop shipping unoffered work (EveryInc#1385)

* docs(solutions): record why skill gates state conditions, not git commands (EveryInc#1386)

* fix(skills): drop the residual-findings record file for real sinks (EveryInc#1387)

* fix(ce-doc-review): run the cross-model pass when CROSS_MODEL_PEERS is unset (EveryInc#1389)

* fix(ce-proof): sync with current Proof v3 contract (EveryInc#1390)

* fix(skill-authoring): make goal-first the default when authoring and reviewing skills (EveryInc#1391)

* fix(cross-model): let reviews run on Fable and pin model/effort from CE config (EveryInc#1392)

* docs(cross-model): point superseded peer benchmarks at the luna/xhigh decision (EveryInc#1393)

* fix(cross-model): discover the Codex.app-bundled codex CLI and name the peer-CLI requirement (EveryInc#1395)

* feat(cross-model): add cross_model_review_mode checkout egress gate (EveryInc#1396)

* fix(ce-compound-refresh): compare knowledge-track learnings against guidance they name (EveryInc#1399)

* docs(solutions): capture the named-guidance contradiction-check learning (EveryInc#1400)

* fix(ce-compound): prefer the repo's own frontmatter vocabulary over the Rails-era enums (EveryInc#1394)

* fix(ce-work): stop asking about branches before starting work (EveryInc#1397)

* fix(review): answer covered cases on skill prose with the condition, not a patch (EveryInc#1401)

* fix(scratch): fall back to $TMPDIR when /tmp cannot host the scratch root (EveryInc#1398)

* feat(ce-skill-work): repo-local skill for authoring, editing, reviewing, and responding to review on skills (EveryInc#1402)

* fix(ce-pov): reject non-final peer positions instead of folding them in (EveryInc#1403)

* feat(manifest): add Agent Plugins v1.0.0 manifest support (EveryInc#1345)

* chore: release main (EveryInc#1354)

* fix(ce-work): run cross-model verification on warm checkouts (EveryInc#1404)

* fix(orca): reconcile upstream skill contracts

* fix(orca): preserve additive Codex session roots

---------

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant