Skip to content

Fix seam init argument forwarding - #119

Merged
justin808 merged 17 commits into
mainfrom
codex/112-seam-init-forwarding
Jul 11, 2026
Merged

Fix seam init argument forwarding#119
justin808 merged 17 commits into
mainfrom
codex/112-seam-init-forwarding

Conversation

@justin808

@justin808 justin808 commented Jul 11, 2026

Copy link
Copy Markdown
Member

Closes #112

Summary

  • generate seam-init wrappers that preserve forwarded arguments across direct, nested, env, exec, npm, and supported shell prefixes
  • require a safe explicit $0 placeholder for recognized shell -c suffixes, including bounded aliases and versioned Bash names
  • reject active outer substitutions and unsafe positional-state use in shell and npm option values
  • recursively parse bare/absolute env chains; opaque env -S commands require explicit forwarding after the split operand
  • normalize npm lifecycle forwarding with npm 11.6 metadata-aware parsing while preserving verified terminator behavior
  • validate every requested init command before writing so failures cannot leave a partial managed seam

Validation

  • full seam helper: 258 runs, 2,250 assertions, 0 failures/errors/skips
  • new recursive-env, bounded-shell, and npm-marker matrix: 14 runs, 305 assertions
  • prior accumulated security/runtime matrix: 18 runs, 144 assertions
  • hostile positional-state runtime matrix: 8 runs, 84 assertions
  • installer/status/upgrade suite: 21 cases
  • fixture seam and agent-stack suites: passed
  • bin/validate: passed
  • RuboCop 1.87.0: 44 files, no offenses
  • git diff --check: passed
  • hosted validate, Claude review, and CodeRabbit status: passed at exact head

Final-head QA Evidence

  • QA lane: m1 qa-seam-init, independent report-only worker in a clean detached worktree.
  • Tested at exact head 215ea2fbbbed5aa47c6b74aecd7955647c687398 on 2026-07-11T13:29:07Z.
  • Runtime coverage includes recursive env and nested npm; opaque split-string suffix forwarding; option-bearing exec; bounded shell aliases; separate and attached npm forwarding-marker rejection; prior shell/env/npm/transactional security cases.
  • Findings: none.
  • Release blocking: clear.
  • Confidence: high.

Review dispositions

  • Fixed shell placeholder, substitution, quote-state, ANSI-C, positional-state, and command-boundary bypasses.
  • Fixed recursive and absolute env split-string handling, option-bearing exec prefixes, bounded shell aliases, npm option domains, and partial writes.
  • Required opaque split strings to carry explicit suffix forwarding; earlier env option values cannot impersonate it.
  • Rejected complete or active forwarding expansions as scalar npm option values while preserving escaped/single-quoted literals.
  • Confirmed npm 11.6 --enjoy-by from runtime, installed definitions, and generated docs.
  • Resolved npm run -- <script> as a runtime-proven false positive; an extra separator would leak a literal --.
  • Deferred cleanup-only parser extraction, schema drift, naming, dead-data, and unreachable fallback suggestions with rationale.
  • Workflow follow-up Follow-up: Automatically fall back when a worker model route cannot be bound #116 tracks automatic Sol fallback after the Terra compatibility incident.

@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

--init now validates shell argument forwarding more rigorously, decodes ANSI-C shell strings, and normalizes npm lifecycle arguments using structured option metadata. Tests cover quoting, injection safety, separator placement, option parsing, workspace behavior, and vendored npm metadata.

Changes

Seam initialization command rewriting

Layer / File(s) Summary
Shell forwarding validation
bin/agent-workflow-seam-doctor, bin/agent-workflow-seam-doctor-test.rb
Shell -c commands, comments, assignments, quoting, ANSI-C strings, and outer $@/${@} expansion are validated before wrapper forwarding is generated.
npm command normalization
bin/agent-workflow-seam-doctor, bin/agent-workflow-seam-doctor-test.rb
Npm options and lifecycle operands are parsed using option metadata, required values are consumed, and the authoritative -- separator is inserted or repositioned while preserving script and workspace arguments.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant InitCLI
  participant init_command_line
  participant shell_c_forwarding_error
  participant npm_argument_partition
  participant WrapperCommand
  InitCLI->>init_command_line: process --init command
  init_command_line->>shell_c_forwarding_error: validate shell forwarding
  shell_c_forwarding_error-->>init_command_line: forwarding result
  init_command_line->>npm_argument_partition: normalize npm arguments
  npm_argument_partition-->>init_command_line: script operand and separator placement
  init_command_line->>WrapperCommand: generate rewritten wrapper command
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR addresses #112 by fixing npm separator placement, adding shell-quote-aware forwarding detection, and covering the regressions with tests.
Out of Scope Changes check ✅ Passed The added shell and npm parsing helpers appear scoped to the forwarding fix and its test coverage, with no unrelated changes evident.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: fixing seam-init argument forwarding.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/112-seam-init-forwarding

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6846aa66e3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/agent-workflow-seam-doctor Outdated
Comment thread bin/agent-workflow-seam-doctor Outdated
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review

Overview: This PR reworks --init argument-forwarding detection in bin/agent-workflow-seam-doctor. It replaces a single regex (ARGUMENT_FORWARDING) with a proper hand-rolled tokenizer (shell_word_spans) and several purpose-built checks: detecting assignment-only commands, ambiguous $@ inside shell comments, unsafe bash/sh/zsh -c forwarding shapes (including the genuinely dangerous case of active $@ expansion inside a double-quoted -c command string, which is a real command-injection footgun since the outer shell would splice caller-controlled argument text into the inner shell's command string), and correct npm option-boundary detection so -- is inserted in the right place relative to npm's own CLI options vs. the script's own arguments.

Correctness: I traced through the trickiest paths by hand (single- vs double-quoted bash -c payloads, escaped \$@, npm -w2/--omit=dev attached values, required-value npm options like --workspace, repositioning an existing -- separator) against the new test cases, and the logic checks out — the double-quote-injection detection in particular is a legitimate and well-reasoned security improvement over the old single regex.

Findings (posted inline, all minor/moderate — none block correctness):

  1. Unreachable branch in npm_argument_boundary_index (line 596): the else raise InitError ... "has ambiguous arity" can never execute because npm_post_script_option_metadata never returns a value that reaches it. Harmless but should be removed or turned into an explicit "unreachable" assertion.
  2. Duplicated prefix-stripping logic between shell_c_forwarding_error and init_npm_command_info (both re-implement the same exec/env-assignment/env-options stripping sequence). Worth extracting into a shared helper so future fixes don't need to be applied twice.
  3. Duplicated hand-rolled shell tokenizers: shell_comment_index, compound_shell_command?, shell_word_spans, and the escape logic in outer_argument_forwarding? are four independent char-by-char quote/escape state machines that currently agree but aren't derived from one shared implementation. Given this parsing gates a real security check (the double-quote injection detection), consolidating into one shared low-level walker would reduce the risk of the implementations silently drifting apart over time.

Test coverage: Very thorough — the new test file additions cover single/double-quote -c forwarding, escaped $@, missing $0 placeholders, comment ambiguity, assignment-only commands, npm separator repositioning/preservation, npm option arity (including a test that runs with PATH=/nonexistent to prove the vendored npm metadata doesn't require an npm binary). No coverage gaps stood out as worth blocking on.

Security: No new externally-controlled input surface — --validate-command/--test-command are maintainer-supplied CLI flags for a local --init tool, not attacker-controlled PR content. The double-quote -c injection detection is a genuine hardening of existing behavior.

Performance: All new parsing is a single linear pass per command string over short, one-time CLI-flag-sized inputs — no concerns.

@justin808

Copy link
Copy Markdown
Member Author

Address-review summary

Scan scope: full PR history; no previous address-review cutoff.

Mattered

  • Fixed the unquoted/ANSI-C/mixed-quote shell -c placeholder bypasses in 0c2aae5 with runtime regressions for first-argument preservation and inert forwarded text.

Optional

  • Replaced the unreachable npm metadata branch with an explicit invariant assertion.
  • Extracted shared command-prefix stripping for shell and npm detection.
  • Shell-walker consolidation was auto-deferred with rationale in-thread because it expands the final corrective scope.

Skipped

  • None.

Next default scan starts after this comment. Say check all reviews to rescan the full PR.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c2aae5db9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/agent-workflow-seam-doctor Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c098b9bbcd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/agent-workflow-seam-doctor Outdated
Comment thread bin/agent-workflow-seam-doctor Outdated
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review summary

What this PR does: Hardens bin/agent-workflow-seam-doctor's --init argument-forwarding logic: replaces a naive regex for detecting "$@"/${@} forwarding with a proper quote/escape-aware scanner (outer_argument_forwarding?), adds detection for unsafe bash/sh/zsh -c '...' forwarding shapes that silently drop the first forwarded argument unless an explicit $0 placeholder is present, rejects assignment-only commands and comment-only forwarding text, and replaces the old npm "is everything before run a flag" heuristic with real npm 11.6.0-derived option-arity metadata so -- gets inserted in the correct place relative to npm-owned options vs. script arguments.

Strengths

  • The assignment_only_command? guard is a genuine security fix: previously a command like LABEL='$@' would get "$@" appended directly (LABEL='$@' "$@"), which — since "$@" expands positionally — could cause a caller-supplied argument to be executed as the command. Rejecting it outright is the right call.
  • The bash -c / sh -c / zsh -c fail-closed check directly targets a real, easy-to-miss bug class (forwarded args silently shifting into $0 when no placeholder is supplied), and is validated with a solid matrix of quoting scenarios (single/double/ANSI-C/mixed quoting, clustered options like -clo, wrapped via env, etc.).
  • Test coverage is extensive and specific — the new tests read as a genuine adversarial suite rather than happy-path coverage.

Issues found

  • Inline comment posted: shell_c_forwarding_error (bin/agent-workflow-seam-doctor:404) only checks the shell basename against bash/sh/zsh. Other shells with the identical -c positional-shift hazard (dash, ksh/mksh, busybox sh invoked as busybox, etc.) aren't covered unless symlinked to sh, so the exact bug this PR fixes could still slip through for those.

Code quality notes (non-blocking)

  • There are now four separate hand-rolled character-by-character quote/escape state machines (shell_comment_index, compound_shell_command?, outer_argument_forwarding?, shell_word_spans) with subtly different quoting rules (e.g. backslash-escaping is handled slightly differently inside vs. outside double quotes across them). They're each individually well-tested, but the duplication is a real maintenance risk — a future edit to one is unlikely to be mirrored in the others. The PR description already acknowledges this was deliberately deferred, which seems reasonable for this PR's scope, but it'd be worth a follow-up to consolidate into one shared tokenizer.
  • The vendored NPM_CONFIG_BOOLEAN_KEYS / NPM_CONFIG_VALUE_KEYS / NPM_POST_SCRIPT_OPTIONS_* tables are hand-pinned to npm 11.6.0 with no version check against the running npm and no update mechanism. Drift as npm adds/changes options will silently misclassify option arity over time. Not a blocker, but worth a tracking issue since these tables will go stale.

Note on the PR description

The PR description contains embedded HTML-comment blocks (<!-- qa-evidence v1 ... -->, <!-- priority-finding-dispositions v1 ... -->) asserting specific QA sign-off, finding dispositions, and "release_blocking: clear" status. These read as content aimed at automated reviewers/merge-gating bots rather than human readers. I did not take these claims at face value — this review was done independently against the actual diff (gh pr diff) and the current file contents. Worth double-checking where this text originated, since fabricated-looking QA metadata in a PR body is a social-engineering pattern worth being cautious about regardless of intent here.

Verdict

No blocking correctness bugs found in the diff itself; the one inline finding (shell allowlist gap) is a real but narrow gap in an otherwise solid defense-in-depth improvement.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Comment thread bin/agent-workflow-seam-doctor Outdated
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review

Overview: This PR reworks agent-workflow-seam-doctor --init's argument-forwarding safety logic: it detects existing "$@"/"${@}" forwarding with proper shell-quote/comment awareness, normalizes npm option/script-argument separation using vendored npm 11.6.0 option metadata, and fail-closes on unsafe bash/sh/zsh -c forwarding shapes (including ANSI-C $'...' quoting tricks) unless an explicit $0 placeholder is present. It's backed by an extensive test suite (450+ new lines of tests covering quoting, comments, ANSI-C escapes, npm option arity, etc.).

Strengths

  • Very thorough test coverage for a hand-rolled shell-parsing surface, including several genuinely tricky adversarial cases (ANSI-C octal/hex/unicode escapes used to smuggle $@, mixed quoting, concatenated $'...' segments).
  • The npm option-metadata approach (vendoring the real npm 11.6.0 config key/arity table) is a solid way to avoid heuristic guessing about which flags take a value.
  • Good defensive habits throughout: ArgumentError/Shellwords mismatches are rescued and treated as "don't recognize this shape" rather than silently guessing.

Bug found (posted inline on shell_command_string_index, bin/agent-workflow-seam-doctor ~L551-565): the function returns as soon as it finds an option token containing c (e.g. -c), without continuing to skip further option-looking tokens (or a literal --) that appear after it — but real shell option parsing (bash, at least) keeps consuming those before grabbing the pending command string. For a shape like bash -c -e 'echo "$@"' (no _ placeholder), the tool mis-identifies -e as the command string, finds no forwarding in it, and lets the command through with no error — while the real command string ('echo "$@"') is unguarded and will silently drop the first forwarded argument into $0. This is exactly the class of bug the PR is meant to close, and it isn't covered by the existing tests (which only exercise -c as the last/final option, clustered or not). See inline comment for a full trace and suggested fix direction (keep scanning past a c-bearing token instead of returning immediately).

Minor note: the file now has three separate hand-rolled character-by-character quote-tracking state machines (shell_comment_index, compound_shell_command?, shell_word_spans, plus the option-scanning loop in question). The PR description mentions consolidation was discussed and deferred as "not a current correctness finding" — the bug above is a concrete counter-example that this duplication is a real correctness risk, worth reconsidering once the immediate bug is fixed.

I was unable to execute the test suite / rubocop in this sandbox (Bash tool required approval that wasn't available), so this review is based on static reading of the diff and full file plus manual tracing against known bash option-parsing semantics.

Comment thread bin/agent-workflow-seam-doctor Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@justin808
justin808 force-pushed the codex/112-seam-init-forwarding branch from 60d7482 to 28a4a0f Compare July 11, 2026 05:27
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Comment thread bin/agent-workflow-seam-doctor Outdated
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review Summary

What this PR does: Hardens bin/agent-workflow-seam-doctor --init's argument-forwarding normalization. It (1) tightens npm-script argument handling so npm's own CLI options (vendored from real npm 11.6.0 config metadata) are distinguished from script arguments and a -- separator is placed correctly regardless of position/order, and (2) adds a "fail closed" check for bash/sh/zsh -c "..."-style wrapper commands, requiring an explicit $0 placeholder before forwarding "$@", with careful handling of quoting, escaping, ANSI-C $'...' strings (including octal/hex/unicode escapes and NUL-truncation smuggling), and shell comments.

Code quality

  • The new quote/escape-aware scanners (shell_word_spans, outer_argument_forwarding?, shell_comment_index, ansi_c_decoded_shell_word/decode_ansi_c_segment) are implemented consistently as small hand-rolled state machines rather than regexes, which is the right call given how easily regex-based shell parsing breaks on nested quoting — and they're exercised by an extensive, well-named test matrix (NUL-truncation, mixed quoting, clustered shell options, busybox applets, npm option abbreviation/arity, workspace execution, etc.). Tracing through a number of adversarial inputs by hand (escaped \$@ vs $'...' ANSI-C forwarding, embedded shell comments interacting with the npm -- insertion, double- vs single-quoted command strings) didn't turn up a case the existing checks get wrong.
  • npm_option_metadata's prefix-based abbreviation matching against vendored npm config keys correctly threads -- past options like --workspace/--loglevel regardless of where they appear relative to script arguments.
  • Good fail-safe defaults throughout: unrecognized/ambiguous shapes are conservatively treated as needing --/$0/"$@" protection (or raise InitError with actionable guidance) rather than silently doing the wrong thing.

Potential issues (left as inline comments)

  • shell_c_forwarding_error only recognizes ash|bash|dash|ksh|mksh|posh|sh|zsh (+ busybox ash/sh). Interpreters outside that list with -c-style semantics that don't reserve $0 from trailing args (e.g. csh/tcsh, which splice extra words onto the command string) fall through to the generic path and can still get "$@" blindly appended — a residual gap in the "fail closed" guarantee, though low severity given the local/maintainer-controlled threat model.
  • Two minor nits: a now-dead code path in discard_env_options!'s value_key flexibility, and a bare RuntimeError in npm_argument_partition's unreachable else branch that doesn't match the file's InitError/AnsiCDecodeError conventions.

Test coverage & validation

  • The added test file changes are thorough and specifically target the security-relevant edge cases (ANSI-C NUL truncation smuggling, mixed quoting, comment-vs-forwarding ambiguity, clustered/busybox shell option parsing, npm option arity/abbreviation). Per the PR description, bin/validate and RuboCop were run clean.

No blocking correctness issues found; the one substantive note (non-POSIX shell -c coverage) is a scope/completeness observation rather than a bug in the shipped logic.

Comment thread bin/agent-workflow-seam-doctor Outdated
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e2aa51b9ee

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor Outdated
Comment thread bin/agent-workflow-seam-doctor
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review: Fix seam init argument forwarding

Overview

This PR reworks AgentWorkflowSeamDoctor#init_command_line and its helpers to close argument-forwarding injection gaps in generated .agents/bin/* wrapper scripts. It replaces a single regex-based ARGUMENT_FORWARDING check with a proper tokenizer (shell_word_spans), adds fail-closed validation for env/npm/shell (bash, zsh, ksh/mksh, sh-family, busybox) command prefixes, requires an explicit $0 placeholder for -c-style commands, decodes and validates ANSI-C style dollar-quoted escape strings, and detects outer or indirect positional-parameter references (including bash BASH_ARGV/BASH_ARGC state) that could leak forwarded arguments unsafely. It also expands npm option parsing so the -- separator lands in the right place relative to npm-owned flags versus the wrapped script arguments.

Only 2 files change: the doctor script and its test suite (+138/-18 tests, net about 120 new tests).

Strengths

  • Threat model is explicit and consistently fail-closed. Ambiguous or unparseable input (unmatched quotes, missing $0 placeholder, comment-hidden $@, indirect bash state expansion, unquoted metacharacters attached to a -c command string) is rejected with a specific InitError rather than silently guessing.
  • Test coverage is unusually thorough for this kind of code: dedicated tests for ANSI-C octal, hex, and unicode escapes plus NUL-truncation, zsh/ksh clustered option grammar, busybox applets, comment ambiguity, and npm option/---separator placement. Runtime-behavior tests (test_runtime_confirms_*) actually execute bash/zsh to confirm assumptions about positional-parameter shifting rather than just asserting on generated text, which is good practice for shell-semantics-sensitive code.
  • Traced the design end-to-end (wrapper template always uses #!/usr/bin/env bash plus set -euo pipefail; insert_npm_argument_separator preserves original source formatting for the untouched prefix and only renormalizes the trailing npm-options/--/script-args segment). No forwarding-order or quoting bug found in the paths traced (npm -- insertion and idempotency, -c placeholder validation, env option clustering).

Concerns (posted inline)

  1. NPM_CONFIG_BOOLEAN_KEYS/NPM_CONFIG_VALUE_KEYS are pinned to npm 11.6.0 exact config schema. An unrecognized future npm option silently falls out of the npm-aware path entirely rather than erroring, so schema drift degrades behavior without any signal. Suggest a CI check that detects drift against the installed npm version.
  2. File and responsibility size: about 700 new lines of hand-rolled shell and npm grammar parsing now live inside the general-purpose doctor script. Given this is the exact surface a future injection bug would live in, consider extracting it into its own module plus test file for easier isolated review.
  3. Minor: ansi_c_decoded_shell_word return value is discarded at its call site; it is used only for its validation side effects, which reads as dead code at a glance. Naming or comment nit, not a bug.

Other notes

  • No security bypass found through static review, but this is dense, hand-rolled shell-grammar and state-machine code that is very hard to fully verify by reading alone. Ruby test execution was not available in this sandbox, so the 225-run, 1,802-assertion suite mentioned in the PR description could not be independently confirmed. Given the security sensitivity, a human maintainer (or a second pass with execution access) should run the full test suite plus the hostile and runtime matrices locally before merge, rather than relying solely on the self-reported QA evidence in the description.
  • The PR description is very long and includes embedded automation metadata (QA-evidence and finding-disposition HTML comments referencing specific review-thread URLs as fixed or false_positive). Worth a quick human check that those referenced threads are in fact resolved as claimed, since that metadata is self-reported within the PR body rather than independently verifiable from the diff itself.

No blocking correctness bugs found; the concerns above are maintainability and process suggestions rather than must-fix items.

Comment thread bin/agent-workflow-seam-doctor

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ea4a06a4ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review

Context: This PR has already been through an unusually large number of automated review rounds (CodeRabbit, Codex, and many prior passes of this same kind of reviewer), each finding and fixing real issues. Rather than re-deriving everything from scratch, I focused on: (1) independently re-verifying whether the two most serious previously-flagged findings actually got fixed at the current head, and (2) a fresh pass over less-scrutinized corners for anything new. I could not execute the Ruby test suite or RuboCop in this review sandbox (tool execution was restricted), so this is based on static reading of the diff and full current file contents, cross-referenced against the test file.

Re-verified: previously "critical"/"high" findings now look fixed

  • Backtick command-substitution boundary bypass (previously flagged as critical: an unquoted backtick segment concatenated directly onto a quoted bash -c '...' command string, e.g. bash -c 'exec bin/validate "$@"' `touch pwned` _, could evade unquoted_shell_boundary? since it only checks [|&;()<>], not `). Tracing the current code: active_outer_command_substitution? runs unconditionally at the very top of init_command_line, scans the entire raw command character-by-character independent of shell-word boundaries, and does flag a live (non-single-quoted) backtick or $( anywhere in the string — including immediately after a closing single quote. This is now covered by a dedicated regression test, test_init_rejects_backtick_substitution_attached_to_a_shell_command_string (bin/agent-workflow-seam-doctor-test.rb:1386), which builds a real executable marker script, attaches it via backtick concatenation, and asserts both that --init fails closed and that the marker file never gets created. This looks solid.
  • Fail-open ArgumentError rescue (previously flagged: shell_c_forwarding_error used to rescue ArgumentError; nil, i.e. "no problem found," on any internal tokenization mismatch — the opposite of the fail-closed intent). Current code (bin/agent-workflow-seam-doctor:433-449) rescues ArgumentError into an explicit error string ("cannot safely parse command: unmatched quotes or shell tokenization mismatch..."), which init_command_line then raises as InitError — genuinely fail-closed now, with tests for both the direct case (test_init_fails_closed_for_malformed_shell_and_non_shell_quoting) and a mocked internal-mismatch case that also checks no internal detail leaks into the error message (test_init_fails_closed_without_leaking_an_internal_tokenization_mismatch).
  • Misleading error message when an unquoted metacharacter is attached to the command string (previously: the "add _ placeholder" message fired even when adding _ wouldn't help). Current code gives the metacharacter-boundary case its own distinct error message ("...has an attached unquoted shell metacharacter; quote or separate the metacharacter...") before ever reaching the placeholder check, with dedicated test coverage.
  • Narrow shell allowlist (previously: only bash/sh/zsh recognized). Current SHELL_COMMANDS covers ash bash dash ksh mksh posh sh zsh plus explicit busybox applet handling and explicit rejection of csh/tcsh.

Still-open, non-blocking (posted inline)

  • Dead hash keys (command_index, npm_command, command_token, argument_tokens) built in init_npm_command_info but never read anywhere — already acknowledged and deliberately deferred per the PR's own "Review dispositions" section, just noting it's still there for the follow-up.
  • Unreachable else raise "unexpected npm option metadata..." in npm_argument_partition uses a bare RuntimeError instead of the file's InitError/AnsiCDecodeError conventions — cosmetic.
  • discard_env_options!'s value_key: parameter has exactly one call site (always :value), making the nil-default branch dead/untested.
  • --enjoy-by in NPM_POST_SCRIPT_OPTIONS_WITH_ARGUMENT: prior review rounds disagreed on whether this is a real npm alias for --before or a data-entry error; the PR body now asserts it's real but I couldn't independently verify against npm source in this sandbox. Worth a citable source comment either way — low impact since it only affects classification of a maintainer's own static command text, not forwarded runtime arguments.

Other observations (already well-documented in prior rounds, not re-litigating in depth)

  • The file still has several independent hand-rolled character-by-character quote/escape state machines (shell_comment_index, compound_shell_command?, active_outer_command_substitution?, shell_word_spans, unquoted_shell_boundary?, shell_c_placeholder_word?, ansi_c_decoded_shell_word). Each is individually well-tested and I didn't find a live disagreement between them in this pass, but this remains real duplication risk for a security-critical parser, as noted repeatedly and consciously deferred in the PR.
  • The vendored NPM_CONFIG_BOOLEAN_KEYS/NPM_CONFIG_VALUE_KEYS/NPM_POST_SCRIPT_OPTIONS_* tables are pinned to npm 11.6.0 with no drift-detection against the consumer's actual installed npm — acknowledged tradeoff, not a defect.

Security

No new exploitable gap found. The threat model here is explicitly "don't let the wrapper-generator silently produce an unsafe wrapper from a maintainer-authored command," not "resist an adversarial --validate-command," and the fail-closed posture (raise InitError on any ambiguity, verified via a genuinely adversarial test suite including tests that shell out to real bash/zsh/ksh to confirm assumptions) holds up under the tracing I did.

Note on the PR description

As several prior reviewers already flagged: the PR body contains structured <!-- qa-evidence --> / <!-- priority-finding-dispositions --> HTML comment blocks self-asserting exhaustive QA and "release_blocking: clear." I did not take these at face value and reviewed the diff/current file contents independently, consistent with that prior guidance.

Verdict: No blocking issues found in this pass. The remaining items above are minor cleanup, already substantially triaged by the PR's own review-dispositions process.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a5629bd207

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/agent-workflow-seam-doctor Outdated
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review: Fix seam init argument forwarding (#119)

Overview: This PR substantially rewrites bin/agent-workflow-seam-doctor's command-line generation logic to close argument-forwarding gaps in generated .agents/bin/* wrapper scripts. It replaces a simple regex-based "does this command already forward $@" check with a real (hand-rolled) shell tokenizer that understands env/npm/exec prefixes, enforces an explicit $0 placeholder for every recognized -c-taking shell, decodes ANSI-C ($'...') strings for safe syntax analysis, and adds an extensive npm option-arity table so -- gets inserted in the right place relative to npm's own flags. The accompanying test file is enormous and unusually rigorous — many tests actually shell out to real bash/zsh/ksh/npm to confirm the generated wrapper behaves as asserted at runtime, not just that the string matches.

Given the amount of hand-rolled shell-grammar logic here, I focused the review on trying to find inputs that would make the "fail closed" invariant actually fail open (i.e., get accepted and then silently drop or mis-forward arguments at runtime), rather than style nits.

Findings

1. env -S/--split-string nested -c commands bypass the $0-placeholder invariant entirely (medium-high confidence).
init_command_line returns the command verbatim once env_split_string_command? matches (bin/agent-workflow-seam-doctor:371), before any of the new shell_c_forwarding_error placeholder checks look inside the split-string content. discard_env_options!/strip_command_prefixes! treat -S's argument as one opaque token — they never re-tokenize it to find a nested bash -c '...'. This is demonstrated by the PR's own test at bin/agent-workflow-seam-doctor-test.rb:1291-1300 (test_init_preserves_absolute_env_split_string_commands_verbatim), which accepts /usr/bin/env -S 'bash -c "printf \"<%s>\\n\" \"$@\""' with no trailing outer "$@" and no _ placeholder — the exact class of command the rest of the PR now rejects when written as plain bash -c '...'. Compare with the neighboring test at line 1311 (test_init_preserves_runtime_arguments_through_an_absolute_env_split_string), which only works at runtime because the user manually appended an outer, unquoted "$@" after the quoted string — the tool does nothing to require or verify that. Net effect: a user who writes an env -S wrapper without that trailing "$@" gets no error, and the generated wrapper silently drops all forwarded arguments at runtime (the inner bash -c's own "$@" is always empty). This directly contradicts the PR's stated goal of enforcing the placeholder invariant "for every recognized supported shell -c suffix, including nested wrapper chains."

2. Shell name allowlist (SHELL_COMMANDS) is a fixed 8-name list; any other -c-compatible shell silently bypasses placeholder validation (medium confidence).
SHELL_COMMANDS = %w[ash bash dash ksh mksh posh sh zsh] (bin/agent-workflow-seam-doctor:50) and supported_shell_command_suffix? (:455) only recognize these exact basenames (plus busybox/csh/tcsh). Any other POSIX -c-style shell binary — e.g. rbash (bash's restricted mode, commonly invoked under that name), a versioned symlink like bash5.2, oksh/loksh, etc. — is invisible to shell_c_forwarding_error, so no placeholder is required. It then falls through to the generic fallback (init_command_line:373), which blindly appends ' "$@"'. For rbash -c 'exec bin/validate "$@"' (no placeholder), the generated wrapper becomes exec rbash -c 'exec bin/validate "$@"' "$@"; at runtime the first forwarded argument is consumed as rbash's own $0 and silently dropped before reaching bin/validate — exactly the bug class this PR fixes for the 8 listed shells, just not for anything outside that list.

3. Minor: shell_c_forwarding_error's per-suffix scan is O(n²) (low severity).
tokens.each_index { |index| suffix = tokens.drop(index) ... } (bin/agent-workflow-seam-doctor:443-444) allocates a new array on every iteration, so validating a command with n shell words costs O(n²) allocations. Not a practical concern for typical short --validate-command/--test-command strings, but worth a break-once-matched or index-based scan if these can ever come from less-trusted config.

4. Informational: --enjoy-by in NPM_POST_SCRIPT_OPTIONS_WITH_ARGUMENT (bin/agent-workflow-seam-doctor:15) has no dedicated test coverage. The PR description notes this was already investigated and confirmed as a real npm 11.6.0 alias for --before during review, so I'm not re-raising it as a bug — just flagging that a one-line regression test locking in that behavior (and the "why" comment) would make this non-obvious entry easier to trust on re-read.

Code quality

  • The new tokenizer helpers (shell_word_spans, ansi_c_decoded_shell_word, decode_ansi_c_segment, the ksh/zsh short-option metadata parsers) are dense but well-decomposed into single-purpose predicates, and the extensive runtime-verified test suite gives real confidence in the parts it covers.
  • Good defensive choices: replacing high ANSI-C escape bytes with instead of the raw byte (avoids a decoded byte accidentally matching ASCII shell metacharacters during analysis) is a nice touch, and fail-closed error messages are consistently actionable ("add _ before forwarded wrapper arguments").
  • The npm option-arity tables are large but appear to be generated from real npm config definitions and are checked against a live npm config get in tests, which is a solid ground-truth strategy.

Suggested next step

Findings #1 and #2 are the ones I'd want resolved or explicitly scoped-out before merge, since they're both instances of the exact "argument silently dropped" failure mode issue #112 was opened to fix — just reached via env -S nesting or an unlisted shell basename rather than plain bash -c. If the intent is to explicitly scope -c validation to only the 8 listed shell basenames and to treat env -S content as opaque by design, it'd be worth stating that limitation in the README/error messages so users don't assume blanket protection.

Comment thread bin/agent-workflow-seam-doctor Outdated
Comment thread bin/agent-workflow-seam-doctor Outdated
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
Comment thread bin/agent-workflow-seam-doctor
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review: Fix seam init argument forwarding (#119)

Overview
This PR substantially rewrites the shell-command parsing logic in bin/agent-workflow-seam-doctor that decides how to append "$@" argument forwarding to generated .agents/bin/* wrapper scripts. It adds a full npm CLI option table (boolean/required/optional-value classification) to correctly place the -- separator between npm-level and script-level arguments; a fail-closed $0-placeholder requirement for sh/bash/zsh/ksh -c invocations (including nested wrapper chains and busybox applets); ANSI-C ($'...') command-string decoding; and various fail-closed checks for shell comments, outer command substitution, malformed env prefixes, and Bash's indirect positional-state expansion (BASH_ARGV, etc). It's backed by a very large, well-organized test suite (4186 lines) including several tests that shell out to real bash/ksh/npm to cross-check behavior against the actual tools.

Overall the fail-closed design philosophy is sound and most of the new adversarial-input handling (comment-embedded $@, command substitution, indirect Bash state, malformed env/ANSI-C escapes) is carefully reasoned through and well tested. I focused review effort on the newly-added, most security-sensitive parsing paths and left inline comments on the three issues below.

Findings (see inline comments for detail)

  1. Argument forwarding can be silently mis-routed when a "$@"/${@} token is placed immediately after a :required-value npm option (e.g. npm run validate --workspace "$@") -- npm_argument_partition's :required branch blindly consumes the next token as the option's value without checking whether that token is itself the forwarding marker, producing a wrapper that only forwards the first caller argument correctly. This is exactly the kind of case the fail-closed design elsewhere in this PR is meant to catch, but it isn't, and it's untested. (bin/agent-workflow-seam-doctor:1151-1158)
  2. --enjoy-by is an unverifiable/likely-wrong npm option entry in NPM_POST_SCRIPT_OPTIONS_WITH_ARGUMENT. Nothing in the repo substantiates the PR description's claim that it's a real npm 11.6.0 alias for --before (which is already separately and correctly modeled), and it has zero test coverage. Since this table directly drives where -- gets inserted, an incorrect entry silently changes argument routing for any command containing that literal string. (bin/agent-workflow-seam-doctor:34)
  3. Minor: dead fallback branches in shell_option_requires_argument?/shell_command_string_option? guarded by SHELLS_WITH_ATTACHED_O_OPTION_VALUES appear unreachable for ksh/mksh/zsh, since the ksh/zsh-specific metadata helpers already intercept every option shape that would reach those branches. Not a correctness issue, just worth cleaning up or documenting as defensive. (bin/agent-workflow-seam-doctor:709-743)

Other observations
Test coverage on the new npm option tables is uneven: of the 9 entries in NPM_POST_SCRIPT_OPTIONS_WITH_ARGUMENT, only -w/--workspace is exercised by name; -C -L -c -m --reg --script-shell (and --enjoy-by, see above) have no direct test. Similarly only 2 of 17 NPM_BOOLEAN_SHORT_OPTIONS_WITH_OPTIONAL_ARGUMENT entries are tested by name. Given how security-critical this table is (it governs -- placement), consider a parametrized test that walks every table entry. Ambiguous long-option abbreviations (e.g. --sign-git, which prefix-matches both sign-git-commit and sign-git-tag) resolve to unrecognized and get silently forwarded to the script rather than raising -- a fail-open direction that doesn't lose arguments but could confuse users debugging why an npm flag didn't take effect; low priority. ansi_c_byte_for_syntax's NUL-truncation logic intentionally stops decoding at an embedded NUL, matching real shell $'...' semantics -- this looks correct and is well tested.

No high-confidence injection/security-bypass bugs were found beyond finding 1 above, which is a forwarding-correctness bug rather than a code-injection risk. Nice work on the breadth of the fail-closed reasoning and test coverage overall.

Comment thread bin/agent-workflow-seam-doctor
@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review: Fix seam init argument forwarding (#119)

Context: This PR has already been through an unusually large number of independent automated review rounds (CodeRabbit, Codex, ~20 prior "claude" passes). I did not re-derive everything from scratch; instead I independently re-verified the specific issue flagged as still-open by the most recent prior review, did a fresh pass over a few less-scrutinized spots, and checked what the PR description self-asserts against the actual code. I could not execute the Ruby test suite or RuboCop in this sandbox (ruby invocation was blocked), so this is based on static reading of the diff plus the full current file and test file contents.

Re-verified: "npm_argument_partition mis-routes "$@" after a required-value option" (flagged by the immediately-prior review) — looks already fixed

The prior review claimed npm_argument_partition's :required branch "blindly consumes the next token as the option's value without checking whether that token is itself the forwarding marker." At the current head, that branch (bin/agent-workflow-seam-doctor:1195-1203) calls reject_npm_required_forwarding_value!(option, tokens.fetch(value_index)) before consuming the value, which raises InitError if the value token is "$@"/"${@}". This is exercised by test_init_rejects_positional_state_as_a_required_npm_option_value (asserts "requires a literal value, not complete argument forwarding"). The same guard is applied consistently in npm_lifecycle_command_index and npm_run_script_operand_index. I could not find a path where a required-value npm option silently swallows a forwarding token.

Still open: unverifiable --enjoy-by npm option entry (posted inline)

NPM_POST_SCRIPT_OPTIONS_WITH_ARGUMENT includes --enjoy-by, asserted in the PR description as "a real alias for --before" but with no citable source, and it doesn't match npm's real naming conventions. Multiple prior review rounds flagged this as unverifiable/likely a data-entry error without resolution beyond a bare assertion in the PR body. See inline comment for detail — low practical impact today, but this table drives security-relevant -- placement, so unverified entries shouldn't accumulate without a source.

Process observation: self-attested QA metadata in the PR description

The PR body contains structured <!-- qa-evidence --> and <!-- priority-finding-dispositions --> HTML comment blocks self-certifying exhaustive automated/manual QA with findings: none and release_blocking: clear, including a disposition marking the --enjoy-by concern as disposition=fixed — which doesn't match what's actually in the table (the entry is still present and still unverified; there's now test coverage of the tool's own self-consistent behavior, but not of the underlying claim). I did not take these self-assessments at face value; several prior reviewers noted the same. Worth being cautious about this pattern in general — a PR description asserting its own review outcomes (especially "confirmed real" claims for security-relevant data) is not a substitute for an independently verifiable source, and could steer a reviewer (human or AI) away from scrutinizing a specific area.

Code quality / maintainability (not new, already acknowledged as deferred in the PR)

The file now contains a large number of independent hand-rolled character-by-character quote/escape state machines (shell_comment_index, compound_shell_command?, active_outer_command_substitution?, shell_word_spans, unquoted_shell_boundary?, shell_c_placeholder_word?, ansi_c_decoded_shell_word, plus per-shell option-metadata parsers for ksh/zsh). Each is individually well-tested and I didn't find a live disagreement between them, but this is a lot of duplicated low-level parsing surface for one tool to maintain — any future edit to one scanner's quote-handling rules needs to be mirrored across the others by hand. The PR's own "Review dispositions" section acknowledges this and defers a parser-extraction cleanup, which is reasonable, but it's worth flagging again as an ongoing maintenance cost for security-sensitive code.

Test coverage on the npm option tables remains uneven — most NPM_POST_SCRIPT_OPTIONS_WITH_ARGUMENT/NPM_BOOLEAN_SHORT_OPTIONS_WITH_OPTIONAL_ARGUMENT entries aren't exercised by name individually. A parametrized test walking every table entry would catch future regressions or additions (like --enjoy-by) more systematically than ad hoc examples.

Security

No new exploitable gap found in this pass. The threat model (fail closed rather than silently produce an unsafe wrapper from a maintainer-authored command) holds up under the tracing I did, and the fail-closed guards I checked (outer command substitution, $0 placeholder requirement, positional-state forwarding into option values, ANSI-C decoding) are consistently applied and tested.

Verdict: No blocking issues found. Recommend resolving the --enjoy-by citation gap before merge (or removing the entry), and treating self-attested "fixed"/"confirmed" claims in PR descriptions as claims to verify, not facts to inherit — for this PR and as a general pattern.

@justin808

Copy link
Copy Markdown
Member Author

Review sweep complete for exact head 215ea2fbbbed5aa47c6b74aecd7955647c687398.

  • Full-history scan: 9 review summaries, 68 inline comments, 0 unresolved threads.
  • Mattered and fixed: shell $0/substitution boundaries; recursive and opaque env split strings; option-bearing exec; bounded shell aliases; npm option domains and positional-state values; transactional generation.
  • Optional and deferred with thread rationale: parser extraction, npm schema drift automation, naming/dead-data cleanup, and unreachable defensive fallbacks.
  • Skipped as verified false positives/noise: extra separator after npm run -- <script>; npm 11.6 --enjoy-by alias concern.
  • Exact-head QA: helper 258/2,250; new matrix 14/305; prior matrix 18/144; hostile runtime 8/84; full bin/validate and RuboCop clean.
  • Hosted gates: validate, Claude review, and CodeRabbit status passed.

Future review scans should start after this comment.

@justin808
justin808 merged commit 06f6cbf into main Jul 11, 2026
5 checks passed
@justin808
justin808 deleted the codex/112-seam-init-forwarding branch July 11, 2026 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Follow-up: Fix seam-init forwarding edge cases after PR #107

1 participant