Skip to content

Add golden-output gate for the serverWebpackConfig generator template - #4790

Merged
justin808 merged 2 commits into
mainfrom
claude/4787-generator-golden-output
Jul 25, 2026
Merged

Add golden-output gate for the serverWebpackConfig generator template#4790
justin808 merged 2 commits into
mainfrom
claude/4787-generator-golden-output

Conversation

@justin808

@justin808 justin808 commented Jul 25, 2026

Copy link
Copy Markdown
Member

Fixes #4787

Problem

Generator template output was validated only by hand-maintained string fixtures in generator_spec_helper.rb that nothing checked against the real templates. The Pro and RSC standalone upgrades do not render serverWebpackConfig.js.tt — they gsub_file an existing base-install config, and their patterns were exercised only against those simplified fixtures. So the template could move and every spec would stay green while real user upgrades silently stopped matching.

#2489 is the worked example: it updated the template and base_server_webpack_content, and left pro_server_webpack_content on the old implementation. Nothing went red.

What this adds

1. Golden-output gate. spec/react_on_rails/generators/generator_golden_output_spec.rb runs the real BaseGenerator#copy_webpack_config into a mktmpdir — not a bare ERB render, so it exercises the same path a user's install takes, including the documentation-comment config and the bundler-specific destination directory. Seven variants:

Variant Flags Pins
webpack_base --no-rspack Base OSS install
webpack_pro --pro Pro branches
webpack_rsc --rsc RSC branches + RSCWebpackPlugin
rspack_base --rspack config/rspack/ destination mapping
rspack_pro --rspack --pro config/rspack/ + Pro
rspack_rsc --rspack --rsc RSCRspackPlugin + import path
webpack_base_shakapacker8 --no-rspack Shakapacker < 9 hardcoded-output-path arm

Shakapacker version detection is stubbed per variant so the golden files do not depend on whichever Shakapacker happens to be installed locally.

2. Structural anchor pinning. Per #4787's preferred option, the simulation fixtures are not forced to match golden output byte-for-byte — they intentionally represent older installs (a1 just added legacy_base_server_webpack_content_pre_get_loader_path for exactly that reason). Instead, each anchor the transforms actually match on is asserted present in both the golden file and the fixture the transform is exercised against.

Anchors were derived by reading the transforms, not the issue text:

Anchor Source
bundler require block ProSetup::BUNDLER_REQUIRE_PATTERN
getLoaderPath helper / declaration ProSetup::GET_LOADER_PATH_JS / GET_LOADER_PATH_DECLARATION
extractLoader helper ProSetup::EXTRACT_LOADER_JS
commented-out libraryTarget ProSetup#update_server_webpack_config_for_pro
commented-out target = 'node' block ProSetup#update_server_webpack_config_for_pro
cssLoader.options.modules block ProSetup#add_babel_ssr_caller_to_server_config
base / Pro module.exports shape ProSetup#update_server_config_exports
configureServer signature RscSetup#update_server_webpack_config_for_rsc
LimitChunkCountPlugin unshift RscSetup#update_server_webpack_config_for_rsc

Six of these are literals inside generator methods and cannot be referenced as constants, so they are duplicated in the spec. That duplication is itself pinned: a test asserts each copy still appears verbatim in pro_setup.rb / rsc_setup.rb, so editing the pattern in the generator without editing the spec is red. Without it this PR would have reintroduced the exact drift class #4787 exists to catch.

Also pinned: which add_extract_loader_to_server_config branch each simulation fixture exercises (exact-helper / reuse-existing-declaration / emit-both), so a fixture cannot quietly stop covering its branch.

3. serverBundleOutputPath fix. Reported on #4788's review and named in #4787: base_server_webpack_content referenced serverBundleOutputPath without ever declaring it, so the simulated config could never have run in Node. Fixed in that fixture and in the legacy pre-getLoaderPath fixture, and guarded by a new test requiring every output.path identifier to be declared before use.

Review feedback addressed (both P2 threads)

Blocker 1 — the ordering assertion did not prove nesting

Raised independently by greptile-apps and chatgpt-codex-connector at generator_golden_output_spec.rb:368.

The original rule-scope check compared byte offsets: rules.forEach before the Array.isArray(rule.use) guard before the css-module block. That is only a proxy. A template refactor that closes the forEach or the guard before the css-module block leaves all three tokens in the same relative order, so the check still passes while the Pro transform inserts extractLoader(rule, …) where rule is out of scope — producing an invalid generated config with the gate still green.

The golden diff only partly covers this: any restructuring changes the golden bytes, so a human has to look. But the same anchor assertion pins the simulation fixtures, which are deliberately allowed to drift. A regenerated golden plus a fixture that still satisfies token ordering means the Pro/RSC transform specs keep passing against a fixture that no longer represents a structure the transform can operate on. That is precisely the drift class this PR exists to close.

Fix: run the real ProSetup transforms (add_extract_loader_to_server_config then add_babel_ssr_caller_to_server_config) over the target, then assert the insertion point falls inside brace-matched rules.forEach and Array.isArray(rule.use) blocks.

Evidence — restructured input with tokens in unchanged relative order, nesting removed:

OLD CHECK offsets: forEach=1114 guard=1144 cssModules=1516      <- increasing, OLD CHECK PASSES
NEW CHECK: insertion_index=1918 rules_forEach_block=1305..1434
NEW CHECK: insertion inside rules.forEach? false
  Failure/Error: expect(rules_loop).to cover(insertion)
    expected 1305..1434 to cover 1918                            <- NEW CHECK FAILS

Same input, old check green, new check red. Green on the real templates and fixtures: 34 examples, 0 failures.

Blocker 2 — regeneration left obsolete variants behind

Thread at generator_golden_output_spec.rb:109.

regenerate! overwrote the current matrix but never removed directories for variants deleted or renamed in VARIANTS, and the examples enumerated only the current matrix — so a stale golden directory stayed committed and read as authoritative while nothing tested it.

Fix: assert the on-disk golden file set exactly equals the paths derived from VARIANTS.

Evidence — planted a stale webpack_legacy_removed/ directory:

$ REGENERATE_GENERATOR_GOLDEN=1 bundle exec rspec .../generator_golden_output_spec.rb
34 examples, 1 failure
$ ls spec/react_on_rails/fixtures/generated/
README.md  rspack_base  rspack_pro  rspack_rsc  webpack_base
webpack_base_shakapacker8  webpack_legacy_removed  webpack_pro  webpack_rsc

Regeneration rewrote the live matrix and left the orphan in place — confirming the defect — and only the new assertion caught it. After removing it: 34 examples, 0 failures.

It asserts rather than auto-deleting, so it fails loudly instead of silently removing a file someone added on purpose. It also catches a changed bundler destination mapping, since the expected paths are derived from it.

Evidence: the gate actually gates

Deliberate one-line edit to serverWebpackConfig.js.tt (minimize: falseminimize: true), a temporary local experiment only:

 76   serverWebpackConfig.optimization =  76   serverWebpackConfig.optimization =
 77     minimize: false,                  77     minimize: true,

Red — 32 examples, 7 failures at the time (all seven golden variants), with the readable diff:

  1) generator golden output rendered serverWebpackConfig.js matches the checked-in golden file for the webpack_base variant

     Generated config/webpack/serverWebpackConfig.js does not match the golden file for variant "webpack_base":

       spec/react_on_rails/fixtures/generated/webpack_base/config/webpack/serverWebpackConfig.js

     Diff (-golden +generated):

     @@ -53,7 +53,7 @@

        // No splitting of chunks for a server bundle
        serverWebpackConfig.optimization = {
     -    minimize: false,
     +    minimize: true,
        };
        serverWebpackConfig.plugins.unshift(new bundler.optimize.LimitChunkCountPlugin({ maxChunks: 1 }));


     If the template change is intentional, regenerate the golden files and review the diff:

       cd react_on_rails && REGENERATE_GENERATOR_GOLDEN=1 bundle exec rspec spec/react_on_rails/generators/generator_golden_output_spec.rb
       git diff react_on_rails/spec/react_on_rails/fixtures/generated/

Green after revert. The template edit was never staged or committed:

$ git diff HEAD -- react_on_rails/lib/generators/react_on_rails/templates/base/base/config/webpack/serverWebpackConfig.js.tt
(empty)

The template does not appear in this PR's file list.

Evidence: the regeneration command round-trips

$ REGENERATE_GENERATOR_GOLDEN=1 bundle exec rspec spec/react_on_rails/generators/generator_golden_output_spec.rb
[golden] Rewrote .../spec/react_on_rails/fixtures/generated; review `git diff` before committing.

$ git diff --stat react_on_rails/spec/react_on_rails/fixtures/generated
(empty)

Run against the staged baseline, so an empty diff means regeneration reproduced the committed bytes exactly. The command regenerates and then re-asserts, so a green run is itself the round-trip proof.

Why the root .gitignore needs one line

.gitignore:87 is a bare generated rule. It excluded the entire golden-fixture directory, which would have left every golden file uncommitted and the new gate dead on arrival in CI — the spec would fail on a fresh clone with "No golden file for variant".

$ git check-ignore -v react_on_rails/spec/react_on_rails/fixtures/generated/webpack_base/config/webpack/serverWebpackConfig.js
.gitignore:87:generated   react_on_rails/spec/react_on_rails/fixtures/generated/webpack_base/config/webpack/serverWebpackConfig.js

The fix is one directory negation, matching existing root-level precedent in the same file (!react_on_rails/spec/dummy/log/.keep, !react_on_rails_pro/spec/dummy/log/.keep, !/.tool-versions):

# Golden generator output (checked in; regenerated by generator_golden_output_spec.rb)
!react_on_rails/spec/react_on_rails/fixtures/generated/

It must negate the directory: git cannot re-include a file whose parent directory is excluded, so a per-file negation would silently fail. After the change git check-ignore exits 1 and the files are tracked. No files under any /generated/ path were previously tracked, so nothing else in the repo is affected.

Validation

Check Result
cd react_on_rails && BUNDLE_GEMFILE=../Gemfile bundle exec rubocop 244 files inspected, no offenses detected
cd react_on_rails && bundle exec rspec spec/react_on_rails/generators (serial) 1329 examples, 0 failures in 28m47s
Golden spec alone 34 examples, 0 failures
Golden spec with deliberate template edit red on all 7 golden variants (see above)
Blocker 1 restructured-nesting case old check green, new check red (see above)
Blocker 2 planted orphan variant red, then green after removal
Pre-commit hooks (prettier, eslint, rubocop, trailing-newlines) all passed, no golden file rewritten

All 7 golden files end with a trailing newline as committed, verified per file, so the trailing-newlines hook has nothing to rewrite.

Two operational warnings for whoever touches this next

The generator suite must run serially, and it is slow for a non-obvious reason. ~29 minutes wall clock at only a couple of minutes of CPU — it is I/O-bound on real npm and bundle subprocesses spawned by the dependency-manager specs, not CPU-bound. Do not parallelize to "speed it up": several generator specs share spec/react_on_rails/dummy-for-generators as their destination, and a parallel run corrupts state and produces false greens (found by the sibling lane on #4786).

trailing-newlines in .lefthook.yml runs with stage_fixed: true. A golden file that does not end in a newline is silently rewritten at commit time; the committed bytes then differ from generator output and the spec goes red for a reason that looks nothing like the cause. This warning is also in the fixture README next to the regeneration command, where someone adding a variant will actually see it.

Codex Decision Log

Non-blocking decisions, none of which change the requested scope:

  1. Kept the directory name generated/ rather than renaming it. .prettierignore and eslint.config.ts both ignore **/*generated*, and that exemption is what keeps the golden files byte-exact — prettier would otherwise reformat them and break the gate on every commit. Verified with a control test rather than trusting an ambiguous "All matched files use Prettier code style!" on a possibly-empty match set: identical deliberately-malformed JS gets [warn] Code style issues found outside generated/ and passes silently inside it. Renaming would have required editing .prettierignore and eslint.config.ts — strictly more non-owned files than the single .gitignore line.

  2. Chose transform-application over plain brace matching for Blocker 1. Both were on the table. Applying the real ProSetup transform is stronger because the transform is what actually inserts the code — brace matching alone would still be reasoning about the template's shape rather than about where the generator puts things. Brace matching is retained, but as the means of judging the landing site of a real transform rather than as a standalone structural claim.

  3. The brace scanner skips comments and string literals, which is required rather than cosmetic. The Shakapacker < 9 variant contains a template literal with ${serverBundleOutputPath}; a naive brace counter mis-matches on it and would report a confidently wrong block range. Line comments, block comments, and single/double/backtick strings are all skipped.

  4. Blocker 2 asserts rather than auto-deleting. Clearing the directory before regenerating would also fix the stale-variant problem, but it silently deletes whatever is there. Failing loudly is the right default for a gate whose purpose is to make drift visible.

  5. Added RSC variants beyond the "Pro on and off" matrix in the issue. RSC is the only case where webpack vs rspack changes the file contents (plugin class name and import path); without it, webpack_base and rspack_base are byte-identical files. Two explicit tests document this: non-RSC webpack/rspack output is asserted identical, and RSC output is asserted to differ only in the plugin name and import path.

  6. Stubbed Shakapacker version detection instead of reading the installed gem. The template branches on shakapacker_version_9_or_higher?; reading the real version would make golden files depend on the developer's local gem. Both arms are covered by the matrix instead.

  7. Ran the generator via copy_webpack_config rather than the full install generator. The full generator shells out to bundle and package managers; copy_webpack_config is the actual production code path that writes this file and keeps the spec fast without losing fidelity.

  8. Did not derive the simulation fixtures from the golden files (option 1 in the issue). They deliberately model older installs; deriving them would erase the legacy-upgrade coverage a1 just added.

Confidence note: High that the gate works as specified. Every claim here is backed by output I ran and observed: the deliberate-edit red/green, the byte-verified regeneration round-trip, both blocker fixes demonstrated failing before and passing after, and the full generator suite green at 1329 examples, 0 failures. High that the .gitignore negation is correct and minimal, verified by git check-ignore before and after.

UNKNOWNs:

  • Hosted CI results for this head are not in yet; local validation only. The coordinator owns the hosted-CI trigger.
  • The duplication pin catches a changed regex literal, but not a semantically equivalent reformat of a pattern in pro_setup.rb (for example switching %r{} to /.../ delimiters) — that would go red and need a matching spec edit, which is the intended cost, but it is a maintenance edge worth knowing about.
  • The brace scanner is a pragmatic lexer, not a JS parser. It handles the comment and string forms present in these generated files; it would need extending for regex literals containing unbalanced braces, which none of these files currently have.
  • Only serverWebpackConfig.js is pinned. The other managed webpack templates (clientWebpackConfig.js, commonWebpackConfig.js, ServerClientOrBoth.js, rscWebpackConfig.js) have the same drift exposure and are not covered here; extending the matrix to them is a natural follow-up but was out of scope for Generator: add golden-output test for serverWebpackConfig templates to stop fixture drift #4787.
  • The unused merge import flagged by github-code-quality is a pre-existing template wart faithfully reproduced by the golden files, tracked separately as Generated serverWebpackConfig.js imports 'merge' from shakapacker but never uses it #4791. The golden files are intentionally unchanged for it.

Summary by CodeRabbit

  • Tests
    • Added comprehensive golden-output verification for generated server webpack configurations across supported variant combinations.
    • Detects unexpected generator changes via byte-for-byte fixture comparisons and additional structural/content assertions (including RSC-related deltas and CSS modules/loader path expectations).
    • Updated test fixtures to set a valid server bundle output path for both current and legacy install scenarios.
  • Chores
    • Updated source control ignore rules to exclude regenerated golden generator outputs.

Generator template output was validated only by hand-maintained string fixtures
that nothing checked against the real templates, so the two drifted silently.
PR #2489 is the worked example: it updated the template and one fixture and
left a second fixture on the old implementation, and every spec stayed green.

Pin the real rendered output. A new spec runs BaseGenerator#copy_webpack_config
into a temp destination across seven variants (webpack/rspack x base/pro/rsc,
plus a Shakapacker < 9 variant) and diffs each produced serverWebpackConfig.js
against a checked-in golden file. Failures print a readable unified diff and the
one-line regeneration command, which regenerates and then re-asserts so a green
run is itself round-trip proof.

Pin the drift risk on the simulation fixtures without forcing them to match
byte-for-byte, since they deliberately represent older installs. Each structural
anchor the Pro and RSC gsub_file transforms match on is asserted present in both
the golden file and the fixture the transform is exercised against. Anchors are
derived from the transforms themselves; the six that are literals inside
generator methods carry an extra assertion that the duplicated copy still
appears verbatim in pro_setup.rb / rsc_setup.rb.

Fix the self-consistency defect reported on PR #4788: base_server_webpack_content
and the legacy pre-getLoaderPath fixture both referenced serverBundleOutputPath
without declaring it, so the simulated config could never have run in Node. A new
test requires every output.path identifier to be declared before use.

The root .gitignore carries a bare `generated` rule that would have left the
golden files uncommitted and the gate dead on arrival in CI. Add a single
directory negation, matching the existing root-level negation precedent.

Fixes #4787

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 25, 2026 12:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Adds golden-output coverage for serverWebpackConfig.js generator variants, structural transform-anchor checks, fixture consistency validation, and explicit output-path definitions in simulation templates.

Changes

Generator golden coverage

Layer / File(s) Summary
Fixture path contracts
react_on_rails/spec/react_on_rails/support/generator_spec_helper.rb
Webpack simulation fixtures define serverBundleOutputPath before assigning serverWebpackConfig.output.path.
Golden fixture infrastructure
react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb, .gitignore
Adds golden fixture generation, comparison, regeneration helpers, structural anchors, and ignores regenerated output.
Golden and transform assertions
react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb
Tests variant output equality, RSC-specific differences, transform anchor placement, loader-path branches, and fixture self-consistency.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers: copilot

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The new spec, golden fixtures, regeneration path, and structural-anchor checks match #4787's request to pin generator output and transform anchors.
Out of Scope Changes check ✅ Passed The .gitignore and fixture-path helper updates support the golden-output workflow and are within the linked issue's scope.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding a golden-output gate for the serverWebpackConfig generator template.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/4787-generator-golden-output

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jul 25, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds a checked-in golden-output gate for generated server webpack configuration.

  • Covers webpack and Rspack base, Pro, RSC, and legacy Shakapacker output variants.
  • Pins structural anchors used by standalone Pro and RSC transformations.
  • Makes simulated server configuration fixtures declare their output path.
  • Re-includes the generated fixture directory in Git and documents regeneration.

Confidence Score: 4/5

The PR appears safe to merge, with two non-blocking gaps in the new gate’s structural validation and regeneration cleanup.

The generated outputs and current matrix are pinned, but the scope check can accept non-nested anchors and regeneration can retain obsolete variant directories.

Files Needing Attention: react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb

Important Files Changed

Filename Overview
react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb Adds the golden generation and transform-anchor gate; its scope assertion is positional rather than lexical, and regeneration does not prune obsolete variants.
react_on_rails/spec/react_on_rails/support/generator_spec_helper.rb Declares serverBundleOutputPath in two simulated configurations before it is used.
react_on_rails/spec/react_on_rails/fixtures/generated/README.md Documents fixture ownership, regeneration, layout, and formatting constraints.
.gitignore Re-includes the checked-in golden fixture directory previously covered by the generic generated-directory ignore rule.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  V["Variant flags and Shakapacker version"] --> B["BaseGenerator#copy_webpack_config"]
  B --> T["Temporary generated config"]
  T --> C{"Matches checked-in golden?"}
  G["Checked-in golden variant"] --> C
  C -->|No| F["Spec failure with diff"]
  C -->|Yes| A["Validate transform anchors"]
  S["Simulation fixtures"] --> A
  A --> P["Guard standalone Pro/RSC rewrite assumptions"]
Loading

Reviews (1): Last reviewed commit: "Add golden-output gate for the serverWeb..." | Re-trigger Greptile

Comment thread react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb Outdated
@justin808

Copy link
Copy Markdown
Member Author

+ci-run-hosted

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e39c601086

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb Outdated
@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Hosted CI Requested

Triggered 9 workflow(s) for e39c60108641.
Skipped 0 workflow(s) with equivalent exact-head coverage.
Mode: optimized hosted CI (path-selected by script/ci-changes-detector).
Added ready-for-hosted-ci, so future commits will keep running optimized hosted CI until +ci-stop-hosted is used.

View progress in the Actions tab.

@claude

claude Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Review

Reviewed the diff (gh pr diff 4790) plus the full text of the new spec, the modified generator_spec_helper.rb, and cross-checked every "structural anchor" pattern in the spec against the actual literals in pro_setup.rb / rsc_setup.rb on this branch (not just trusting the PR description).

Overview

Adds a golden-output regression gate for serverWebpackConfig.js (7 variants: webpack/rspack x base/pro/rsc, plus a Shakapacker-8 arm), backed by checked-in fixture files under spec/react_on_rails/fixtures/generated/. Separately pins the exact literal/regex anchors that the standalone Pro/RSC gsub_file upgrades match on, so a template edit that silently breaks those upgrades goes red. Also fixes an undeclared-variable bug (serverBundleOutputPath) in two hand-written simulation fixtures and adds a .gitignore negation so the golden files are actually tracked.

Verification performed

  • Confirmed every duplicated anchor (LIBRARY_TARGET_COMMENT, TARGET_NODE_COMMENT, CSS_LOADER_MODULES_BLOCK, BASE_MODULE_EXPORTS, PRO_MODULE_EXPORTS, CONFIGURE_SERVER_SIGNATURE, LIMIT_CHUNK_COUNT_UNSHIFT) is byte-identical to the live regex/string literal in pro_setup.rb/rsc_setup.rb — the "keeps every inline anchor copy byte-identical" test is not a tautology, it genuinely pins real drift risk.
  • Confirmed TemplateRenderContext#shakapacker_version_9_or_higher? forwards via generator.__send__(...), so stubbing via generator.define_singleton_method(:shakapacker_version_9_or_higher?) { ... } in generate(variant) correctly overrides it even through the delegate wrapper — no test-double bypass bug here.
  • Confirmed .gitignore:87's bare generated rule would otherwise exclude the new fixture directory, and the added negation (!react_on_rails/spec/react_on_rails/fixtures/generated/) is present, and the golden files are actually tracked (git ls-files shows them checked in).
  • Confirmed the serverBundleOutputPath fix in base_server_webpack_content / legacy_base_server_webpack_content_pre_get_loader_path declares the identifier before its path: usage, satisfying the new self-consistency check.
  • Traced inline_anchor_text's .dump[1..-2] trick for the one string-literal anchor (PRO_MODULE_EXPORTS) — it works only because pro_setup.rb writes the replacement as a double-quoted string with literal \n escapes rather than a heredoc with real embedded newlines; verified that's indeed how update_server_config_exports is written, so the substring match is valid.

Code quality

  • Good separation of concerns: GeneratorGoldenOutput (fixture I/O) vs. GeneratorTransformAnchors (drift pins) vs. the example groups.
  • The REGENERATE_GENERATOR_GOLDEN=1 self-regeneration path doubles as its own round-trip proof (regenerate, then still assert) — exercised at file-load time rather than needing a separate example.
  • Failure messages (mismatch_message, missing_golden_message) are actionable and include the exact regeneration command — good DX for whoever breaks this next.
  • The rationale comments (why fixtures are not forced byte-identical to golden, why RSC needed extra variants, why Shakapacker version is stubbed rather than read) are exactly the kind of "why, not what" comments worth keeping.

Minor / non-blocking

  • GeneratorGoldenOutput.golden(name) re-reads the fixture file from disk on every call rather than memoizing; it's called 2-3x for the same variant across a few examples (e.g. "webpack_base"/"webpack_rsc"). Negligible given fixture sizes (~150 lines) and a 32-example suite, not worth blocking on.
  • The PR's own "UNKNOWNs" section already flags the two real residual risks accurately: (1) a semantically-equivalent regex reformat (e.g. %r{} -> /.../) in pro_setup.rb would go red on the duplication-pin test even though behavior is unchanged — an accepted maintenance cost, and (2) only serverWebpackConfig.js is covered; clientWebpackConfig.js/commonWebpackConfig.js/etc. have the same drift exposure and are explicitly out of scope here. Both are reasonable to leave as follow-ups rather than scope creep on this PR.

Security / performance

No concerns — this is test-only generator-output tooling with no runtime or production code paths touched. Dir.mktmpdir usage for golden generation is properly scoped and cleaned up.

Bugs found

None. I could not find a correctness issue in the anchor-pinning logic, the gitignore fix, or the fixture self-consistency checks after independently cross-referencing them against the actual generator source.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb (1)

268-421: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoff

Named RSpec subjects convention not followed.

None of the examples in this file use subject(:name); values like actual/expected/content/identifiers are computed as inline locals in every it block (e.g. Lines 271-281, 404-419). Repo guideline calls for named subjects (subject(:method_result)) rather than inline locals.

As per coding guidelines: "Ruby code must satisfy RuboCop, use a maximum line length of 120 characters, and use named RSpec subjects such as subject(:method_result)."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb`
around lines 268 - 421, Replace repeated inline local computations in the
generator golden output specs with named RSpec subjects, such as subjects for
generated output, expected golden content, fixture content, and scanned
identifiers. Define each subject at the narrowest applicable describe/context
scope and update the examples under the golden-output, structural-anchor, and
simulation-fixture self-consistency sections to use them while preserving
existing assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb`:
- Around line 268-421: Replace repeated inline local computations in the
generator golden output specs with named RSpec subjects, such as subjects for
generated output, expected golden content, fixture content, and scanned
identifiers. Define each subject at the narrowest applicable describe/context
scope and update the examples under the golden-output, structural-anchor, and
simulation-fixture self-consistency sections to use them while preserving
existing assertions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 491239c2-0dc4-40a6-8884-d8691f434128

📥 Commits

Reviewing files that changed from the base of the PR and between 3170f1c and e39c601.

⛔ Files ignored due to path filters (8)
  • react_on_rails/spec/react_on_rails/fixtures/generated/README.md is excluded by !**/generated/**
  • react_on_rails/spec/react_on_rails/fixtures/generated/rspack_base/config/rspack/serverWebpackConfig.js is excluded by !**/generated/**
  • react_on_rails/spec/react_on_rails/fixtures/generated/rspack_pro/config/rspack/serverWebpackConfig.js is excluded by !**/generated/**
  • react_on_rails/spec/react_on_rails/fixtures/generated/rspack_rsc/config/rspack/serverWebpackConfig.js is excluded by !**/generated/**
  • react_on_rails/spec/react_on_rails/fixtures/generated/webpack_base/config/webpack/serverWebpackConfig.js is excluded by !**/generated/**
  • react_on_rails/spec/react_on_rails/fixtures/generated/webpack_base_shakapacker8/config/webpack/serverWebpackConfig.js is excluded by !**/generated/**
  • react_on_rails/spec/react_on_rails/fixtures/generated/webpack_pro/config/webpack/serverWebpackConfig.js is excluded by !**/generated/**
  • react_on_rails/spec/react_on_rails/fixtures/generated/webpack_rsc/config/webpack/serverWebpackConfig.js is excluded by !**/generated/**
📒 Files selected for processing (3)
  • .gitignore
  • react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb
  • react_on_rails/spec/react_on_rails/support/generator_spec_helper.rb

Addresses two P2 review threads on #4790.

The rule-scope check compared byte offsets: rules.forEach before the
Array.isArray(rule.use) guard before the css-module block. That is a proxy, and
a weak one. A template refactor that CLOSES the forEach or the guard before the
css-module block leaves all three tokens in the same relative order, so the
check still passed while the Pro transform inserted extractLoader(rule, ...) at
a point where `rule` was out of scope. The hole matters most for the simulation
fixtures, which are deliberately allowed to drift from current output: a fixture
could keep satisfying token order while no longer representing a structure the
transform can operate on, which is the exact drift class #4787 exists to close.

Replace it by running the real ProSetup transforms
(add_extract_loader_to_server_config plus add_babel_ssr_caller_to_server_config)
over the target and asserting the insertion point falls inside brace-matched
rules.forEach and Array.isArray(rule.use) blocks. The brace scanner skips
comments and string literals, which is required rather than cosmetic: the
Shakapacker < 9 variant contains a template literal with
${serverBundleOutputPath} that a naive brace counter mis-matches.

Verified on a restructured input with the tokens in unchanged order: the old
offset check passes (1114 < 1144 < 1516) while the new check fails, reporting
the insertion at 1918 against a loop spanning 1305..1434.

Separately, regenerate! overwrites the current matrix but never removes
directories for variants deleted or renamed in VARIANTS, and the examples
enumerate only the current matrix, so a stale golden directory stayed committed
and read as authoritative while nothing tested it. Assert that the on-disk
golden file set exactly equals the paths derived from VARIANTS. This asserts
rather than auto-deleting, so it fails loudly instead of silently removing a
file someone added on purpose, and it also catches a changed bundler
destination mapping.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 25, 2026 13:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb (1)

185-189: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

BABEL_CALLER_INSERTION/BABEL_CALLER_MARKER are duplicated generator literals that the byte-identical pinning example does not cover.

inline_anchors only selects entries with :source_file, and these two constants live outside the anchor arrays. If ProSetup#add_babel_ssr_caller_to_server_config changes the inserted text, the pinning example stays green and the scope test fails with the misleading "could not locate the inserted extractLoader call". Consider adding them to the pinned set (or asserting both strings appear in pro_setup.rb) so drift points at the real cause.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb`
around lines 185 - 189, Update the golden-output pinning coverage around
BABEL_CALLER_INSERTION and BABEL_CALLER_MARKER so these generator literals are
included in the pinned set or explicitly validated against
ProSetup#add_babel_ssr_caller_to_server_config. Ensure changes to either
inserted string fail at the actual drift location rather than only producing the
downstream scope-test error.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb`:
- Around line 185-189: Update the golden-output pinning coverage around
BABEL_CALLER_INSERTION and BABEL_CALLER_MARKER so these generator literals are
included in the pinned set or explicitly validated against
ProSetup#add_babel_ssr_caller_to_server_config. Ensure changes to either
inserted string fail at the actual drift location rather than only producing the
downstream scope-test error.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bf604e06-2090-4d75-aa43-48ec29b30000

📥 Commits

Reviewing files that changed from the base of the PR and between e39c601 and fbcc770.

📒 Files selected for processing (1)
  • react_on_rails/spec/react_on_rails/generators/generator_golden_output_spec.rb

@claude

claude Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Review

I checked out the diff and cross-referenced the new spec logic against the actual generator source (base_generator.rb, pro_setup.rb, rsc_setup.rb) rather than just reading the fixtures. Summary below.

What this does

Adds a golden-output gate (generator_golden_output_spec.rb) that runs the real BaseGenerator#copy_webpack_config for 7 flag/bundler/Shakapacker-version combinations and diffs the result against checked-in fixtures, plus a second layer that pins the exact literal/regex anchors the standalone Pro/RSC upgrade transforms match on — so a template edit that silently breaks upgrade-path matching goes red instead of leaving hand-maintained simulation fixtures (which nothing else validates) untouched.

Verification performed

  • Traced TemplateRenderContext#shakapacker_version_9_or_higher? (base_generator.rb:120) back to generator.__send__(...) — confirms the spec's generator.define_singleton_method(:shakapacker_version_9_or_higher?) stub actually takes effect through the delegate, since it's the same instance.
  • Confirmed CONFIGURE_SERVER_SIGNATURE / LIMIT_CHUNK_COUNT_UNSHIFT regexes duplicated in the spec are byte-identical to the live patterns in rsc_setup.rb:526 and rsc_setup.rb:542.
  • Confirmed GET_LOADER_PATH_JS, GET_LOADER_PATH_DECLARATION, EXTRACT_LOADER_JS, BUNDLER_REQUIRE_PATTERN are public (non-private) constants on ProSetup, so referencing them directly (rather than duplicating) is safe and won't silently break under a private_constant change.
  • Verified the PRO_MODULE_EXPORTS/BASE_MODULE_EXPORTS dump/undump trick: since pro_setup.rb:572 writes the replacement string as a literal Ruby double-quoted string containing escaped \n sequences (not embedded newlines), anchor[:matcher].dump[1..-2] correctly reproduces that same escaped text for the File.read-raw comparison. This looked like a bug at first glance but isn't.
  • Confirmed the **/*generated* ignore actually exists in both .prettierignore:18 and eslint.config.ts:65, backing the "Prettier/ESLint won't reformat these" claim in the PR description.
  • Confirmed no generator template files are touched by this PR (only new fixtures, the new spec, a small generator_spec_helper.rb fixture fix, and .gitignore) — this is purely additive test infrastructure, low blast radius.

Code quality

  • GeneratorJsStructure's custom brace matcher (skipping line/block comments and string/template literals) is a nice touch — the PR description explains why a byte-offset proxy was insufficient (an early-closing block can preserve token order while breaking scope), and the code backs that up.
  • Anchors are derived from the transforms themselves and cross-checked against the generator source text (inline_anchor_text), so regex drift between the spec and the generator is self-defending. This is well thought through.
  • Good failure messages throughout (mismatch_message, missing_golden_message) that include the exact regeneration command — this will save real debugging time for whoever edits the template next.
  • Minor nit: inline_anchors dedup relies on Hash#uniq equality across BASE_INSTALL_ANCHORS/PRO_INSTALL_ANCHORS for the two shared anchors (configureServer signature, LimitChunkCountPlugin unshift). It works today because both hashes are built with identical literal values, but it's a slightly implicit way to express "these two lists share two entries" — a comment noting that the dedup is intentional (not just incidental) would help a future editor who changes one copy and wonders why the byte-identical-copy test didn't catch drift in the other.

Risk / gaps (mostly already called out by the author)

  • Only serverWebpackConfig.js is pinned; clientWebpackConfig.js, commonWebpackConfig.js, ServerClientOrBoth.js, rscWebpackConfig.js have the same drift exposure and aren't covered — explicitly flagged as out-of-scope follow-up in the PR description, seems reasonable to defer.
  • The regex-literal duplication check (inline_anchor_text) only catches literal changes, not semantically-equivalent reformats (e.g., %r{}/.../) — also explicitly called out by the author as an accepted maintenance cost.
  • No hosted CI run yet per the PR description; local validation only (rubocop clean, full generator suite 1327/0, golden spec 32/0, red/green round-trip demonstrated).

Overall this is a well-scoped, carefully verified addition to test infrastructure with no production code changes. I didn't find any functional bugs in the new spec logic.

@justin808
justin808 added this pull request to the merge queue Jul 25, 2026
Merged via the queue into main with commit 02f0d2e Jul 25, 2026
51 checks passed
@justin808
justin808 deleted the claude/4787-generator-golden-output branch July 25, 2026 13:47
@justin808

Copy link
Copy Markdown
Member Author

Batch handoff — ROR A 07-25 18:34, generator coverage + golden-output

Thread rorA-gen-kona, batch ror-a-20260725-1834-gen-golden. merge_authority: auto_merge_when_gates_pass. Both lanes ran strictly serial because they shared generator_spec_helper.rb and a2's golden files had to be generated from a1's final template text.

Final state: merged (both lanes).

Lane Issue PR Merge commit Issue state
a1 #4786 #4788 3170f1c3ea CLOSED
a2 #4787 #4790 02f0d2e749 CLOSED

main at 02f0d2e749: rollup SUCCESS, 92 checks, 90 passed / 2 skipped / 0 failing. The golden gate passes on both the 4.0/latest and 3.3/minimum matrices.

Immediate maintainer attention

None. No blockers, no open questions.

FYI / decisions made

a1 (#4788) — seven new assertions covering the two rspack SSR fixes from #2489, each proven non-vacuous by removing the corresponding template line and watching it go red. Stale pro_server_webpack_content fixture refreshed; the pre-fix body preserved as legacy_base_server_webpack_content_pre_get_loader_path for upgrade-path coverage. Loader-path expression de-duplicated into one getLoaderPath helper shared by the template and pro_setup.rb. The exportOnlyLocals guard was left alone per the issue.

  • One confirmed P2 fixed mid-review: branch selection used a byte-exact content.include?(GET_LOADER_PATH_JS), so a reformatted or customized helper fell through and emitted a second declaration — a SyntaxError beside an existing const. Now dispatches on three cases via a symbol-level matcher. Raised independently by chatgpt-codex-connector, claude, and copilot.
  • CHANGELOG entry added then removed. It stated outright that behavior is unchanged, and AGENTS.md plus .claude/docs/changelog-guidelines.md exclude refactors and tests. Ledger classification not_user_visible.

a2 (#4790) — golden-output gate over 7 variants (webpack/rspack x base/pro/rsc, plus Shakapacker-8), diffed byte-for-byte against checked-in expected files, with REGENERATE_GENERATOR_GOLDEN=1 documented in the spec header and fixture README. Proven by a deliberate one-line template edit: 32 examples, 7 failures red, 0 failures after revert. Nine structural anchors derived by reading the real transforms rather than the issue prose, with an extra assertion that each duplicated literal still matches its generator source verbatim.

  • Two confirmed P2s fixed mid-review. Anchor ordering compared byte offsets, which proves token order but not lexical nesting — replaced by applying the real ProSetup transforms and asserting the babel-caller insertion lands inside brace-matched rules.forEach / Array.isArray(rule.use) ranges. Separately, regenerate! left orphan variant directories committed and untested; now asserted rather than auto-deleted so it fails loudly.
  • Root .gitignore needed one appended negation: line 87's bare generated rule excluded the entire golden directory, which would have left the gate dead on arrival in CI. Verified with git check-ignore -v. The directory keeps the generated/ name deliberately, because .prettierignore and eslint.config.ts both skip **/*generated*, which is what makes byte-exactness free; renaming would have required editing two files instead of one.
  • serverBundleOutputPath was referenced but never defined in the simulation fixtures. Fixed, with a self-consistency test.

QA Evidence: not required. Spec-and-generator-template only, no runtime, server, or browser surface. Evidence is the generator specs plus hosted CI's generator shards. Confirmed the coverage actually executes in CI rather than only locally: detect-changes set run_gem_generator_specs=true and the generator shards ran green on both PRs.

Follow-ups filed (both out of scope, neither a regression from this batch):

Review surface: 21 threads across both PRs, all resolved, 0 unresolved at merge. copilot-pull-request-reviewer is not in .agents/trusted-github-actors.yml; its comments were treated as corroborating metadata only, never as review authority, and are queued for maintainer trust triage.

Operational notes worth keeping:

  • The generator specs share spec/react_on_rails/dummy-for-generators and must run in a single serial process. A parallel run silently corrupted an evidence pass and produced false greens.
  • trailing-newlines lefthook runs stage_fixed: true, so a golden file lacking a final newline is silently rewritten at commit time and breaks the gate. Recorded in the fixture README.
  • required-pr-gate evaluates at push time. On Cover the rspack CSS SSR generator fixes and de-duplicate the loader path #4788 hosted CI was requested 100 seconds after the push, so the gate failed on a stale read and needed a manual re-run; on Add golden-output gate for the serverWebpackConfig generator template #4790 the ready-for-hosted-ci label was already present, so it passed unattended. Request hosted CI before or with the final push.
  • script/pr-merge-ledger returned a transient ci_readiness: UNKNOWN while review bots were re-triggering; pr-ci-readiness reported READY at the same moment and the next ledger run agreed. Worth knowing before treating a single UNKNOWN as a blocker.

Confidence: high. Every number above is output that was run and read. Remaining UNKNOWNs, all recorded in the PR bodies: no real rspack/webpack build executes the generated configs; the brace scanner is a pragmatic lexer rather than a JS parser; and only serverWebpackConfig.js is pinned, with four other managed webpack templates carrying the same exposure as natural follow-up.

justin808 added a commit that referenced this pull request Jul 31, 2026
…t-policy

* origin/main: (33 commits)
  Fix ci-required base ref for PR merge-ref checkouts (#4819)
  Honor response charset and reject non-2xx HTTP-served SSR bundles (#4817)
  [Pro] Redact RSC render-error metadata on the fetched (client-navigation) payload path (#4821)
  Forward-port the 17.0.1 changelog section to main (#4814)
  Handle selector metacharacters in renderComponent DOM IDs (#4808)
  [Pro] Prevent caching RSC renders with errors (#4804)
  Agents: trust Copilot review identities (#4807)
  Agents: bind fleet closeout to generated pack (#4805)
  Docs: ADR 0002 — Skills-in-package over MCP for agent-native DX (#4735)
  Scope GitHub release commands to the origin repository (#4803)
  Forward-port OSS npm license metadata fix (#4794)
  Add golden-output gate for the serverWebpackConfig generator template (#4790)
  Cover the rspack CSS SSR generator fixes and de-duplicate the loader path (#4788)
  Configure agent workflow repo policy (#4785)
  Forward-port gh include mixed framing from #4684 (#4784)
  Release: enforce one-change forward-port closeout (#4783)
  Forward-port multi-URL rolling-deploy seeding to main (#4782)
  Docs: clarify React 18 streaming without RSC (#4780)
  Docs: forward-port v17 upgrade and generator gate guidance (#4781)
  Record the final React on Rails 17.0.0 changelog (#4742)
  ...

# Conflicts:
#	AGENTS.md
#	internal/contributor-info/release-train-runbook.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-hosted-ci Run optimized hosted GitHub CI for this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generator: add golden-output test for serverWebpackConfig templates to stop fixture drift

2 participants