Skip to content

feat(doctor): publish actionable JSON remediation - #4611

Merged
justin808 merged 8 commits into
mainfrom
jg-codex/4602-doctor-contract
Jul 12, 2026
Merged

feat(doctor): publish actionable JSON remediation#4611
justin808 merged 8 commits into
mainfrom
jg-codex/4602-doctor-contract

Conversation

@justin808

@justin808 justin808 commented Jul 12, 2026

Copy link
Copy Markdown
Member

Summary

react_on_rails:doctor FORMAT=json now gives agents a stable, validated remediation contract instead of requiring them to parse human-oriented prose. Every check keeps its existing stable ID and deterministic order while adding severity, check-specific documentation, nullable safe commands, and structured remediation context; existing pass/warn/fail exit behavior remains unchanged.

The runtime validates the published v1 invariants before emitting JSON, including status/severity consistency, unique IDs, nested detail shapes, exact summary counts, and overall status. Generated agent guidance points consumers at the contract, and the new API page documents fields, exit codes, safe command handling, and scoped runs. The docs URL is narrowly excluded from online link checks only until the new page deploys, matching the repository's existing planned-deployment policy.

Fixes #4602

Validation

  • Doctor and rake-task specs: 382 examples, 0 failures.
  • Generated-agent guidance spec: 1 example, 0 failures.
  • Manual broken-config smoke: valid JSON, stable failing check ID, structured remediation, and exit 1.
  • CI-equivalent RuboCop: clean for all changed Ruby files.
  • Prettier, ESLint, docs sidebar, generated llms-full.txt, and online Markdown links: clean.
  • Independent review found and fixed unsafe section-wide repair commands and incomplete semantic validation; current head re-review is clean.

Codex Decision Log

  • Non-blocking: Keep warning exits at 0 for compatibility.
    • Decision: Preserve the explicit fail/warn/pass classification and map warn/pass to 0 through the public contract.
    • Why: Existing CI behavior treats warnings as advisory; retaining the semantic distinction prevents future exit-policy drift.
    • Review later: None.

Confidence note: high for the JSON contract and compatibility surface; focused automated and manual coverage passed. Hosted CI remains the final repository-wide confirmation.


Compound Engineering
Codex

Summary by CodeRabbit

  • New Features
    • Enhanced react_on_rails:doctor with machine-readable FORMAT=json, including a versioned schema, stable check IDs, deterministic check ordering, structured remediation guidance, docs links, and CI-friendly exit codes (0 for informational/warnings, 1 for failures).
  • Documentation
    • Added API reference documentation for the JSON doctor contract, updated API navigation, and adjusted agent/install guidance to use JSON mode.
  • Bug Fixes
    • Improved link checking by excluding problematic generated doctor doc URLs.
  • Tests
    • Strengthened coverage for the JSON contract validation, deterministic output, and remediation payload expectations.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@coderabbitai

coderabbitai Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 69c3f04c-6a2e-4828-bd7c-1076687ee665

📥 Commits

Reviewing files that changed from the base of the PR and between ad78ed3 and b926424.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • react_on_rails/lib/react_on_rails/doctor.rb
  • react_on_rails/lib/react_on_rails/doctor_schema.rb
  • react_on_rails/spec/lib/react_on_rails/doctor_spec.rb
🚧 Files skipped from review as they are similar to previous changes (4)
  • CHANGELOG.md
  • react_on_rails/spec/lib/react_on_rails/doctor_spec.rb
  • react_on_rails/lib/react_on_rails/doctor.rb
  • react_on_rails/lib/react_on_rails/doctor_schema.rb

Walkthrough

The doctor command now emits a schema-validated JSON report with stable check metadata, remediation prompts, deterministic summaries, and CI-compatible exit codes. Documentation, generated agent guidance, changelog entries, and contract tests describe and verify the new format.

Changes

Doctor JSON contract

Layer / File(s) Summary
Schema contract and validation
react_on_rails/lib/react_on_rails/doctor_schema.rb
Defines report and check fields, status/severity mappings, metadata, remediation shapes, summary rules, and runtime validation.
Doctor report integration
react_on_rails/lib/react_on_rails/doctor.rb
Builds, validates, and prints JSON reports with schema-driven fields, remediation data, deterministic aggregation, and mapped exit codes.
Contract and behavior tests
react_on_rails/spec/lib/react_on_rails/doctor_spec.rb
Validates report structure, nested fields, deterministic output, remediation behavior, passing checks, duplicate IDs, and exit-code mappings.
Documentation and agent guidance
docs/oss/api-reference/doctor.md, docs/sidebars.ts, llms-full.txt, react_on_rails/lib/generators/react_on_rails/templates/agent_files/AGENTS.md, CHANGELOG.md, .lychee.toml, react_on_rails/spec/react_on_rails/generators/install_generator_spec.rb
Documents the JSON API, adds it to navigation and generated references, updates scaffolded guidance and changelog content, adjusts link checking, and verifies generated guidance.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RailsCommand
  participant Doctor
  participant DoctorSchema
  participant Stdout
  participant Stderr
  RailsCommand->>Doctor: run_json_diagnosis
  Doctor->>Doctor: build_json_report
  Doctor->>DoctorSchema: validate!(report)
  DoctorSchema-->>Doctor: validated report
  Doctor->>Stdout: print JSON report
  Doctor->>Stderr: print incidental output
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 13.64% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and directly describes the PR's main change: publishing actionable JSON remediation for doctor.
Linked Issues check ✅ Passed The PR delivers the requested JSON doctor contract, stable IDs, remediation prompts, exit-code mapping, docs, and spec coverage for #4602.
Out of Scope Changes check ✅ Passed The changes stay focused on doctor JSON output, docs, tests, and the related link-check exclusion needed for the new API page.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jg-codex/4602-doctor-contract

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.44.1)
react_on_rails/lib/react_on_rails/doctor.rb

ast-grep timed out on this file

react_on_rails/spec/lib/react_on_rails/doctor_spec.rb

ast-grep timed out on this file


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@justin808

Copy link
Copy Markdown
Member Author

+ci-status

@github-actions

Copy link
Copy Markdown
Contributor

CI Status

Head SHA: 2440336e878c
Changed files: 10
Docs-only heuristic (matches ci-changes-detector metadata paths): no
ready-for-hosted-ci label: absent
force-full-hosted-ci label: absent
Current hosted-CI waiver: not present for this SHA

Only the required gate is active unless hosted CI is requested.

@justin808

Copy link
Copy Markdown
Member Author

+ci-run-hosted

@github-actions github-actions Bot added the ready-for-hosted-ci Run optimized hosted GitHub CI for this PR label Jul 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hosted CI Requested

Triggered 9 workflow(s) for 2440336e878c.
Mode: optimized hosted CI (path-selected by script/ci-changes-detector).
Added ready-for-hosted-ci, so future commits will keep running optimized hosted CI until +ci-stop-hosted is used.

View progress in the Actions tab.

@greptile-apps

greptile-apps Bot commented Jul 12, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds a stable JSON contract for the React on Rails doctor command. The main changes are:

  • New DoctorSchema validation and metadata for JSON reports.
  • Expanded doctor JSON fields for severity, docs, remediation, and safe commands.
  • Updated doctor API docs, generated agent guidance, sidebar, and LLM docs.
  • Added specs for schema validation, deterministic output, and exit behavior.

Confidence Score: 5/5

This looks safe to merge after confirming the new schema file is included in packaged releases.

  • The current doctor checks match the new schema metadata and validation rules.
  • JSON report shape, summaries, details, and exit behavior have targeted test coverage.
  • The remaining concern is a packaging check for the newly required file.

react_on_rails/lib/react_on_rails/doctor.rb

Important Files Changed

Filename Overview
react_on_rails/lib/react_on_rails/doctor.rb Wires JSON output through DoctorSchema and adds structured check fields.
react_on_rails/lib/react_on_rails/doctor_schema.rb Adds the public doctor JSON contract, metadata, status helpers, exit codes, and runtime validation.
react_on_rails/spec/lib/react_on_rails/doctor_spec.rb Adds coverage for schema invariants, malformed reports, deterministic output, and exit codes.
docs/oss/api-reference/doctor.md Documents the Doctor JSON API fields, exit behavior, remediation workflow, and stable check IDs.
react_on_rails/lib/generators/react_on_rails/templates/agent_files/AGENTS.md Updates generated agent guidance to use the JSON doctor output.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Doctor FORMAT=json] --> B[Collect check results]
  B --> C[Build report]
  C --> D[Validate with DoctorSchema]
  D --> E[Emit JSON]
  E --> F[Exit from schema exit codes]
  C --> G[Add severity, docs, remediation, details]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
  A[Doctor FORMAT=json] --> B[Collect check results]
  B --> C[Build report]
  C --> D[Validate with DoctorSchema]
  D --> E[Emit JSON]
  E --> F[Exit from schema exit codes]
  C --> G[Add severity, docs, remediation, details]
Loading

Reviews (1): Last reviewed commit: "Allow doctor docs URL before deployment" | Re-trigger Greptile

Comment thread react_on_rails/lib/react_on_rails/doctor.rb
Comment thread react_on_rails/lib/react_on_rails/doctor_schema.rb
Comment thread docs/oss/api-reference/doctor.md Outdated
Comment thread react_on_rails/spec/lib/react_on_rails/doctor_spec.rb
@claude

claude Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Review

Overview

This PR turns bin/rails react_on_rails:doctor FORMAT=json into a versioned, validated public contract: each check gains severity, fix_command, docs_url, and a structured remediation (prompt/files/expected_end_state), and DoctorSchema.validate! asserts the invariants (status↔severity consistency, unique IDs, exact summary counts, nested details shape) before the report is printed. It adds a new docs/oss/api-reference/doctor.md page, updates the generated AGENTS.md template and CHANGELOG, and extends the exit-code handling to a pass/warn/fail0/0/1 table via DoctorSchema::EXIT_CODES. Test coverage is broad: schema validation, determinism, per-field shape, and the generator spec are all updated in step with the code.

Correctness issue

  • fix_command is dead code / always null. DoctorSchema::CHECK_METADATA (in doctor_schema.rb) defines files and expected_end_state for every check but never a fix_command key, so metadata[:fix_command] in doctor.rb#build_check_entry / #build_remediation is nil for every check in every possible report. This contradicts the PR's own documentation ("fix_command is present only when a single mechanical command is safe") and CHANGELOG entry ("safe mechanical fix commands when available") — a real capability gap for the stated goal of giving coding agents an actionable contract. Left inline comments on doctor_schema.rb, docs/oss/api-reference/doctor.md, and doctor_spec.rb (the existing spec only ever asserts fix_command is nil, so there's no positive test that would have caught this).

Code quality

  • Good consolidation: overall_status/summary logic moved out of Doctor and centralized in DoctorSchema, removing the prior duplication and giving the invariants a single source of truth.
  • DoctorSchema.validate! is a nice fail-fast guard against future contract drift, and the mutation-based spec (aggregate_failures over targeted mutations) is a good pattern for covering the invariant matrix concisely.
  • Minor nit: DoctorSchema.metadata uses Hash#fetch, so an unregistered check ID raises a KeyError before validate! ever runs (which would otherwise raise a clearer ArgumentError). Not a bug today since CHECK_METADATA keys mirror CHECK_SECTIONS 1:1, but worth keeping in mind if a new check section is ever added without updating CHECK_METADATA in the same commit.

Security

  • No new attack surface: fix_command (once real values exist) is documented as informational only and is never executed by the gem itself. capture_stdout's fd-level redirection is unchanged from prior behavior.

Test coverage

  • Solid for the validation/shape contract itself. The gap noted above (no case where fix_command is actually populated) is the main hole.

Nice overall structure — the main thing to resolve before merge is reconciling the fix_command promise in the docs/changelog with the fact that no check currently populates it.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
react_on_rails/spec/lib/react_on_rails/doctor_spec.rb (1)

192-192: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Replace Marshal.load(Marshal.dump(...)) with the JSON round-trip already used elsewhere in this file.

Static analysis flags Marshal.load/Marshal.dump as a deserialization risk. Here it's cloning locally-constructed, trusted data (not attacker input), so the risk is minimal, but this file already has a safer idiom in use (e.g. line 168) that avoids the lint noise entirely.

♻️ Proposed fix
-          mutated_report = Marshal.load(Marshal.dump(report))
+          mutated_report = JSON.parse(JSON.generate(report), symbolize_names: true)

Apply the same change at line 212 (mutated_report[:checks].first[:details].first case).

Also applies to: 212-212

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@react_on_rails/spec/lib/react_on_rails/doctor_spec.rb` at line 192, Replace
the Marshal.load/M棁arshal.dump cloning expressions assigned to mutated_report in
both affected cases with the JSON round-trip idiom already used elsewhere in
doctor_spec.rb, including the mutated_report[:checks].first[:details].first
case. Preserve the existing cloned data and subsequent mutations.

Source: Linters/SAST tools

react_on_rails/lib/react_on_rails/doctor_schema.rb (1)

19-78: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

CHECK_METADATA and doctor.rb's CHECK_SECTIONS must stay in sync manually — no enforcement.

metadata(check_id) uses CHECK_METADATA.fetch(check_id), and doctor.rb#build_check_entry calls this before DoctorSchema.validate! runs. If a future CHECK_SECTIONS entry is added in doctor.rb without a matching CHECK_METADATA entry here, doctor will crash with an unhandled KeyError instead of the friendly ArgumentError that validate_check!'s assert(CHECK_METADATA.key?(check[:id]), ...) is meant to provide (that assertion never gets a chance to run, since build_check_entry blows up first).

Consider adding a load-time or spec-level guard that the two id sets match, e.g. in doctor.rb after CHECK_SECTIONS_BY_ID is defined:

missing_metadata = CHECK_SECTIONS.map { |section| section[:id] } - DoctorSchema::CHECK_METADATA.keys
raise "Missing DoctorSchema::CHECK_METADATA for: #{missing_metadata.join(', ')}" if missing_metadata.any?
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@react_on_rails/lib/react_on_rails/doctor_schema.rb` around lines 19 - 78, Add
an explicit consistency guard between DoctorSchema::CHECK_METADATA and
doctor.rb’s CHECK_SECTIONS/CHECK_SECTIONS_BY_ID identifiers, preferably
immediately after the section index is defined, so missing or extra IDs fail at
load or spec time before build_check_entry calls metadata. Keep validate_check!
responsible for its existing friendly ArgumentError while ensuring both ID sets
remain synchronized.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CHANGELOG.md`:
- Around line 87-91: Update the “Agent-legible doctor contract” changelog entry
to append the standard `[PR ...] by [author]` attribution, preserving the
existing issue link and entry content.

---

Nitpick comments:
In `@react_on_rails/lib/react_on_rails/doctor_schema.rb`:
- Around line 19-78: Add an explicit consistency guard between
DoctorSchema::CHECK_METADATA and doctor.rb’s CHECK_SECTIONS/CHECK_SECTIONS_BY_ID
identifiers, preferably immediately after the section index is defined, so
missing or extra IDs fail at load or spec time before build_check_entry calls
metadata. Keep validate_check! responsible for its existing friendly
ArgumentError while ensuring both ID sets remain synchronized.

In `@react_on_rails/spec/lib/react_on_rails/doctor_spec.rb`:
- Line 192: Replace the Marshal.load/M棁arshal.dump cloning expressions assigned
to mutated_report in both affected cases with the JSON round-trip idiom already
used elsewhere in doctor_spec.rb, including the
mutated_report[:checks].first[:details].first case. Preserve the existing cloned
data and subsequent mutations.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1c4fa399-c49d-43cf-ba30-3f54e4fc5eb2

📥 Commits

Reviewing files that changed from the base of the PR and between 9f18542 and 2440336.

📒 Files selected for processing (10)
  • .lychee.toml
  • CHANGELOG.md
  • docs/oss/api-reference/doctor.md
  • docs/sidebars.ts
  • llms-full.txt
  • react_on_rails/lib/generators/react_on_rails/templates/agent_files/AGENTS.md
  • react_on_rails/lib/react_on_rails/doctor.rb
  • react_on_rails/lib/react_on_rails/doctor_schema.rb
  • react_on_rails/spec/lib/react_on_rails/doctor_spec.rb
  • react_on_rails/spec/react_on_rails/generators/install_generator_spec.rb

Comment thread CHANGELOG.md

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/oss/api-reference/doctor.md (1)

68-74: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reference the correct remediation field path.

files is documented as nested under remediation, but the workflow refers to it as though it were a top-level check field. This can cause agents to miss the relevant file paths.

Suggested wording
-2. Review `message`, `files`, and any `fix_command`.
+2. Review `message`, `remediation.files`, and any `fix_command`.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/oss/api-reference/doctor.md` around lines 68 - 74, Update the
broken-configuration workflow to reference file paths through the check’s
remediation field, alongside remediation.prompt, while preserving the existing
array-order handling and id, severity, message, and fix_command guidance.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@docs/oss/api-reference/doctor.md`:
- Around line 68-74: Update the broken-configuration workflow to reference file
paths through the check’s remediation field, alongside remediation.prompt, while
preserving the existing array-order handling and id, severity, message, and
fix_command guidance.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a8f4381e-65ad-4c30-8be7-d30bd8d9efc1

📥 Commits

Reviewing files that changed from the base of the PR and between 2440336 and ad78ed3.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • docs/oss/api-reference/doctor.md
  • llms-full.txt
  • react_on_rails/lib/react_on_rails/doctor_schema.rb
🚧 Files skipped from review as they are similar to previous changes (3)
  • CHANGELOG.md
  • llms-full.txt
  • react_on_rails/lib/react_on_rails/doctor_schema.rb

Comment thread react_on_rails/lib/react_on_rails/doctor.rb Outdated
Comment thread react_on_rails/lib/react_on_rails/doctor_schema.rb
@claude

claude Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Review: feat(doctor): publish actionable JSON remediation

Overview

This PR extends react_on_rails:doctor FORMAT=json with a versioned, self-validating contract: severity, docs_url, fix_command (currently always null), and a structured remediation block (prompt/files/expected_end_state) per check, plus a new DoctorSchema module that validates the emitted report shape before it's printed. A new docs page (docs/oss/api-reference/doctor.md) documents the contract, and the generated AGENTS.md template now points agents at FORMAT=json.

Overall this is a well-scoped, well-tested change. The runtime self-validation (DoctorSchema.validate!) before emitting JSON is a nice safety net — it converts silent contract drift into a loud, immediate failure. Test coverage for the new module is thorough (unique IDs, severity/status consistency, nested detail shapes, summary/status consistency, determinism).

Strengths

  • DoctorSchema.validate! runs on every JSON-mode invocation, so a future code change that breaks the documented contract fails fast in specs/CI rather than shipping a subtly wrong payload to consumers.
  • Good test coverage: schema validation happy-path, missing keys, semantic mismatches (status/severity, summary, duplicate IDs), and determinism across two independent Doctor instances.
  • Docs page is clear and the exit-code table matches the implementation (warn maps to exit 0, matching prior CI behavior — a good compatibility call, called out explicitly in EXIT_CODES).
  • .lychee.toml exclusion for the not-yet-deployed docs URL follows the existing "planned deployment" pattern already used elsewhere in that file.

Issues found (posted inline)

  1. Dead/untested code path (doctor.rb build_remediation, ~line 347): metadata[:fix_command] can never be truthy since no CHECK_METADATA entry defines a :fix_command key — the "Suggested command: ..." prompt-line insertion is unreachable and has no test exercising it. Either drop this until a real diagnosis populates fix_command, or add one worked example with a spec.
  2. Fragile manual sync between two files (doctor_schema.rb, DoctorSchema.metadata): CHECK_METADATA (in doctor_schema.rb) and CHECK_SECTIONS (in doctor.rb) must stay in lockstep by hand. If a new check section is ever added without a matching metadata entry, the failure is a bare KeyError rather than the friendly ArgumentError family used everywhere else in this module. Today this is only caught incidentally via specs that happen to stub every section. Consider a friendlier error message or an explicit parity spec.

Minor / non-blocking

  • The docs table's fix_command explanation is honest about the current limitation ("Version 1 currently emits null..."), which is good — but it does mean the field and its supporting code (see TODO for first version #1) are entirely speculative for this release. Worth double-checking this is intentional forward-compatibility rather than premature abstraction, per repo conventions against designing for hypothetical future requirements.

Security / Performance

No concerns — no new external input is parsed or shelled out to fix_command (it's descriptive text only, always null today), and the added validation is O(number of checks) with negligible overhead.

@justin808

Copy link
Copy Markdown
Member Author

Address-review summary

Scan scope: full PR history (no prior address-review summary cutoff).

Mattered

Optional

  • Metadata drift error: fixed in b926424 with a descriptive ArgumentError and regression spec.
  • 2 review-summary-only optional items remain pending/unselected outside the parent-selected live-thread fix set; no action taken this run.

Skipped

  • None.

Non-cutoff status only. The next review pass must use check all reviews.

@claude

claude Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Review: feat(doctor): publish actionable JSON remediation (#4611)

Overview

Extends react_on_rails:doctor FORMAT=json with a versioned, self-validating contract: severity, fix_command (currently always null), docs_url, and a structured remediation block (prompt/files/expected_end_state), plus a new DoctorSchema module that both builds and runtime-validates the emitted report. New docs page, sidebar entry, llms-full.txt regeneration, .lychee.toml exclusion for the not-yet-deployed docs URL, and the generated AGENTS.md template are updated to match.

Code quality

  • Clean extraction: DoctorSchema centralizes the schema version, status/severity mapping, per-check metadata, docs-URL construction, and validation, removing the duplicate overall_status implementation that used to live only in Doctor.
  • DoctorSchema.validate! reconstructs summary/status independently from the serialized checks array (not just re-checking the same in-memory Ruby objects), which is a good defense against a future regression silently desyncing summary/status from the actual checks payload.
  • CHECK_METADATA keys line up 1:1 with CHECK_SECTIONS ids (verified both lists — 13/13 match), and docs_url anchor slugs (#check-id-...) match the headings added in docs/oss/api-reference/doctor.md.
  • Good call keeping fix_command null in v1 rather than emitting a per-section "repair command" that could be unsafe for a subset of the findings a section aggregates (called out explicitly in the PR description as something an earlier review pass caught).
  • DoctorSchema.metadata raises ArgumentError ("Missing DoctorSchema::CHECK_METADATA entry for check id ...") if a future check section is added without matching metadata. That's a deliberate fail-fast guard (there's a dedicated spec for it) rather than a bug, but worth flagging: it means FORMAT=json would hard-crash in production for any app until the omission is fixed, rather than degrading gracefully (e.g., omitting remediation for that one check). Given this is caught at PR-review/CI time for a small, gem-owned list, current behavior seems reasonable — just confirming it's intentional.

Potential issues / nits (non-blocking)

  1. Duplicate exit-code derivation path (doctor.rb:1947-1955): diagnosis_exit_code recomputes fail/warn/pass from checker.errors?/checker.warnings? independently of the already-validated report[:status] that build_json_report/DoctorSchema.validate! produced from the same underlying messages. Today they can't diverge (same message list backs both), but since the whole point of this PR is a single source of truth for status, deriving the exit code from report[:status] (e.g. DoctorSchema::EXIT_CODES.fetch(report[:status].to_sym)) instead of re-deriving from checker would remove the possibility of the two ever drifting after a future refactor.
  2. No RBS signature was added for DoctorSchema in sig/. Doctor-related code doesn't appear to have RBS coverage before this PR either, so this is consistent with existing practice, not a regression — just noting it in case the project intends to expand RBS coverage going forward.
  3. Minor: validate_check_field_types! doesn't assert check[:id] is a String before the CHECK_METADATA.key? lookup in validate_check!. Not exploitable (a non-string id simply fails the "unknown check id" assertion), just slightly less precise error messaging if that ever happens.

Test coverage

Strong — new specs cover: schema validation passing on a real emitted report, zero-count summary handling, missing-metadata error message, missing required key rejection, six distinct semantic-mismatch mutations (severity/status, overall status, summary, duplicate IDs, non-string version, non-string title), malformed nested details, per-check severity/fix_command/docs_url/remediation assertions, fix_command staying null across unrelated failing sections, and determinism across two independent Doctor instances. The generator spec was updated for the new AGENTS.md wording. This is thorough for a public-contract change.

Security

No security concerns. fix_command is deliberately kept null in v1, avoiding emission of any command string an agent might blindly execute — the safest choice given a section can aggregate multiple distinct diagnoses.

Performance

Negligible — validation is O(checks) over at most ~13 entries per run.

Overall this is a well-scoped, well-tested change with good internal consistency between the Ruby implementation, the runtime validator, the generated docs, and the AGENTS.md template. The two items above are minor and non-blocking.

Comment thread react_on_rails/lib/react_on_rails/doctor.rb
@justin808
justin808 added this pull request to the merge queue Jul 12, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 12, 2026
@justin808
justin808 added this pull request to the merge queue Jul 12, 2026
Merged via the queue into main with commit a07fb81 Jul 12, 2026
64 checks passed
@justin808
justin808 deleted the jg-codex/4602-doctor-contract branch July 12, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-hosted-ci Run optimized hosted GitHub CI for this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agent-legible doctor: structured --format=json output, stable check IDs, copy-promptable fixes

1 participant