Skip to content

Scroll-priority streaming: candidate architecture evaluation (#4835) - #4841

Merged
AbanoubGhadban merged 1 commit into
mainfrom
scroll-priority-evaluation-4835
Aug 3, 2026
Merged

Scroll-priority streaming: candidate architecture evaluation (#4835)#4841
AbanoubGhadban merged 1 commit into
mainfrom
scroll-priority-evaluation-4835

Conversation

@AbanoubGhadban

@AbanoubGhadban AbanoubGhadban commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Independent evaluation of the six candidate architectures from the scroll-priority streaming design doc (#4769), plus 30+ additional approaches discovered during research. Addresses issue #4835.

Produced via 4 deep-research workflows (24 agents, ~1.5M subagent tokens, 631 tool invocations) with adversarial verification against primary sources.

This PR adds only the evaluation document. No code changes.

The Fundamental Tension

A key finding: zero duplicates + CDN-served + first-visit + bot-safe are mutually exclusive. No architecture satisfies all four. The evaluation makes this tension explicit and ranks approaches by which constraint they relax.

Candidate Verdicts

Candidate Verdict Constraint violated
C1 — DO edge-state Conditionally viable Adds hosting cost (~$47-94/mo per 1M views)
C2 — SW stream-stitching Ruled out Fails first visit (SW doesn't exist)
C3window.stop() + pull Ruled out Fails bot safety (kills all network)
C4 — No-stop client pull Viable — recommended Accepts 21-35KB duplicate bytes
C5prerender/resume Conditionally viable — defer Blocked on unstable_postpone()
C6 — Pull-only tail Ruled out Fails bot safety (no JS = skeletons)

Key Findings Beyond the Design Doc

  1. C2 (SW) fails the first-visit constraint — the navigation response goes directly from network to HTML parser; the SW cannot retroactively intercept it, even with clients.claim(). 35-75% of traffic is uncontrolled.

  2. SW mid-section abort causes HTML parser state corruption — an unclosed <div hidden id="S:4"> traps subsequent sections inside it. $RC("B:4","S:4") then swallows the trapped sections.

  3. CDN abort propagation is unreliable — Cloudflare Workers' request.signal has an active regression (workerd#6832). CloudFront and Fastly docs don't address viewer disconnect propagation.

  4. New approach: Server-side section skipping (Rank 3) — client fetches + confirms receipt; server skips that section in the stream. Zero duplicates when origin holds the stream. Bot-safe via timeout fallback. ~40 lines delta.

Recommended Layered Architecture

Layer Approach When Effort
1 C4 fetch + $RC (accept duplicates) Ship now 7-10 days
2 Skip-delay POST (complementary) Ship in parallel 2-3 days
3 Server-side section skipping (zero dup, origin only) When origin streams 2.5 days
4 React PPR When postpone() stable (2027+) 21-31 days

Files Changed

  • internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md (new file, 557 lines)

Closes #4835

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added a comprehensive evaluation of scroll-priority streaming approaches.
    • Documented browser, CDN, HTML parsing, React, security, SEO, and back-forward cache considerations.
    • Compared multiple implementation options across bandwidth, portability, upgrade safety, and bot completeness.
    • Included recommended approaches, implementation phases, acceptance criteria, estimates, and supporting research references.

Constraint-filtered evaluation of 6 candidates from the design doc (#4769)
plus 30+ additional approaches. Tested against two hard production
constraints: first-visit compatibility and bot/SEO safety.

Key findings:
- Zero duplicates + CDN-served + first-visit + bot-safe are mutually
  exclusive (the fundamental tension)
- C2 (SW) ruled out: fails first visit, Safari 10s idle timeout
- C4 (no-stop client pull) recommended: already built, zero blast radius
- C3 (window.stop) ruled out: kills all network, breaks bots
- C6 (server islands) ruled out: bots see permanent skeletons
- New Rank 3: server-side section skipping (~40 lines, zero duplicates
  when origin holds stream)

Method: 24 agents, ~1.5M tokens, 631 tool invocations with adversarial
verification against primary sources.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The PR adds a technical evaluation of scroll-priority streaming architectures. It compares browser, CDN, React, bot, SEO, CSP, and deployment constraints, recommends a layered approach, and defines phased prototypes, acceptance criteria, and research references.

Changes

Scroll-priority streaming evaluation

Layer / File(s) Summary
Evaluation scope and candidate verdicts
internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md
Defines production constraints, compares candidate architectures, recommends client fetch with $RC, and rules out approaches that fail first-visit, bot-completeness, portability, or delivery requirements.
Candidate architecture analysis
internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md
Analyzes Service Worker stream interception, client-fetch delivery, origin-side section skipping, Cloudflare Durable Objects, and React PPR.
Cross-cutting findings and layered architecture
internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md
Adds comparison matrices, resolves technical questions, records CSP and SEO findings, and defines bfcache and deployment requirements.
Implementation plan and evidence
internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md
Adds phased work plans, prototype and integration-test criteria, trigger conditions, drop decisions, and categorized source references.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

  • Issue 4776 — Directly evaluates the same scroll-priority candidates and constraints, with less coverage of bfcache and readiness behavior.
  • Issue 4774 — Covers the external React Fizz runtime and CSP strategy referenced by this architecture evaluation.

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the scroll-priority streaming architecture evaluation documented by this pull request.
Linked Issues check ✅ Passed The document evaluates all six candidates, verifies open questions, assesses React on Rails fit and costs, and provides recommendations and prototype criteria for issue #4835.
Out of Scope Changes check ✅ Passed The pull request changes only the requested evaluation document and contains no unrelated implementation or code changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch scroll-priority-evaluation-4835

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 3, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds an internal evaluation of scroll-priority streaming architectures and recommends a layered client-fetch and server-skipping strategy.

  • Compares six proposed candidates and more than thirty additional approaches.
  • Documents first-visit, bot-safety, CDN, React-compatibility, and bandwidth tradeoffs.
  • Provides implementation phases and prototype acceptance criteria.

Confidence Score: 4/5

The documentation-only PR is safe to merge after correcting the non-blocking inverted server-skipping predicate.

The architecture narrative consistently describes skipping confirmed sections and sending unconfirmed sections after a timeout, but its illustrative predicate does the opposite for confirmed sections.

Files Needing Attention: internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md

Important Files Changed

Filename Overview
internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md Adds a comprehensive architecture evaluation, but the server-skipping pseudocode inverts the confirmation and timeout semantics described by the proposed design.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Scroll[Client scrolls to section] --> Fetch[Fetch cached section]
  Fetch --> Reveal[Reveal through React boundary]
  Reveal --> Confirm[POST receipt confirmation]
  Confirm --> Decision{Client confirmed?}
  Decision -->|Yes| Skip[Skip section in original stream]
  Decision -->|No, timeout elapsed| Send[Send section for bot-safe fallback]
Loading

Reviews (1): Last reviewed commit: "Scroll-priority candidate architecture e..." | Re-trigger Greptile

Comment on lines +233 to +236
def should_send_section?(stream_id, index, section_queued_at)
client_confirmed?(stream_id, index) ||
(monotonic_now - section_queued_at) > BOT_TIMEOUT_SECONDS
end

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Section-skipping predicate is inverted

When a client confirms receipt, should_send_section? returns true and sends the section again, contradicting the documented requirement to skip confirmed sections and defeating the zero-duplicate behavior.

Suggested change
def should_send_section?(stream_id, index, section_queued_at)
client_confirmed?(stream_id, index) ||
(monotonic_now - section_queued_at) > BOT_TIMEOUT_SECONDS
end
def should_send_section?(stream_id, index, section_queued_at)
!client_confirmed?(stream_id, index) &&
(monotonic_now - section_queued_at) > BOT_TIMEOUT_SECONDS
end

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 600313b44c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +234 to +235
client_confirmed?(stream_id, index) ||
(monotonic_now - section_queued_at) > BOT_TIMEOUT_SECONDS

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Invert the confirmed-section send predicate

For the section-skipping design, this predicate sends exactly the sections that have already been confirmed by the client: when a scrolled user posts confirmation, client_confirmed? becomes true and the stream still emits the duplicate, while unconfirmed/bot requests wait until the timeout. That contradicts the preceding “skip it in the stream” contract and would eliminate the promised zero-duplicate behavior; the confirmed case needs to suppress sending, not allow it.

Useful? React with 👍 / 👎.


### Rank 2: C4 + Origin Skip-Delay Signal (Both Already Built)

**Status:** Both modes exist; composing them is ~3 lines of client code.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not mark missing prototypes as already built

In this commit's tree, rg --fixed-strings 'fetchAndRevealSection', mode=fetch, selective_hydration_skip_delay, and signalSection only hit this new analysis file, and git ls-tree -r eee509ce has no selective_hydration_scroll_demo.js or skip-delay route. So for readers implementing Layer 2, this “Both modes exist” status leaves the required fetch/skip-delay code absent while the plan budgets only a 3-line composition and “no server changes”; either include the prototype reference/branch/commit or mark this work as not yet built.

Useful? React with 👍 / 👎.

Comment on lines +232 to +237
# Server: two-phase delivery with bot-timeout fallback
def should_send_section?(stream_id, index, section_queued_at)
client_confirmed?(stream_id, index) ||
(monotonic_now - section_queued_at) > BOT_TIMEOUT_SECONDS
end
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pseudocode looks inverted relative to the design it's illustrating. The surrounding diagram says the server should skip the section once the client has confirmed receipt, and only send it if the timeout fires with no confirmation:

→ Server: "client has 7, skip it in the stream"   (zero dup)
...
→ Server timeout: send section anyway after 10s   (bot-safe)

But as written, should_send_section? returns true (send) whenever client_confirmed? is true — i.e. it sends because the client confirmed, which is the opposite of "skip it in the stream." Confirmation should suppress sending, not trigger it:

def should_send_section?(stream_id, index, section_queued_at)
  !client_confirmed?(stream_id, index) &&
    (monotonic_now - section_queued_at) > BOT_TIMEOUT_SECONDS
end

Since this is the illustrative snippet for Rank 3's entire selling point ("zero duplicate bandwidth"), it's worth fixing so a future implementer doesn't copy the inverted logic and end up re-sending every section the client already fetched.


### Rank 1: C4 — Client Fetch + `$RC` While Stream Continues

**Status:** Already built and working (`?mode=fetch`).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "Already built and working" claim (and the selective_hydration_scroll_demo.js lines 243-279 cited just below, plus ?mode=fetch, the /cache/selective_hydration_demo/section*.html URLs, and the /selective_hydration_skip_delay/:stream_id endpoint referenced later) don't correspond to anything in this repository — I searched the full tree at this PR's merge commit and there's no selective_hydration_scroll_demo.js, no fetchAndRevealSection, and no matching route. The only related file is react_on_rails_pro/spec/dummy/app/views/pages/selective_hydration_demo.html.erb, which doesn't contain this scroll-priority/fetch logic.

The doc's frontmatter names Branch: selective-hydration-scroll-priority-demo (line 6), but that branch isn't linked anywhere and I couldn't confirm it exists. Since "Layer 1: Ship Now" (§8) is scoped as just "extract fetchAndRevealSection into Pro module" — i.e., it assumes this code already exists and just needs moving — it'd be worth adding a direct link (branch URL, commit SHA, or draft PR) so this load-bearing claim is verifiable and the effort estimates in §8 aren't based on code nobody else can currently find.

@claude

claude Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review summary

This is a docs-only PR (one new file, internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md, no code changes), so the review focused on internal consistency and verifiability of the claims rather than code correctness/security/perf in the usual sense. Overall the evaluation is well-organized (clear constraint framework, ranked options, explicit sourcing tiers), but I found two things worth fixing before this is relied on for implementation decisions:

  1. Logic bug in the Rank 3 example code (lines 232-237): the should_send_section? pseudocode sends the section because the client confirmed receipt, which is the inverse of what the surrounding diagram describes ("client has 7, skip it in the stream"). As written it would defeat the "zero duplicate bandwidth" claim that's the whole point of Rank 3. Left an inline comment with the corrected condition.

  2. Unverifiable "already built" claim underpinning the top recommendation (line 160 area): the doc treats selective_hydration_scroll_demo.js (with specific line numbers, ?mode=fetch, and several endpoint paths) as existing, "already built and working" code, but none of it is present anywhere in this repository at the PR's merge commit. The frontmatter names a branch (selective-hydration-scroll-priority-demo) but doesn't link it. Since Layer 1 of the recommendation ("ship now") is scoped as just extracting this existing code, it'd help to add a concrete link (branch/commit/PR) so the claim — and the 7-10 day effort estimate built on top of it — can actually be checked.

Nothing else jumped out as incorrect; the other citations I spot-checked against the current codebase (e.g. streamServerRenderedReactComponent.ts line 287 nonce usage) were accurate.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

🧹 Nitpick comments (1)
internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md (1)

124-124: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add languages to all fenced code blocks.

markdownlint-cli2 reports MD040 at Lines 124, 220, and 397. Add text to each diagram fence, or use the correct language for code fences.

Also applies to: 220-220, 397-397

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` at line
124, Add an appropriate language identifier to every fenced code block in the
document, specifically the fences around lines 124, 220, and 397; use text for
diagram or plain-content fences, or the accurate language where applicable, so
all blocks satisfy markdownlint MD040.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md`:
- Around line 154-162: The document has an inconsistent survivor count: Section
4 claims five architectures, while Section 5.4 identifies only three original
candidates and two C4 variants. Update the relevant sections to distinguish the
three original candidates from ranked C4 deployment variants, or revise the
count to three original candidates, preserving the intended ranking.
- Around line 32-44: Update the four-way exclusion in the document to apply
specifically to “pure static CDN replay,” including the related occurrence near
the C1 discussion. Clarify that C1 uses stateful edge delivery via a Durable
Object and therefore relaxes the no-per-request-state or pure-static-replay
constraint, while preserving the existing tradeoff analysis.
- Around line 218-246: Update should_send_section?, client_confirmed?, and
confirmSectionReceived so receipt confirmations use a short-lived, unguessable
capability bound to the specific response, section, session/CSRF context, and
bot-timeout window; validate and consume that capability before suppressing
streamed HTML, rejecting fabricated or replayed POSTs.
- Around line 337-344: The C4 evaluation must verify client-component support in
the fetched lazy-load path before retaining its compatibility claim. Update the
evaluation around fetchAndRevealSection to add a fetched-section
client-component fixture and confirm RSC references, hydration data, actions,
and event handlers work after adoption; alternatively, explicitly restrict the
path to server-only sections and revise the affected compatibility claims.
- Around line 76-81: Revise the C3 entry to remove the claim that window.stop()
cancels all in-flight network activity, and describe only its effect on further
resource loading in the browsing context. Keep the verification focused on
rendered page/document behavior or cite observed behavior for any fetch/XHR
effects.
- Line 70: Update the SharedWorker row in the compatibility analysis to reflect
Safari 16 support, removing the outdated claim that Safari lacks SharedWorker
and has no plans to restore it. Reassess the compatibility conclusion and revise
the associated 15–25% traffic-share impact so SharedWorker is not ruled out on
stale information.
- Around line 383-385: Update the SEO section’s source classification and
attribution: replace the asklantern.com AI-crawler reference with the official
Vercel study link, and revise the “1B+ monthly bot requests” statement to match
the study’s measured scope and publication date.
- Around line 273-285: Update the “Rank 5: React PPR Single-Response Variant”
section to describe scroll-controlled data-resolution order as a hypothesis
rather than confirmed behavior. Remove or qualify the “confirmed” claim and
document that a prototype must validate ordering, abort behavior, and
same-response streaming before treating the mechanism as viable.
- Around line 484-485: Update the acceptance criteria in the
AbortController/pagehide lifecycle section to detect bfcache restoration with
pageshow and event.persisted, rather than relying on
PerformanceNavigationTiming.type === 'back_forward'. Treat notRestoredReasons
only as optional diagnostic evidence: allow it to be absent, unsupported, or
null, and avoid requiring stable reason text.
- Around line 265-271: The Cloudflare Durable Object cost entry must make the
~$47–94 per 1M views estimate reproducible. Expand the “Hosting cost”/related
cost analysis in this document with a small calculator table covering billed
duration per view, memory/concurrency or plan assumptions, free-tier/allowance
assumptions, and calculated monthly costs for the Worker, Durable Object, R2,
and CDN components as applicable; reconcile the stated $12.50/million GB-s and
128 MB memory with the 30–60-second duration range.
- Around line 194-200: Update the scroll-targeted section fetch guidance to pass
priority: 'high' in the fetch() RequestInit options, replacing fetchpriority:
'high'; do not apply this as an HTML element attribute.
- Around line 218-237: Update the two-phase delivery flow around
should_send_section? so each section’s confirmation decision is resolved before
any bytes of that complete section are written; if confirmation is uncertain,
send the entire section rather than truncating or cancelling it. Do not assume
JavaScript-capable bots never POST, and ensure BOT_TIMEOUT_SECONDS is
independent of any shorter upstream, proxy, or crawler timeout.

---

Nitpick comments:
In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md`:
- Line 124: Add an appropriate language identifier to every fenced code block in
the document, specifically the fences around lines 124, 220, and 397; use text
for diagram or plain-content fences, or the accurate language where applicable,
so all blocks satisfy markdownlint MD040.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: cadc5b48-1b8d-4b34-ba2a-2c909a17e801

📥 Commits

Reviewing files that changed from the base of the PR and between cf39d07 and 600313b.

📒 Files selected for processing (1)
  • internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md

Comment on lines +32 to +44
| Want | Requires |
| -------------------------- | -------------------------------------------------------- |
| Zero duplicate bytes | Someone must know what was already delivered and skip it |
| CDN-served (static replay) | No per-request state, no skip logic |
| First-visit compatible | No Service Worker |
| Bot-safe | Stream must deliver all content |

**Zero duplicates + CDN-served + first-visit + bot-safe are mutually exclusive.** No architecture satisfies all four. You must relax one:

- Relax "zero duplicates" → **C4** (accept 21-35KB of harmless duplicate bytes) ← recommended
- Relax "CDN-served" → **Server-side section skipping** (origin holds stream, skips confirmed sections)
- Relax "no hosting cost" → **Cloudflare DO** (stateful edge, ~$47-94/mo per 1M views)
- Relax "bot-safe" → C6 Server Islands (bots see skeletons) ← unacceptable

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

Limit the four-way exclusion to pure static replay.

The document defines CDN delivery as having no per-request state, but later calls the stateful Durable Object “the CDN.” Under the broader meaning of CDN delivery, C1 is a counterexample to the stated mutual exclusion. Rename the constraint to pure static CDN replay and state which property C1 relaxes.

Also applies to: 269-271

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 32 - 44, Update the four-way exclusion in the document to apply
specifically to “pure static CDN replay,” including the related occurrence near
the C1 discussion. Clarify that C1 uses stateful edge delivery via a Durable
Object and therefore relaxes the no-per-request-state or pure-static-replay
constraint, while preserving the existing tradeoff analysis.

| Navigation API `intercept()` | Cannot intercept the initial page load — only subsequent same-origin navigations |
| SW with `skipWaiting` + `clients.claim` | SW cannot intercept the navigation that registered it — fundamental to the SW lifecycle spec |
| ReadableStream `tee()` | Requires SW to access navigation response; without SW the response is inaccessible |
| SharedWorker | Safari removed SharedWorker in 2015, no re-add planned — blocks 15-25% of web traffic |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file =="
git ls-files | rg 'internal/analysis/scroll-priority-candidate-evaluation-2026-08-02\.md$' || true

echo "== file excerpt =="
if [ -f internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md ]; then
  nl -ba internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md | sed -n '55,80p'
fi

echo "== search SharedWorker mentions =="
rg -n "SharedWorker|safari|safari 16|shared worker|x-origin|15-25" internal internal/analysis/scroll-priority-candidate-evaluation-2025-08-02.md internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md 2>/dev/null || true

Repository: shakacode/react_on_rails

Length of output: 1934


🌐 Web query:

WebKit SharedWorker Safari 16 support

💡 Result:

Safari 16 introduced official support for the SharedWorker API [1][2]. This feature, which allows JavaScript to run in the background and communicate across multiple tabs or windows sharing the same origin, was highlighted as a key addition in the Safari 16.0 release [3][2]. The implementation enables a SharedWorker to remain active as long as at least one tab or window remains open for a given domain, similar in concept to Service Workers but with a distinct lifetime model [3][2]. Before this release, Shared Workers were not supported in Safari (specifically, they were unsupported from Safari 7 through 15.6) [4][5]. Following its introduction in version 16.0, support has been maintained in subsequent versions of Safari [4][5].

Citations:


🌐 Web query:

MDN SharedWorker browser compatibility Safari

💡 Result:

As of August 3, 2026, the SharedWorker API is supported in Safari starting from version 16.0 [1][2][3]. Key details regarding Safari support for SharedWorker: - Historical Context: WebKit originally provided support for SharedWorker in Safari 5 and 6, but this support was removed in Safari 6.1 [3]. - Modern Support: Support was reintroduced in Safari 16.0 and has been maintained in all subsequent versions [1][2]. - Cross-Platform: This support applies to both macOS Safari and iOS Safari (iPhone/iPad) from version 16.0 onwards [3]. For environments that must support older versions of Safari (e.g., Safari 15.6 or earlier, such as those on macOS Monterey or older iOS versions), the SharedWorker constructor will be undefined [3]. In such cases, developers often use a polyfill or fall back to alternative patterns, such as using Dedicated Workers in combination with the BroadcastChannel API for cross-context communication [3].

Citations:


Update the SharedWorker compatibility claim.

The “Safari removed SharedWorker in 2015, no re-add planned” row is stale. Safari has SharedWorker support from Safari 16, so this should not be used to rule out SharedWorker as an implementation path. Update the claim, compatibility check, and traffic-share analysis before merging this row.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` at line
70, Update the SharedWorker row in the compatibility analysis to reflect Safari
16 support, removing the outdated claim that Safari lacks SharedWorker and has
no plans to restore it. Reassess the compatibility conclusion and revise the
associated 15–25% traffic-share impact so SharedWorker is not ruled out on stale
information.

Source: MCP tools

Comment on lines +76 to +81
| **C3: `window.stop()` + client pull** | `window.stop()` kills ALL in-flight network activity including for JS-executing bots (Googlebot renderer) |
| **C6: Pull-only tail (Server Islands)** | No JS = permanent skeleton placeholders. AI crawlers (GPTBot, ClaudeBot, PerplexityBot) do not render JS at all. |
| SSE replacing HTML stream | Bots request `text/html`; SSE returns `text/event-stream` — no renderable content |
| WebSocket replacing HTML | Bots do not perform WebSocket protocol upgrade |
| Turbo Frames / htmx lazy fragments | Without JS, lazy fragments never fetch; permanent empty placeholders |
| Range request after `window.stop()` | Requires `window.stop()` (same problem); also incompatible with chunked transfer |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

MDN Window.stop() stops loading of page and stops further resource loading in the browsing context

💡 Result:

The window.stop method stops further resource loading in the current browsing context, functioning exactly like the Stop button in a web browser [1][2][3]. Key details about this method include: - Functionality: It terminates the loading of ongoing resources such as images, new windows (including iframes or frames), and scripts that have been deferred [1][4][3]. - Execution Limitations: Because the method is called by a script already running within the document, it cannot stop the loading of the parent document that contains the script itself [1][3]. Instead, it effectively interrupts resources that are still in the process of loading when the method is invoked [1][3]. - Return Value: The method returns undefined [1][2]. - Practical Application: It is often used to programmatically halt background loading tasks, such as clearing heavy image requests or preventing further network activity before initiating a new action like a page reload [4][3].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the reviewed document and any nearby references to C3/window.stop.
file="internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md"
if [ -f "$file" ]; then
  printf '--- referenced lines ---\n'
  sed -n '60,90p' "$file"
  printf '\n--- all C3/window.stop mentions ---\n'
  rg -n "C3|window\.stop|C3:" "$file" || true
else
  printf 'file not found: %s\n' "$file"
fi

Repository: shakacode/react_on_rails

Length of output: 1934


Do not describe window.stop() as cancelling all network activity.

window.stop() stops further resource loading in the browsing context, but it does not establish cancellation for every in-flight fetch() or XHR request started by scripts. Keep the verification scope tied to the rendered page/document context or cite observed behavior.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 76 - 81, Revise the C3 entry to remove the claim that window.stop()
cancels all in-flight network activity, and describe only its effect on further
resource loading in the browsing context. Keep the verification focused on
rendered page/document behavior or cite observed behavior for any fetch/XHR
effects.

Source: MCP tools

Comment on lines +154 to +162
## 4. Surviving Approaches (Ranked)

Five architectures pass both C1-FIRST and C2-BOT. They decompose into four fundamental mechanisms.

### Rank 1: C4 — Client Fetch + `$RC` While Stream Continues

**Status:** Already built and working (`?mode=fetch`).

IntersectionObserver detects scroll to a skeleton → `fetch()` pulls the cached section file → `DOMParser` + `adoptNode` + `$RC()` reveals it immediately → stream continues delivering all sections for bots. `$RC` idempotency handles the duplicate harmlessly.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Resolve the candidate-count inconsistency.

Section 4 says five architectures pass both constraints, but Section 5.4 lists only C1, C4, and C5 as surviving original candidates. Rank 2 and Rank 3 are C4 variants, not candidates from the six-candidate matrix. Label these as ranked deployment variants or change the count to three original candidates.

Also applies to: 324-333

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 154 - 162, The document has an inconsistent survivor count: Section 4
claims five architectures, while Section 5.4 identifies only three original
candidates and two C4 variants. Update the relevant sections to distinguish the
three original candidates from ranked C4 deployment variants, or revise the
count to three original candidates, preserving the intended ranking.

Comment on lines +194 to +200
**Recommended improvements to the existing prototype:**

1. Lower IntersectionObserver threshold with `rootMargin: '0px 0px 200px 0px'` (prefetch 200px before visible)
2. `AbortController` per fetch — abort in-flight fetch if stream delivers first; abort all in `pagehide` for bfcache
3. Prefetch N+1 when section N becomes visible (via `requestIdleCallback`)
4. `fetchpriority: 'high'` on scroll-targeted section fetches

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate and inspect the referenced file and surrounding lines.
if [ -f "internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md" ]; then
  sed -n '180,210p' internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md | cat -n
else
  echo "Referenced file not found"
fi

# Search for fetchpriority/fetchpriority usage in tracked files.
printf '\nRelevant fetchpriority occurrences:\n'
rg -n "fetchpriority|fetch\(|RequestInit|priority:" internal || true

Repository: shakacode/react_on_rails

Length of output: 1934


🌐 Web query:

MDN RequestInit priority fetchpriority attribute fetch() high priority

💡 Result:

The Fetch API's RequestInit object includes a priority property, which allows developers to provide a hint to the browser regarding the relative priority of a network request [1][2]. This property is part of the Priority Hints specification [2]. Key details include: Values: The property accepts three possible string values: - high: Signals that the request is important and should be prioritized over others [1][2]. - low: Signals that the request is less important and can be deprioritized [1][2]. - auto: The default value; the browser uses its own internal heuristics to determine priority [1][2]. Behavior: The priority property acts as a hint to the browser's fetch scheduler [2]. It is used to influence the request's priority relative to other requests, but the actual impact remains browser-dependent [3][2]. Because it is a hint, it is safe to use even in environments where it might be ignored; browsers that do not support or choose to ignore the hint will simply default to their standard scheduling behavior [4]. Usage: The priority is specified when constructing a request, such as in the fetch call: fetch('https://api.example.com/data', { priority: 'high' }); Browser Support: As of August 2026, the priority option for fetch is supported in major modern browsers, including Chrome/Edge (since version 101), Safari (since 17.2), and Firefox (since 132) [5][4]. The terminology differs slightly between the JS API and HTML: while the HTML attribute used on elements like or <script> is called fetchpriority, the corresponding property in the JavaScript RequestInit dictionary is simply named priority [2][6].

Citations:


Use priority: 'high' for the fetch() hint.

fetchpriority is an HTML element attribute; priority is the RequestInit option for controlling fetch scheduling.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 194 - 200, Update the scroll-targeted section fetch guidance to pass
priority: 'high' in the fetch() RequestInit options, replacing fetchpriority:
'high'; do not apply this as an HTML element attribute.

Source: MCP tools

Comment on lines +265 to +271
| Property | Value |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Duplicate bandwidth | Zero |
| Server changes | Significant — Worker + DO + R2 + wrangler.toml |
| CDN compatible | ✅ (it IS the CDN) — **Cloudflare only** |
| Hosting cost | **~$47-94/mo per 1M views** (DO duration billing, cannot hibernate during streaming) |
| Key tradeoff | Hard vendor lock-in. Verified: no other edge platform can replicate this (Fastly, Lambda@Edge, Akamai EdgeWorkers, Deno Deploy, Vercel — all lack cross-request state coordination). |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file =="
git ls-files | rg 'internal/analysis/scroll-priority-candidate-evaluation-2026-08-02\.md$' || true

echo "== relevant lines =="
if [ -f internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md ]; then
  sed -n '220,300p' internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md | cat -n
fi

echo "== pricing/cost references in doc =="
rg -n "([Dd]urable [Oo]bject|DO|\\$[0-9]+|cost|pricing|GB|views|duration|R2|Worker|Cloudflare)" internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md || true

echo "== git diff stat/name-only if available =="
git diff --stat || true
git diff -- internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md | sed -n '1,220p' || true

echo "== deterministic cost calc around 30-60s duration assumptions =="
python3 - <<'PY'
price_per_million_gb_s = 12.50
mem_gb = 128/1024
views = 1_000_000
for sec in [20,30,40,60]:
    gb_s = views * sec * mem_gb
    billed = max(0, gb_s - 1_000_000)
    total = price_per_million_gb_s * billed / 1_000_000
    print(f"{sec}s -> duration_gb_s={gb_s:.2f}, billed_overage_gb_s={billed:.2f}, cost=${total:.2f}")
PY

Repository: shakacode/react_on_rails

Length of output: 1934


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file =="
git ls-files | rg 'internal/analysis/scroll-priority-candidate-evaluation-2026-08-02\.md$' || true

echo "== relevant lines =="
if [ -f internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md ]; then
  sed -n '220,300p' internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md | cat -n
fi

echo "== pricing/cost references in doc =="
rg -n "([Dd]urable [Oo]bject|DO|\\$[0-9]+|cost|pricing|GB|views|duration|R2|Worker|Cloudflare)" internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md || true

echo "== deterministic cost calc around 30-60s duration assumptions =="
python3 - <<'PY'
price_per_million_gb_s = 12.50
mem_gb = 128/1024
views = 1_000_000
for sec in [20,30,40,60]:
    gb_s = views * sec * mem_gb
    billed = max(0, gb_s - 1_000_000)
    total = price_per_million_gb_s * billed / 1_000_000
    print(f"{sec}s -> duration_gb_s={gb_s:.2f}, billed_overage_gb_s={billed:.2f}, cost=${total:.2f}")
PY

Repository: shakacode/react_on_rails

Length of output: 1934


Make the Durable Object cost estimate reproducible.

The Cloudflare Durable Object row should include the billed duration assumptions needed to derive the $47–94 per 1M views range. The stated bandwidth ($12.50/million GB-s) and 128 MB memory imply ~30–60 seconds of billed duration per view if this is a standalone duration estimate, but the entry also lists Worker, DO, R2, and CDN items. Add a small calculator table showing duration, concurrency/plans, free-tier/allowance assumptions, and the resulting monthly costs.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 265 - 271, The Cloudflare Durable Object cost entry must make the ~$47–94
per 1M views estimate reproducible. Expand the “Hosting cost”/related cost
analysis in this document with a small calculator table covering billed duration
per view, memory/concurrency or plan assumptions, free-tier/allowance
assumptions, and calculated monthly costs for the Worker, Durable Object, R2,
and CDN components as applicable; reconcile the stated $12.50/million GB-s and
128 MB memory with the 30–60-second duration range.

Source: MCP tools

Comment on lines +273 to +285
### Rank 5: React PPR Single-Response Variant

**Status:** Blocked on React. `React.postpone()` still `unstable_`. 21-31 days effort when unblocked.

`prerenderToNodeStream` produces shell + postponed state → `resumeToPipeableStream` on the same HTTP response. Scroll priority controls which section's data resolves first during resume (data-resolution order = emission order, confirmed).

| Property | Value |
| ------------------- | -------------------------------------------------------------------------------------- |
| Duplicate bandwidth | Zero |
| Server changes | Significant — 6+ files, PostponedStateStore, new renderer endpoints |
| CDN compatible | Partial — single-response variant streams from origin; loses the CDN-caching advantage |
| React upgrade risk | **High** — `unstable_postpone()`, opaque version-specific postponed state |
| Key tradeoff | Framework-sanctioned long-term path, but premature to build on unstable APIs. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== file context =="
sed -n '240,295p' internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md || true

echo
echo "== nearby references to PPR/resume/scroll priority =="
rg -n "postpone|resume|PPR|Scroll priority|resumeToPipeableStream|prerenderToNodeStream|data-resolution|same HTTP|same response|scroll signal|abort" internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md || true

echo
echo "== repository references to postponed/stream APIs =="
rg -n "postpone|resumeToPipeableStream|prerenderToNodeStream|postponed state|unstable_postpone" . || true

Repository: shakacode/react_on_rails

Length of output: 1934


🌐 Web query:

React server components prerenderToNodeStream postponed state unstable_postpone resumeToPipeableStream data resolution order

💡 Result:

The experimental APIs unstable_postpone, prerenderToNodeStream, and resumeToPipeableStream are central to Partial Prerendering (PPR) in React [1][2]. 1. unstable_postpone: This API is called by a component (often triggered by dynamic functions like cookies or headers) to signal that it cannot be fully prerendered at build time [1][2]. It creates a dynamic hole in the static shell, allowing React to render a Suspense fallback instead of blocking the entire page render [1][2]. 2. prerenderToNodeStream: Used during the build/prerendering phase, this function generates the static shell of the application [3]. If it encounters a component that calls unstable_postpone, the rendering process for that branch is halted, and it returns an opaque postponed state object [3]. This object represents the incomplete state of the component tree and is serializable for later use [3]. 3. resumeToPipeableStream: During the request-time phase, this API takes the previously generated postponed state and resumes rendering from the point where it was halted [4][1]. It does not re-render the entire static shell; instead, it renders only the dynamic parts ("the holes") and streams the resolved content to the client to fill those placeholders [4][1]. 4. Data Resolution Order: React’s streaming architecture is designed to prioritize completion over document order [5][6]. When React streams content via Suspense boundaries, it does not wait for components to resolve in the order they appear in the DOM [5][7]. Instead, each Suspense boundary acts as an independent streaming unit [5][8]. Components resolve and stream as soon as their data is available, meaning faster segments can be painted to the browser before slower, higher-priority-looking segments [5][6]. React handles the reordering on the client-side by using inline scripts and hidden DOM placeholders (e.g., <div hidden id="S:1">) to swap the fallback content with the resolved HTML once the server delivers the chunk [7][6][8]. These APIs currently require specific experimental builds of React (e.g., with enableHalt=true enabled) as they are not available in public stable releases [1][2].

Citations:


Validate the PPR priority mechanism before calling it confirmed.

Official React docs describe prerenderToNodeStream returning opaque postponed state that resumeToPipeableStream resumes, not scrolling as a control for rendering order. The docs also do not document keeping the initial prerender response open or routing client scroll signals into the same server render. Mark this as a hypothesis until a prototype proves ordering, abort behavior, and same-response streaming.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 273 - 285, Update the “Rank 5: React PPR Single-Response Variant” section
to describe scroll-controlled data-resolution order as a hypothesis rather than
confirmed behavior. Remove or qualify the “confirmed” claim and document that a
prototype must validate ordering, abort behavior, and same-response streaming
before treating the mechanism as viable.

Source: MCP tools

Comment on lines +337 to +344
| Criterion | C4 (recommended) |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Strict CSP | ✅ — `fetchAndRevealSection` uses DOM manipulation, never executes fetched scripts. Existing comment: "We NEVER execute the fetched scripts (their cached CSP nonces are stale anyway)" (line 257). |
| RSC compatibility | ✅ — Stream continues; `injectRSCPayload` pipeline delivers payload normally. Fetched sections' stale RSC payload scripts are harmless duplicates. |
| Store hydration | ✅ — Stores in chunk 0 (shell), initialized before scroll triggers. Guard in `initializeStore` prevents re-initialization. |
| SEO | ✅ — Stream delivers all content. Fetch path invisible to crawlers. |
| bfcache | ✅ with mitigation — abort outstanding fetches in `pagehide` handler. |
| Scenario B (origin rendering) | ✅ — Complementary to skip-delay POST. Server releases section sooner, reducing duplicate window. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file =="
git ls-files | rg '(^|/)scroll-priority-candidate-evaluation-2026-08-02\.md$|internal/analysis' || true

echo "== relevant markdown section =="
if [ -f internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md ]; then
  nl -ba internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md | sed -n '1,420p' | rg -n "RSC|fetched|fetchAndRevealSection|injectRSCPayload|initializeStore|Scenario|bfcache|Client|client" -C 3 || true
fi

echo "== search related identifiers in repository =="
rg -n "fetchAndRevealSection|injectRSCPayload|initializeStore|fetched sections|RSC payload|stale|RSC" . --glob '!**/node_modules/**' --glob '!**/.git/**' || true

Repository: shakacode/react_on_rails

Length of output: 1934


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== pwd =="
pwd

echo "== top-level entries =="
ls -la . | sed -n '1,80p'

echo "== git status summary =="
git status --short --branch | sed -n '1,40p' || true

echo "== locate file =="
git ls-files 'internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md' || true

echo "== relevant markdown section =="
if [ -f internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md ]; then
  nl -ba internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md | sed -n '1,420p' | rg -n "RSC|fetched|fetchAndRevealSection|injectRSCPayload|initializeStore|Scenario|bfcache|Client|client" -C 3 || true
fi

echo "== search related identifiers in repository =="
rg -n "fetchAndRevealSection|injectRSCPayload|initializeStore|fetched sections|RSC payload|stale|RSC|scroll" . --glob '!**/node_modules/**' --glob '!**/.git/**' || true

Repository: shakacode/react_on_rails

Length of output: 1934


Recheck the fetched lazy-load path against RSC client-component behavior.

The evaluated path parses HTML without executing fetched scripts. If fetched sections may contain client components, the stream payload must cover client-component references, hydration data, actions, and event handlers before adoption. Add a client-component fixture inside a fetched section, or restrict this path to server-only sections.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 337 - 344, The C4 evaluation must verify client-component support in the
fetched lazy-load path before retaining its compatibility claim. Update the
evaluation around fetchAndRevealSection to add a fetched-section
client-component fixture and confirm RSC references, hydration data, actions,
and event handlers work after adoption; alternatively, explicitly restrict the
path to server-only sections and revise the affected compatibility claims.

Comment on lines +383 to +385
### 7.3 SEO

C4 is inherently SEO-safe: the stream delivers ALL content in the HTML response. The fetch path is purely additive and invisible to crawlers. AI crawlers (GPTBot, ClaudeBot, PerplexityBot) do not render JS at all (Vercel confirmed across 1B+ monthly bot requests). Googlebot's render phase may take hours/days after initial crawl. All SEO-critical content is in the synchronous SSR output.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate file =="
git ls-files | grep 'internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md' || true

echo "== file size =="
wc -l internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md

echo "== relevant sections =="
sed -n '350,400p' internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md
echo "---"
sed -n '510,560p' internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md

echo "== external references near section =="
python3 - <<'PY'
from pathlib import Path
p=Path('internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md')
lines=p.read_text().splitlines()
for i,l in enumerate(lines,1):
    if 'Lantern' in l or 'asklantern' in l.lower() or '1B+ monthly' in l or 'Vercel' in l or 'AI crawlers' in l or 'Official Documentation' in l:
        print(f"{i}: {l}")
PY

echo "== search for Lantern/Vercel citations =="
rg -n "asklantern|Lantern|Vercel|1B|bot requests|AI crawlers|Official Documentation" internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md

Repository: shakacode/react_on_rails

Length of output: 1934


🌐 Web query:

Vercel blog The rise of the AI crawler traffic sample crawler set date

💡 Result:

The Vercel blog post titled "The rise of the AI crawler" was published on December 17, 2024 [1][2]. This research analyzed traffic patterns across the Vercel network to understand the impact and behavior of AI crawlers [1]. Key findings from the report include: - Traffic Volume: During the period analyzed, major AI crawlers (including GPTBot, Claude, AppleBot, and PerplexityBot) combined for nearly 1.3 billion fetches, representing approximately 28% of the 4.5 billion requests generated by Googlebot [1][2]. - JavaScript Rendering: A central conclusion of the report is that major AI crawlers do not execute JavaScript [1][3]. Because they only process the initial HTML response, content that relies on client-side rendering (CSR) remains largely invisible to these systems [3][4]. - Crawler Efficiency: The analysis noted that AI crawlers were relatively inefficient, with a significant portion of their traffic (over 30%) resulting in 404 errors [5][2]. The report emphasizes that for content to be effectively discovered by AI-driven search and answer engines, developers should utilize server-side rendering (SSR), static site generation (SSG), or incremental static regeneration (ISR) to ensure static HTML is available to these bots [3].

Citations:


Correct the source classification and attribution.

This section is “Accepted From Official Documentation,” but the AI-crawler row points to asklantern.com while the text says “Vercel confirmed.” Use the official Vercel study link and replace the undocumented “1B+ monthly bot requests” claim with the study’s measured scope and published date.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 383 - 385, Update the SEO section’s source classification and attribution:
replace the asklantern.com AI-crawler reference with the official Vercel study
link, and revise the “1B+ monthly bot requests” statement to match the study’s
measured scope and publication date.

Source: MCP tools

Comment on lines +484 to +485
**P2:** `AbortController` + `pagehide` lifecycle.
Pass/fail: `notRestoredReasons` API does NOT report `OutstandingNetworkRequestFetch` after navigation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Locate and inspect the referenced evaluation document around the cited lines.
if [ -f "internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md" ]; then
  wc -l internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md
  sed -n '450,510p' internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md
else
  echo "Referenced file not found"
fi

echo "--- relevant references in repository ---"
rg -n "performance\.navigationTiming|type === 'back_forward'|notRestoredReasons|pageshow|persisted|OutstandingNetworkRequestFetch|bfcache" internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md 2>/dev/null || true

Repository: shakacode/react_on_rails

Length of output: 1934


🌐 Web query:

MDN PerformanceNavigationTiming type back_forward pageshow persisted notRestoredReasons notRestoredReasons supported

💡 Result:

The PerformanceNavigationTiming interface provides timing metrics for document navigation [1]. Within this interface, the type property categorizes the navigation, with "back_forward" specifically indicating that the navigation was performed via a history traversal operation [2][3]. To monitor the back/forward cache (bfcache), developers should use the notRestoredReasons property, which returns a NotRestoredReasons object detailing why a document was blocked from using the bfcache [4][5]. If a document was successfully restored from the bfcache, notRestoredReasons will typically be null, and you should instead observe the pageshow event [5][6]. The pageshow event fires both when a page is initially loaded and when it is restored from the bfcache [6]. To distinguish between these, you can check the event.persisted property: if it is true, the page was restored from the bfcache [6]. Regarding browser support, the notRestoredReasons property is not universally supported across all browsers [7][8]. It is supported in modern versions of Chromium-based browsers (such as Chrome 125+) [9][8]. It is recommended to check for its existence before usage, as it returns null for non-history navigations and would return undefined if the browser does not support the API [5]. Historically, developers attempted to use the deprecated PerformanceNavigation.type property to infer bfcache usage, but that method did not provide specific blocking reasons [4].

Citations:


Use bfcache-specific signals in the acceptance criteria.

PerformanceNavigationTiming.type === 'back_forward' proves history traversal, not bfcache restoration. Use pageshow with event.persisted for restore detection. Treat notRestoredReasons as diagnostic: it can be absent, unsupported, or null, and reason text can change.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/analysis/scroll-priority-candidate-evaluation-2026-08-02.md` around
lines 484 - 485, Update the acceptance criteria in the AbortController/pagehide
lifecycle section to detect bfcache restoration with pageshow and
event.persisted, rather than relying on PerformanceNavigationTiming.type ===
'back_forward'. Treat notRestoredReasons only as optional diagnostic evidence:
allow it to be absent, unsupported, or null, and avoid requiring stable reason
text.

Source: MCP tools

@AbanoubGhadban
AbanoubGhadban merged commit a0d8dc0 into main Aug 3, 2026
75 checks passed
@AbanoubGhadban
AbanoubGhadban deleted the scroll-priority-evaluation-4835 branch August 3, 2026 12:22
justin808 added a commit that referenced this pull request Aug 3, 2026
…kaperf-evidence

* origin/main:
  Docs: add missing content — release notes, upgrade guide, config, API references (#4843) (#4844)
  Fix incorrect docs: helper names, defaults, requirements, runtime refs (#4836)
  Scroll-priority streaming: candidate architecture evaluation (#4835) (#4841)
  Fix generated server config lint cleanup (#4840)

# Conflicts:
#	CHANGELOG.md
justin808 added a commit that referenced this pull request Aug 6, 2026
…out-vm-pool

* origin/main:
  Docs: move agent coordination to the HTTP backend (#4764)
  Detect unnoticed changes across generated webpack/Rspack configs (#4839)
  Fix durable ShakaPerf release evidence reuse (#4833)
  Docs: add missing content — release notes, upgrade guide, config, API references (#4843) (#4844)
  Fix incorrect docs: helper names, defaults, requirements, runtime refs (#4836)
  Scroll-priority streaming: candidate architecture evaluation (#4835) (#4841)
  Fix generated server config lint cleanup (#4840)
  Document serialized release backport policy (#4592)
  Package version-matched agent skills and docs (#4809)
  Fix ci-required base ref for PR merge-ref checkouts (#4819)
  Honor response charset and reject non-2xx HTTP-served SSR bundles (#4817)
  [Pro] Redact RSC render-error metadata on the fetched (client-navigation) payload path (#4821)
  Forward-port the 17.0.1 changelog section to main (#4814)

# Conflicts:
#	CHANGELOG.md
justin808 added a commit that referenced this pull request Aug 8, 2026
…ential-broker

* origin/main: (75 commits)
  Silence routine startup diagnostics for Rails commands (#4849)
  Docs: move agent coordination to the HTTP backend (#4764)
  Detect unnoticed changes across generated webpack/Rspack configs (#4839)
  Fix durable ShakaPerf release evidence reuse (#4833)
  Docs: add missing content — release notes, upgrade guide, config, API references (#4843) (#4844)
  Fix incorrect docs: helper names, defaults, requirements, runtime refs (#4836)
  Scroll-priority streaming: candidate architecture evaluation (#4835) (#4841)
  Fix generated server config lint cleanup (#4840)
  Document serialized release backport policy (#4592)
  Package version-matched agent skills and docs (#4809)
  Fix ci-required base ref for PR merge-ref checkouts (#4819)
  Honor response charset and reject non-2xx HTTP-served SSR bundles (#4817)
  [Pro] Redact RSC render-error metadata on the fetched (client-navigation) payload path (#4821)
  Forward-port the 17.0.1 changelog section to main (#4814)
  Handle selector metacharacters in renderComponent DOM IDs (#4808)
  [Pro] Prevent caching RSC renders with errors (#4804)
  Agents: trust Copilot review identities (#4807)
  Agents: bind fleet closeout to generated pack (#4805)
  Docs: ADR 0002 — Skills-in-package over MCP for agent-native DX (#4735)
  Scope GitHub release commands to the origin repository (#4803)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluate scroll-priority streaming candidate architectures (feasibility, pros/cons) from #4769

1 participant