Skip to content

[ci] Share a single browser instance across all test suites in a single job#95589

Merged
bgw merged 1 commit into
canaryfrom
bgw/shared-browser-process
Jul 21, 2026
Merged

[ci] Share a single browser instance across all test suites in a single job#95589
bgw merged 1 commit into
canaryfrom
bgw/shared-browser-process

Conversation

@bgw

@bgw bgw commented Jul 7, 2026

Copy link
Copy Markdown
Member

The idea here is that we waste a bunch of CPU setting up each test suite by spawning an entirely new Chrome process, but playwright can share the same browser across many tests. Tests are still isolated at the browser level, just not at the OS process level.

Hoping to see some performance improvement in e2e tests. Compare to: #95617

Somewhat unscientific (single sample) results:

Comparing the head-commit build-and-test runs — PR #95589 run [28981702219 ](https://github.com/vercel/next.js/actions/runs/28981702219)vs PR #95617 run [28981686692](https://github.com/vercel/next.js/actions/runs/28981686692). Both ran 101 jobs and both succeeded.

  ┌─────────────────────────┬────────────────┬───────────────────────┬───────────────────┐
  │         Metric          │ #95617 control │ #95589 shared browser │       Delta       │
  ├─────────────────────────┼────────────────┼───────────────────────┼───────────────────┤
  │ Raw wall-time sum       │ 616.2 min      │ 571.2 min             │ −45.0 min (−7.3%) │
  ├─────────────────────────┼────────────────┼───────────────────────┼───────────────────┤
  │ Billable (ceil per job) │ 658 min        │ 617 min               │ −41 min (−6.2%)   │
  └─────────────────────────┴────────────────┴───────────────────────┴───────────────────┘

  The shared-browser experiment is cheaper, and the savings land almost exactly where you'd expect — the browser-driven test suites — while non-browser jobs (rust, lint, unit, windows) are flat within noise:

  ┌─────────────────────────────────┬─────────┬────────┬───────┐
  │          Job category           │ control │ shared │ delta │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ test prod                       │   130.7 │  110.5 │ −20.2 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ cache components dev            │    51.5 │   42.9 │  −8.7 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ turbopack dev                   │    75.1 │   69.5 │  −5.5 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ turbopack production            │    86.3 │   81.1 │  −5.2 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ test dev                        │   110.4 │  105.5 │  −4.9 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ cache components prod           │    48.7 │   49.0 │  +0.3 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ firefox and safari              │     5.3 │    7.3 │  +2.0 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ flake-detection jobs (combined) │     ~15 │    ~21 │   +~6 │
  └─────────────────────────────────┴─────────┴────────┴───────┘

  Takeaways:
  - Net savings of ~41 billable minutes per run (~6%), concentrated in the Chromium-driven test prod/dev, turbopack, and cache components suites — consistent with reusing one browser process instead of spawning per-suite.
  - The small regressions are in firefox and safari and the "new/changed tests for flakes" jobs (+~8 min combined). Worth a glance, though they may just be run-to-run variance.

  Caveat: this is a single run per PR, so there's real variance run-to-run. That said, the fact that the deltas track the browser-heavy jobs specifically — and not the Rust/lint/unit jobs — is a good signal the effect is genuine rather than noise. If you want a firmer number, re-running each PR 2–3× and averaging would tighten it up.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Stats from current PR

🔴 1 regression

Metric Canary PR Change Trend
Turbo Build Time (cached) 5.128s 5.619s 🔴 +491ms (+10%) ▁▂▄▁▁
📊 All Metrics
📖 Metrics Glossary

Dev Server Metrics:

  • Listen = TCP port starts accepting connections
  • First Request = HTTP server returns successful response
  • Cold = Fresh build (no cache)
  • Warm = With cached build artifacts

Build Metrics:

  • Fresh = Clean build (no .next directory)
  • Cached = With existing .next directory

Change Thresholds:

  • Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
  • Size: Changes < 1KB AND < 1% are insignificant
  • All other changes are flagged to catch regressions

⚡ Dev Server

Metric Canary PR Change Trend
Cold (Listen) 770ms 812ms ▂▂▂▂▁
Cold (Ready in log) 768ms 793ms ▁▁▂▁▁
Cold (First Request) 1.281s 1.340s ▁▁▂▁▁
Warm (Listen) 812ms 813ms ▂▂▂▂▁
Warm (Ready in log) 776ms 776ms ▂▁▂▁▁
Warm (First Request) 1.296s 1.299s ▂▁▁▁▁
📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric Canary PR Change Trend
Cold (Listen) 813ms 813ms █▂▂▄▂
Cold (Ready in log) 796ms 786ms █▁▂▃▁
Cold (First Request) 3.512s 3.604s █▁▁▄▁
Warm (Listen) 814ms 813ms █▂▂▂▁
Warm (Ready in log) 805ms 785ms █▁▂▂▁
Warm (First Request) 3.605s 3.521s █▁▁▃▁

⚡ Production Builds

Metric Canary PR Change Trend
Fresh Build 5.057s 5.180s ▂▁▃▂▁
Cached Build 5.128s 5.619s 🔴 +491ms (+10%) ▁▂▄▁▁
📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric Canary PR Change Trend
Fresh Build 25.608s 25.995s █▁▂▃▁
Cached Build 26.064s 26.089s █▁▁▃▁
node_modules Size 517 MB 517 MB █████
📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles
Canary PR Change
01njvfbkgn2hi.js gzip 166 B N/A -
03gmyxr7dukvs.js gzip 10.3 kB N/A -
040t1q17lo593.js gzip 8.77 kB N/A -
0bb0vtp8dwsez.js gzip 450 B N/A -
0cz1d0mv5g_q7.js gzip 39.4 kB 39.4 kB
0kb7gdprkm5-s.js gzip 153 B N/A -
0l0sr6gau5xjn.js gzip 3.53 kB N/A -
0nxqz3q165m2l.js gzip 220 B N/A -
0oygs1fnokdzg.js gzip 153 B N/A -
0qr-1tnb7_hg5.js gzip 5.72 kB N/A -
0uilzvklme4ea.js gzip 8.76 kB N/A -
0wmscwh7_m_n7.js gzip 156 B N/A -
0z0y83mfx-l-h.js gzip 10 kB N/A -
14ob2gnup7hmq.js gzip 154 B N/A -
15ok8ydf274z0.js gzip 10.6 kB N/A -
16cqqcigykfea.js gzip 13.6 kB N/A -
17h18zxt1915l.js gzip 1.46 kB N/A -
1cnwkx9jenjj4.js gzip 65.6 kB N/A -
1elt1qium-r2m.css gzip 115 B 115 B
1fq1h7dxad4cv.js gzip 9.44 kB N/A -
1fym-adbk_69_.js gzip 13.1 kB N/A -
1gav7nfh4y8ln.js gzip 8.76 kB N/A -
1jk3928phpavq.js gzip 154 B N/A -
1o34vh44ndvtu.js gzip 13.2 kB N/A -
23pnldcl3-q5n.js gzip 155 B N/A -
2qzmpe44xqltw.js gzip 155 B N/A -
2rypqffktcxqh.js gzip 8.69 kB N/A -
2uqsizxtehgqf.js gzip 8.79 kB N/A -
2vzktdqwbijqc.js gzip 8.74 kB N/A -
30sv6wnb24ko7.js gzip 156 B N/A -
3bfy3uzdkvnyq.js gzip 154 B N/A -
3c244akvx_-be.js gzip 8.74 kB N/A -
3dmlemitx3ef3.js gzip 2.29 kB N/A -
3lk7xgw97hbb3.js gzip 7.4 kB N/A -
3o37r02i9tn88.js gzip 71 kB N/A -
3t19gbtd8i-0c.js gzip 159 B N/A -
3u1fm7t_ty27_.js gzip 159 B N/A -
3vfl1fxa4me-g.js gzip 45.2 kB N/A -
3vtsfutwdus1c.js gzip 151 B N/A -
4237vdj6eyslt.js gzip 8.7 kB N/A -
turbopack-04..ts1w.js gzip 3.8 kB N/A -
turbopack-0n..w_i9.js gzip 3.8 kB N/A -
turbopack-0y..tur0.js gzip 3.8 kB N/A -
turbopack-1e..8msr.js gzip 3.78 kB N/A -
turbopack-1l..wghi.js gzip 3.82 kB N/A -
turbopack-1z..he5j.js gzip 3.8 kB N/A -
turbopack-20..89v1.js gzip 3.8 kB N/A -
turbopack-2p..monf.js gzip 3.8 kB N/A -
turbopack-2s..tv4e.js gzip 3.8 kB N/A -
turbopack-2y..s9qo.js gzip 3.8 kB N/A -
turbopack-30..7wrv.js gzip 3.8 kB N/A -
turbopack-3d..sk8_.js gzip 3.8 kB N/A -
turbopack-3f..tcof.js gzip 3.8 kB N/A -
turbopack-3u..xlau.js gzip 3.8 kB N/A -
00tc16j5-eos9.js gzip N/A 1.46 kB -
02jud9pe218fc.js gzip N/A 7.4 kB -
09kr5li1x8s1m.js gzip N/A 65.6 kB -
0gaw1gmpwuwkn.js gzip N/A 8.8 kB -
0jb-4qq8ts8qg.js gzip N/A 154 B -
12n_-rsyhi_pr.js gzip N/A 155 B -
12zyjjmq4oad1.js gzip N/A 5.72 kB -
19v_2a9bzck7p.js gzip N/A 157 B -
1k9cc4-gpa2cx.js gzip N/A 13.1 kB -
1kfiwrcr725kl.js gzip N/A 13.2 kB -
1n59uogejo81s.js gzip N/A 10.3 kB -
1npei3hlexdv3.js gzip N/A 9.99 kB -
1pa59_fsyijjz.js gzip N/A 154 B -
1ssuiwj_rnpc5.js gzip N/A 8.77 kB -
1wz5t7p9xkdaw.js gzip N/A 156 B -
1xn24za9gg3uu.js gzip N/A 159 B -
1yq2lvpobwc8b.js gzip N/A 71 kB -
2-phc9obd3nyd.js gzip N/A 8.77 kB -
24utq57r3l1cw.js gzip N/A 8.7 kB -
254duj2pnel-n.js gzip N/A 154 B -
25o1d931t8jys.js gzip N/A 166 B -
2fhjq6ek3srmr.js gzip N/A 8.69 kB -
2g0t7upz_v-hf.js gzip N/A 10.6 kB -
2h5nkoal80pda.js gzip N/A 45.2 kB -
2ia9qqy99aie9.js gzip N/A 13.6 kB -
2oza28kdqvoay.js gzip N/A 8.74 kB -
2uu7sxn5cfgot.js gzip N/A 155 B -
2w2g5umt36xzs.js gzip N/A 151 B -
2xs3pwirxs0ct.js gzip N/A 161 B -
2zh8-mmhujy8o.js gzip N/A 2.29 kB -
30h_m0irj-q6p.js gzip N/A 221 B -
36lo4875c1prp.js gzip N/A 155 B -
37ejbfx1u9lpo.js gzip N/A 8.75 kB -
38c63ct4i1cy7.js gzip N/A 449 B -
3jimpal3lvz5e.js gzip N/A 9.44 kB -
3su6a4l8jbfth.js gzip N/A 3.53 kB -
3wb4pe4z4-aim.js gzip N/A 8.77 kB -
42km7qj0--1tt.js gzip N/A 157 B -
turbopack-08..pp0h.js gzip N/A 3.8 kB -
turbopack-09..q81z.js gzip N/A 3.8 kB -
turbopack-0c..gjco.js gzip N/A 3.8 kB -
turbopack-0w..9_5b.js gzip N/A 3.8 kB -
turbopack-16..04_v.js gzip N/A 3.8 kB -
turbopack-18..hutv.js gzip N/A 3.8 kB -
turbopack-1n..8t20.js gzip N/A 3.8 kB -
turbopack-1r..fikm.js gzip N/A 3.8 kB -
turbopack-2f..vy_q.js gzip N/A 3.81 kB -
turbopack-2v.._o-s.js gzip N/A 3.8 kB -
turbopack-2x..x68r.js gzip N/A 3.8 kB -
turbopack-2y..cw41.js gzip N/A 3.78 kB -
turbopack-3f..rvpv.js gzip N/A 3.8 kB -
turbopack-3s..bu-8.js gzip N/A 3.8 kB -
Total 448 kB 448 kB ⚠️ +62 B

Server

Middleware
Canary PR Change
middleware-b..fest.js gzip 781 B 771 B 🟢 10 B (-1%)
Total 781 B 771 B ✅ -10 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 434 B 433 B
Total 434 B 433 B ✅ -1 B

📦 Webpack

Client

Main Bundles
Canary PR Change
2486.HASH.js gzip 169 B N/A -
3146-HASH.js gzip 64.2 kB N/A -
39fcf99b-HASH.js gzip 62.8 kB N/A -
8443-HASH.js gzip 4.68 kB N/A -
9431-HASH.js gzip 5.62 kB N/A -
framework-HASH.js gzip 59.8 kB 59.8 kB
main-app-HASH.js gzip 255 B 254 B
main-HASH.js gzip 39.7 kB 40.1 kB 🔴 +452 B (+1%)
webpack-HASH.js gzip 1.68 kB 1.68 kB
6105-HASH.js gzip N/A 5.63 kB -
764.HASH.js gzip N/A 169 B -
8898-HASH.js gzip N/A 63.6 kB -
9597-HASH.js gzip N/A 4.65 kB -
e1ccab69-HASH.js gzip N/A 62.8 kB -
Total 239 kB 239 kB ✅ -208 B
Polyfills
Canary PR Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Total 39.4 kB 39.4 kB
Pages
Canary PR Change
_app-HASH.js gzip 194 B 194 B
_error-HASH.js gzip 183 B 182 B
css-HASH.js gzip 335 B 335 B
dynamic-HASH.js gzip 1.8 kB 1.8 kB
edge-ssr-HASH.js gzip 255 B 254 B
head-HASH.js gzip 351 B 349 B
hooks-HASH.js gzip 384 B 384 B
image-HASH.js gzip 580 B 580 B
index-HASH.js gzip 259 B 259 B
link-HASH.js gzip 2.49 kB 2.49 kB
routerDirect..HASH.js gzip 319 B 319 B
script-HASH.js gzip 386 B 386 B
withRouter-HASH.js gzip 315 B 313 B
1afbb74e6ecf..834.css gzip 106 B 106 B
Total 7.96 kB 7.95 kB ✅ -7 B

Server

Edge SSR
Canary PR Change
edge-ssr.js gzip 128 kB 128 kB
page.js gzip 283 kB 285 kB 🔴 +2.1 kB (+1%)
Total 411 kB 413 kB ⚠️ +1.77 kB
Middleware
Canary PR Change
middleware-b..fest.js gzip 615 B 617 B
middleware-r..fest.js gzip 156 B 155 B
middleware.js gzip 45.3 kB 45.3 kB
edge-runtime..pack.js gzip 842 B 842 B
Total 46.9 kB 46.9 kB ⚠️ +35 B
Build Details
Build Manifests
Canary PR Change
_buildManifest.js gzip 719 B 720 B
Total 719 B 720 B ⚠️ +1 B
Build Cache
Canary PR Change
0.pack gzip 4.82 MB 4.81 MB 🟢 11.2 kB (0%)
index.pack gzip 119 kB 118 kB 🟢 1.25 kB (-1%)
index.pack.old gzip 119 kB 118 kB
Total 5.06 MB 5.05 MB ✅ -12.8 kB

🔄 Shared (bundler-independent)

Runtimes
Canary PR Change
app-page-exp...dev.js gzip 364 kB 364 kB
app-page-exp..prod.js gzip 201 kB 201 kB
app-page-tur...dev.js gzip 364 kB 364 kB
app-page-tur..prod.js gzip 201 kB 201 kB
app-page-tur...dev.js gzip 360 kB 360 kB
app-page-tur..prod.js gzip 199 kB 199 kB
app-page.run...dev.js gzip 361 kB 361 kB
app-page.run..prod.js gzip 199 kB 199 kB
app-route-ex...dev.js gzip 81.5 kB 81.5 kB
app-route-ex..prod.js gzip 55.4 kB 55.4 kB
app-route-tu...dev.js gzip 81.5 kB 81.5 kB
app-route-tu..prod.js gzip 55.4 kB 55.4 kB
app-route-tu...dev.js gzip 81.1 kB 81.1 kB
app-route-tu..prod.js gzip 55.1 kB 55.1 kB
app-route.ru...dev.js gzip 81.1 kB 81.1 kB
app-route.ru..prod.js gzip 55.1 kB 55.1 kB
dist_client_...dev.js gzip 324 B 324 B
dist_client_...dev.js gzip 326 B 326 B
dist_client_...dev.js gzip 318 B 318 B
dist_client_...dev.js gzip 317 B 317 B
pages-api-tu...dev.js gzip 45.2 kB 45.2 kB
pages-api-tu..prod.js gzip 33.9 kB 33.9 kB
pages-api.ru...dev.js gzip 45.2 kB 45.2 kB
pages-api.ru..prod.js gzip 33.9 kB 33.9 kB
pages-turbo....dev.js gzip 54.6 kB 54.6 kB
pages-turbo...prod.js gzip 39.6 kB 39.6 kB
pages.runtim...dev.js gzip 54.6 kB 54.6 kB
pages.runtim..prod.js gzip 39.5 kB 39.5 kB
server.runti..prod.js gzip 67.5 kB 67.5 kB
use-cache-pr...dev.js gzip 71.4 kB 71.4 kB
use-cache-pr...dev.js gzip 71.4 kB 71.4 kB
use-cache-pr...dev.js gzip 69.7 kB 69.7 kB
use-cache-pr...dev.js gzip 69.7 kB 69.7 kB
Total 3.49 MB 3.49 MB ✅ -1 B
📝 Changed Files (6 files)

Files with changes:

  • pages-api.runtime.dev.js
  • pages-api.ru..time.prod.js
  • pages.runtime.dev.js
  • pages.runtime.prod.js
  • pages-api-tu..ntime.dev.js
  • pages-turbo...ntime.dev.js
View diffs
pages-api.runtime.dev.js

Diff too large to display

pages-api.ru..time.prod.js

Diff too large to display

pages.runtime.dev.js

Diff too large to display

pages.runtime.prod.js

Diff too large to display

pages-api-tu..ntime.dev.js

Diff too large to display

pages-turbo...ntime.dev.js

Diff too large to display

📎 Tarball URL
https://vercel-packages.vercel.app/next/commits/fdc78d7ecc71fa5b761d5f454abe845b49a75710/next

Commit: fdc78d7

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Tests Passed

Commit: fdc78d7

bgw added a commit that referenced this pull request Jul 8, 2026
@bgw
bgw force-pushed the bgw/shared-browser-process branch from 5ef0625 to e04c8b4 Compare July 8, 2026 23:02
@bgw
bgw marked this pull request as ready for review July 10, 2026 02:31
@bgw bgw changed the title [experiment][ci] Share a single browser instance across all test suites in a single job [ci] Share a single browser instance across all test suites in a single job Jul 10, 2026
Comment thread run-tests.js Outdated
console.log(`exiting with code ${code}`)

if (sharedBrowserServer) {
await sharedBrowserServer.close().catch(() => {})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log the error?

Comment thread test/lib/next-webdriver.ts
Comment thread run-tests.js
@bgw
bgw force-pushed the bgw/shared-browser-process branch from e04c8b4 to fdc78d7 Compare July 21, 2026 01:15
@bgw
bgw enabled auto-merge (squash) July 21, 2026 01:15
@bgw
bgw merged commit 390eff3 into canary Jul 21, 2026
127 checks passed
@bgw
bgw deleted the bgw/shared-browser-process branch July 21, 2026 01:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants