Skip to content

Performance Tests: log timestamps, optimize build overhead #78237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jsnajdr merged 3 commits into from
May 14, 2026
Merged

Performance Tests: log timestamps, optimize build overhead #78237

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
71928f6
Performance Tests: add timing to the perf run log
jsnajdr May 13, 2026
ebcbcdc
Optimize build
jsnajdr May 13, 2026
e967096
Reorder test runs to avoid starting/stopping environment
jsnajdr May 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 32 additions & 19 deletions bin/plugin/commands/performance.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ const RESULTS_FILE_SUFFIX = '.performance-results.json';
* @property {string=} wpVersion The WordPress version to be used as the base install for testing.
*/

const perfStartTime = performance.now();

/**
* A logging helper for printing steps and their substeps.
*
Expand All @@ -40,9 +42,21 @@ const RESULTS_FILE_SUFFIX = '.performance-results.json';
* @param {...any} args Rest of the arguments to pass to console.log.
*/
function logAtIndent( indent, msg, ...args ) {
const elapsed = performance.now() - perfStartTime;
const totalSeconds = Math.floor( elapsed / 100 ) / 10;
const minutes = Math.floor( totalSeconds / 60 );
const seconds = ( totalSeconds % 60 ).toFixed( 1 ).padStart( 4, '0' );
const timestamp = `[${ String( minutes ).padStart(
2,
'0'
) }:${ seconds }]`;

const prefix = indent === 0 ? '▶ ' : '> ';
const newline = indent === 0 ? '\n' : '';
return log( newline + ' '.repeat( indent ) + prefix + msg, ...args );
return log(
newline + timestamp + ' ' + ' '.repeat( indent ) + prefix + msg,
...args
);
}

/**
Expand Down Expand Up @@ -346,7 +360,7 @@ async function runPerformanceTests( branches, options ) {

logAtIndent( 2, 'Installing dependencies and building' );
await runShellScript(
`bash -c "source $HOME/.nvm/nvm.sh && nvm install && npm ci && npx playwright install chromium --with-deps && npm run build"`,
`bash -c "source $HOME/.nvm/nvm.sh && nvm install && npm ci && npx playwright install chromium --with-deps"`,
testRunnerDir
);

Expand Down Expand Up @@ -389,7 +403,7 @@ async function runPerformanceTests( branches, options ) {

logAtIndent( 3, 'Installing dependencies and building' );
await runShellScript(
`bash -c "source $HOME/.nvm/nvm.sh && nvm install && npm ci && npm run build"`,
`bash -c "source $HOME/.nvm/nvm.sh && nvm install && npm ci && npm run build -- --skip-types"`,

@ellatrix ellatrix May 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

buildDir
);

Expand Down Expand Up @@ -467,38 +481,37 @@ async function runPerformanceTests( branches, options ) {

const wpEnvPath = path.join( testRunnerDir, 'node_modules/.bin/wp-env' );

for ( const testSuite of testSuites ) {
for ( let i = 1; i <= TEST_ROUNDS; i++ ) {
for ( let i = 1; i <= TEST_ROUNDS; i++ ) {
for ( const [ branchIdx, branch ] of branches.entries() ) {
logAtIndent(
1,
// prettier-ignore
`Suite: ${ formats.success( testSuite ) } (round ${ i } of ${ TEST_ROUNDS })`
`Branch: ${ formats.success( branch ) } (round ${ i } of ${ TEST_ROUNDS })`
);

for ( const [ branchIdx, branch ] of branches.entries() ) {
logAtIndent( 2, 'Branch:', formats.success( branch ) );
const sanitizedBranchName = sanitizeBranchName( branch );
// @ts-ignore
const envDir = branchDirs[ branch ];

const sanitizedBranchName = sanitizeBranchName( branch );
const runKey = `${ testSuite }_${ sanitizedBranchName }_round-${ i }`;
// @ts-ignore
const envDir = branchDirs[ branch ];
logAtIndent( 2, 'Starting environment' );
await runShellScript( `${ wpEnvPath } start`, envDir );

for ( const testSuite of testSuites ) {
logAtIndent( 2, `Suite: ${ formats.success( testSuite ) }` );

logAtIndent( 3, 'Starting environment' );
await runShellScript( `${ wpEnvPath } start`, envDir );
const runKey = `${ testSuite }_${ sanitizedBranchName }_round-${ i }`;

logAtIndent( 3, 'Running tests' );
// Only the head branch saves traces; comparison branches share
// the same artifacts directory and would otherwise overwrite.
await runTestSuite(
testSuite,
testRunnerDir,
runKey,
branchIdx === 0
);

logAtIndent( 3, 'Stopping environment' );
await runShellScript( `${ wpEnvPath } stop`, envDir );
}

logAtIndent( 2, 'Stopping environment' );
await runShellScript( `${ wpEnvPath } stop`, envDir );
}
}

Expand Down
3 changes: 1 addition & 2 deletions packages/e2e-test-utils-playwright/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,10 @@
"build",
"build-types"
],
"main": "./build/index.cjs",
"exports": {
".": {
"types": "./build-types/index.d.ts",
"default": "./build/index.cjs"
"default": "./src/index.ts"

@ellatrix ellatrix May 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I'm reading this right. Are we changing perf tests to run on src instead of build?

@jsnajdr jsnajdr May 14, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, when a test suite running inside Playwright imports an util from @wordpress/e2e-test-utils-playwright, it will import directly the TS file without any intermediate build step.

Many of the *.spec.ts files are already TypeScript, and Playwright does internal preprocessing when executing them. It can handle the raw .ts files just fine.

This removes the need to run npm run build if you only want to run Playwright tests and nothing else. The actual target server is checked out and built in a different folder.

@t-hamano t-hamano May 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change may have affected consumers. See #78832

@jsnajdr jsnajdr Jun 1, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing in #78847.

@anomiex anomiex Jun 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also #79018

},
"./package.json": "./package.json"
},
Expand Down
Loading