Skip to content

Fix flaky test in tabs.spec.js - #80373

Merged
alecgeatches merged 2 commits into
trunkfrom
fix/flaky-tabs-test
Jul 16, 2026
Merged

Fix flaky test in tabs.spec.js#80373
alecgeatches merged 2 commits into
trunkfrom
fix/flaky-tabs-test

Conversation

@alecgeatches

@alecgeatches alecgeatches commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

What?

Fixes the adds and activates a new tab when pressing Enter at the end of a tab label tabs test to pass reliably in CI.

Why?

See #80106. The test has been flaky including 10 trunk failures in the last week. I've personally also seeing very reliable failures in other PRs (#79685), but because of the test flakiness it's hard to tell if it's a PR problem or not.

How?

The test previously ran this code to check for tab panels:

const panels = editor.canvas.getByRole( 'document', {
    name: 'Block: Tab Panel',
} );

await expect( panels ).toHaveCount( 3 );
await expect( panels.nth( 2 ) ).toBeVisible();

This was probably not working as expected. getByRole() by default only returns visible elements:

includeHidden

Option that controls whether hidden elements are matched. By default, only non-hidden elements, as defined by ARIA, are matched by role selector.

That means toHaveCount( 3 ) was expecting 3 visible tab sections at the same time, which doesn't make sense for the test, given that only 1 should be visible. Instead, here's what I think was happening. Visually, after the 3rd tab is added in the test, a direct Block: Tab Panel document query actually results in 4 results (including non-visible items):

tab-panel-selector.mov

This includes the Block: Tab Panels parent item and each of the three child tabs. In local testing with getByRole(), the test was actually returning 2 panels, the newly created tab panel and the parent panel, and none of the hidden panels. When the test was "failing", it was properly reporting the correct number (2) of matching panels. I think in CI, running slower, there's a brief window when both Tab 2 and the new third tab are both present in the DOM before Tab 2 is hidden. The briefly still-visible Tab 2, the new tab, plus the parent DOM item, were adding to 3 incidentally.

To fix it, first use exact: true to avoid matching the parent "Block: Tab Panels" DOM item, and then explicitly includeHidden: true so the test assertion makes sense. Otherwise, we're only querying visible panels. Now we can correctly assert the 3 final panels exist, excluding the parent item, and that the last is visible.

Testing Instructions

See CI test passes.

@alecgeatches
alecgeatches requested a review from t-hamano July 16, 2026 18:08
@alecgeatches alecgeatches self-assigned this Jul 16, 2026
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: alecgeatches <alecgeatches@git.wordpress.org>
Co-authored-by: Mamaduka <mamaduka@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@alecgeatches

Copy link
Copy Markdown
Contributor Author

@t-hamano I've been seeing a reliable tab test failure and flakiness (#80106), here's a PR to address that.

@alecgeatches alecgeatches added the [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. label Jul 16, 2026
@github-actions

Copy link
Copy Markdown

Size Change: 0 B

Total Size: 7.72 MB

compressed-size-action

@github-actions

Copy link
Copy Markdown

Flaky tests detected in 842f67e.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/29522730896
📝 Reported issues:

@Mamaduka Mamaduka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also fixed this in #80192, but makes sense to land separately to resolve flaky tests.

@alecgeatches
alecgeatches merged commit 98f5b26 into trunk Jul 16, 2026
49 of 50 checks passed
@alecgeatches
alecgeatches deleted the fix/flaky-tabs-test branch July 16, 2026 20:13
@github-actions github-actions Bot added this to the Gutenberg 23.7 milestone Jul 16, 2026
shail-mehta pushed a commit that referenced this pull request Jul 17, 2026
* Fix tabs test by excluding parent 'Tab Panels' name and counting all panels

* Extend comment on test fix

Co-authored-by: alecgeatches <alecgeatches@git.wordpress.org>
Co-authored-by: Mamaduka <mamaduka@git.wordpress.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants