Fix flaky test in tabs.spec.js - #80373
Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
Size Change: 0 B Total Size: 7.72 MB |
|
Flaky tests detected in 842f67e. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/29522730896
|
* Fix tabs test by excluding parent 'Tab Panels' name and counting all panels * Extend comment on test fix Co-authored-by: alecgeatches <alecgeatches@git.wordpress.org> Co-authored-by: Mamaduka <mamaduka@git.wordpress.org>
What?
Fixes the
adds and activates a new tab when pressing Enter at the end of a tab labeltabs test to pass reliably in CI.Why?
See #80106. The test has been flaky including 10
trunkfailures in the last week. I've personally also seeing very reliable failures in other PRs (#79685), but because of the test flakiness it's hard to tell if it's a PR problem or not.How?
The test previously ran this code to check for tab panels:
This was probably not working as expected.
getByRole()by default only returns visible elements:That means
toHaveCount( 3 )was expecting 3 visible tab sections at the same time, which doesn't make sense for the test, given that only 1 should be visible. Instead, here's what I think was happening. Visually, after the 3rd tab is added in the test, a directBlock: Tab Paneldocument query actually results in 4 results (including non-visible items):tab-panel-selector.mov
This includes the
Block: Tab Panelsparent item and each of the three child tabs. In local testing withgetByRole(), the test was actually returning 2panels, the newly created tab panel and the parent panel, and none of the hidden panels. When the test was "failing", it was properly reporting the correct number (2) of matchingpanels. I think in CI, running slower, there's a brief window when both Tab 2 and the new third tab are both present in the DOM before Tab 2 is hidden. The briefly still-visible Tab 2, the new tab, plus the parent DOM item, were adding to 3 incidentally.To fix it, first use
exact: trueto avoid matching the parent "Block: Tab Panels" DOM item, and then explicitlyincludeHidden: trueso the test assertion makes sense. Otherwise, we're only querying visible panels. Now we can correctly assert the 3 final panels exist, excluding the parent item, and that the last is visible.Testing Instructions
See CI test passes.