Skip to content

[build-tools] Port Maestro Tests bash step to TypeScript#3641

Merged
hSATAC merged 3 commits intomainfrom
ash/eng-19927-port-bash-maestro-step-to-typescript
Apr 30, 2026
Merged

[build-tools] Port Maestro Tests bash step to TypeScript#3641
hSATAC merged 3 commits intomainfrom
ash/eng-19927-port-bash-maestro-step-to-typescript

Conversation

@hSATAC
Copy link
Copy Markdown
Contributor

@hSATAC hSATAC commented Apr 28, 2026

Why

The Maestro test execution currently lives as an inline bash shellscript step in WorkflowMaestroTestJob (universe). Porting to a typed build function gives us type checking, unit tests, and a base for smart retry (next PR in the stack), which is painful to express in bash.

Linear: ENG-19927.

How

New eas/run_maestro_tests build function with behavior identical to the bash version: runs maestro test ... with the full input flow set, retries the whole set on failure, copies the latest attempt's JUnit XML to final_report_path. Adds a small parseMaestroResults / parseJUnitFile helper shared with eas/report_maestro_test_results.

Universe-side wiring (replacing the bash step with uses: eas/run_maestro_tests) is a separate PR.

Test Plan

  • New unit tests in runMaestroTests.test.ts covering single-attempt success, retry-on-failure, attempt-N JUnit naming, copy-latest behavior, env propagation, error paths.
  • Local end-to-end smoke test against the playground.

@linear
Copy link
Copy Markdown

linear Bot commented Apr 28, 2026

Copy link
Copy Markdown
Contributor Author

hSATAC commented Apr 28, 2026

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

Codecov Report

❌ Patch coverage is 96.39640% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.97%. Comparing base (e74751c) to head (0ccb966).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...es/build-tools/src/steps/functions/maestroTests.ts 95.61% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3641      +/-   ##
==========================================
+ Coverage   55.83%   55.97%   +0.14%     
==========================================
  Files         864      866       +2     
  Lines       37170    37312     +142     
  Branches     7721     7759      +38     
==========================================
+ Hits        20751    20882     +131     
- Misses      16323    16334      +11     
  Partials       96       96              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hSATAC hSATAC force-pushed the ash/eng-19927-port-bash-maestro-step-to-typescript branch from a89b35a to 9806cfe Compare April 29, 2026 07:30
@hSATAC hSATAC requested a review from sjchmiela April 29, 2026 08:04
@hSATAC hSATAC marked this pull request as ready for review April 29, 2026 08:04
@github-actions
Copy link
Copy Markdown

Subscribed to pull request

File Patterns Mentions
**/* @douglowder

Generated by CodeMention

Copy link
Copy Markdown
Contributor

@sjchmiela sjchmiela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

amazing

Comment thread packages/build-tools/src/steps/functions/runMaestroTests.ts Outdated
Comment thread packages/build-tools/src/steps/functions/runMaestroTests.ts Outdated
Comment thread packages/build-tools/src/steps/functions/runMaestroTests.ts Outdated
Comment thread packages/build-tools/src/steps/functions/runMaestroTests.ts Outdated
Comment thread packages/build-tools/src/steps/functions/runMaestroTests.ts Outdated
Comment thread packages/build-tools/src/steps/functions/maestroResultParser.ts Outdated
Comment thread packages/build-tools/src/steps/functions/runMaestroTests.ts Outdated
Comment thread packages/build-tools/src/steps/functions/runMaestroTests.ts Outdated
Replace the bash shellscript step (defined in www/WorkflowMaestroTestJob)
with a typed BuildFunction. Behaviorally identical to the bash version:
runs maestro with the full flow_paths set, retries the whole set on
failure, copies the latest attempt's JUnit XML to final_report_path.

Smart retry (only re-running failed flows) is added in a follow-up PR.

Signed-off-by: Ash Wu <hsatac@gmail.com>
@hSATAC hSATAC force-pushed the ash/eng-19927-port-bash-maestro-step-to-typescript branch from 9806cfe to 9d16d62 Compare April 30, 2026 07:27
Signed-off-by: Ash Wu <hsatac@gmail.com>
@hSATAC hSATAC force-pushed the ash/eng-19927-port-bash-maestro-step-to-typescript branch from 9d16d62 to b3f6380 Compare April 30, 2026 07:44
Signed-off-by: Ash Wu <hsatac@gmail.com>
@github-actions
Copy link
Copy Markdown

⏩ The changelog entry check has been skipped since the "no changelog" label is present.

@hSATAC hSATAC merged commit 8229dc8 into main Apr 30, 2026
11 checks passed
@hSATAC hSATAC deleted the ash/eng-19927-port-bash-maestro-step-to-typescript branch April 30, 2026 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changelog PR that doesn't require a changelog entry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants