feat(starlark): Add per-stage 'if' and 'retry_until' expressions - #1073
Draft
michaelboulton wants to merge 12 commits into
Draft
feat(starlark): Add per-stage 'if' and 'retry_until' expressions#1073michaelboulton wants to merge 12 commits into
michaelboulton wants to merge 12 commits into
Conversation
Writing a whole 'control_flow' script is a lot of ceremony when all you want is
"only run this stage if the last one returned something". Add two per-stage keys
which are single Starlark expressions evaluated by the same embedded
interpreter, with all test variables bound directly as globals:
- name: second stage
if: var_x > 2
- name: poll until ready
max_retries: 20
delay_after: 1
retry_until: response.body["status"] == "ready"
No script, no load(), and stages don't need an 'id'. Both are gated behind the
existing --tavern-experimental-starlark-pipeline flag.
'if' is the Starlark counterpart of the simpleeval 'skip' key (inverted); using
both on one stage is a schema error. It applies to normal stages only, not
'finally' stages.
'retry_until' is an extra success predicate inside the existing retry() loop, so
it reuses max_retries/delay_after as-is. It additionally gets a 'response' struct
with the same properties as the one returned by run_stage(), for which
_create_response_struct is extracted into its own module. max_retries is
required alongside it, enforced by the schema.
Note these expressions are deliberately not format-string interpolated -
variables are real Starlark values, so 'var_x > 2' works and '{var_x} > 2' does
not.
Also fixes the 'pip install tavern[starlark]' hint, which should be
tavern[scriptable].
Using 'skip' on a stage now raises a DeprecationWarning pointing at the 'if' key, which does the same thing with the logic inverted. The 'skip' *marker*, which skips a whole test, is unaffected. The warning fires whenever the key is present, including 'skip: False', so that people migrating see it regardless of the value. The docs for both the boolean and simpleeval forms are marked deprecated and link to the 'if' documentation, with a side by side example of the equivalent expression.
'retry_until' was being evaluated after a stage passed, making it an extra condition on top of the response block. It should instead decide whether to keep retrying a stage that failed - a stage which passes is finished and is never retried, so the expression is not evaluated at all. On failure the expression is evaluated against the response from the failed attempt. If it is true the stage is treated as finished and the test carries on, which is the 'continue_on_fail' behaviour originally intended; if false the stage is retried as before. To make the response available on the failure path, TavernException grows a 'response' attribute which wrapped_run_stage sets when a verifier raises. If the request itself failed there is no response, so the stage is just retried.
…til' Adds coverage for the use case in #751 - polling a long running job until it reaches any terminal state, rather than guessing how many retries it needs. The integration server gets a '/job/<name>' endpoint which is in progress for the first couple of polls and then settles on SUCCESS or FAILED.
'retry_until' can only say when to stop polling, which counts as a pass. 'fail_if' is the negative counterpart - it is evaluated after every attempt at a stage, and if it is true the test fails immediately without any further retries. This means polling a long running job can stop as soon as it reaches a state it will never recover from. #751
- Updated the condition to first check for `retry_until` before evaluating `e.response` in `_core/testhelpers.py`. - Consolidated nested checks to improve readability. - Separated the logic handling cases where `e.response` is `None` for better debugging and maintainability.
michaelboulton
commented
Aug 2, 2026
… 'skip' Per-stage 'if'/'retry_until'/'fail_if' expressions are now format-string interpolated like everywhere else in Tavern, rather than binding test variables as Starlark globals. This makes them consistent with the existing 'skip' key and lets them refer to variables whose names are not valid Starlark identifiers, such as ones containing a dash. Errors quote both the original expression and the interpolated one. 'skip' is also no longer deprecated - 'if' only works on HTTP stages, so it is not a full replacement yet.
- Added a note in `docs/source/core_concepts/marks.md` mentioning that `skip` may be removed in the future in favor of `if`.
This reverts commit 914a02b.
'if', 'retry_until' and 'fail_if' can now be a YAML block scalar with several statements in it, where the value of the last statement decides the result. The helper modules can be load()ed as in a 'control_flow' script, so a condition can use 're' to pull a group out of a response. 'run_stage' is not available - the stage the expression is attached to is already being run - and loading anything other than @tavern_helpers.star is an error. The 're'/'time'/'log' bindings move to a new 'builtins' module so they can be shared with the expression path, which must not import tavern._core.run.
…assignment Returning the value of the last statement is behaviour of the starlark binding rather than something the language spec promises, and a script which ends on an assignment quietly evaluates to None instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
add gha style 'if' and 'retry until' actions. reuses the same starlark code.