Summary
The baseline-phase orchestrator unit tests pass as part of the full suite but can fail when run in isolation (or in a different order) — a classic hermeticity bug, usually a module-level Settings() / env read that leaks state between tests.
Reproduce
.venv/bin/pytest backend/tests/unit/workers/test_orchestrator_baseline_phase.py -v
Fix
Confirmed two-layer root cause (see the spec, not just "module-level state"):
_compute_baseline_wait_s in backend/workers/orchestrator.py calls get_settings() even when the study already carries an explicit trial_timeout_s — forcing a Settings() construction those tests don't need.
backend/tests/unit/test_main_lifespan.py leaks DATABASE_URL_FILE / POSTGRES_PASSWORD_FILE into os.environ at collection time, so in isolation that construction raises ValidationError.
In the full suite test_main_lifespan is collected first, so the env is already set and the tests pass — hence the ordering dependency.
Durable fix: make the settings read in _compute_baseline_wait_s lazy (only when no explicit trial_timeout_s), so the baseline tests never construct Settings(). The leaking fixture in test_main_lifespan.py is documented but left out of scope (fixing it could unmask other silent free-riders).
Definition of done
Background
Captured at docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/idea.md.
Good first issue: teaches hermetic-test patterns + pytest fixtures; bounded reproduction.
Summary
The baseline-phase orchestrator unit tests pass as part of the full suite but can fail when run in isolation (or in a different order) — a classic hermeticity bug, usually a module-level
Settings()/ env read that leaks state between tests.Reproduce
Fix
Confirmed two-layer root cause (see the spec, not just "module-level state"):
_compute_baseline_wait_sinbackend/workers/orchestrator.pycallsget_settings()even when the study already carries an explicittrial_timeout_s— forcing aSettings()construction those tests don't need.backend/tests/unit/test_main_lifespan.pyleaksDATABASE_URL_FILE/POSTGRES_PASSWORD_FILEintoos.environat collection time, so in isolation that construction raisesValidationError.In the full suite
test_main_lifespanis collected first, so the env is already set and the tests pass — hence the ordering dependency.Durable fix: make the settings read in
_compute_baseline_wait_slazy (only when no explicittrial_timeout_s), so the baseline tests never constructSettings(). The leaking fixture intest_main_lifespan.pyis documented but left out of scope (fixing it could unmask other silent free-riders).Definition of done
make test-unitmake test-unitgreenBackground
Captured at
docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/idea.md.Good first issue: teaches hermetic-test patterns + pytest fixtures; bounded reproduction.