Skip to content

Fix baseline-phase unit-test suite-ordering dependency #359

Description

@SoundMindsAI

Summary

The baseline-phase orchestrator unit tests pass as part of the full suite but can fail when run in isolation (or in a different order) — a classic hermeticity bug, usually a module-level Settings() / env read that leaks state between tests.

Reproduce

.venv/bin/pytest backend/tests/unit/workers/test_orchestrator_baseline_phase.py -v

Fix

Confirmed two-layer root cause (see the spec, not just "module-level state"):

  1. _compute_baseline_wait_s in backend/workers/orchestrator.py calls get_settings() even when the study already carries an explicit trial_timeout_s — forcing a Settings() construction those tests don't need.
  2. backend/tests/unit/test_main_lifespan.py leaks DATABASE_URL_FILE / POSTGRES_PASSWORD_FILE into os.environ at collection time, so in isolation that construction raises ValidationError.

In the full suite test_main_lifespan is collected first, so the env is already set and the tests pass — hence the ordering dependency.

Durable fix: make the settings read in _compute_baseline_wait_s lazy (only when no explicit trial_timeout_s), so the baseline tests never construct Settings(). The leaking fixture in test_main_lifespan.py is documented but left out of scope (fixing it could unmask other silent free-riders).

Definition of done

  • The baseline-phase test file passes in isolation AND as part of make test-unit
  • No shared mutable/import-time state between tests
  • make test-unit green

Background

Captured at docs/00_overview/planned_features/02_mvp2/bug_baseline_phase_test_isolation/idea.md.

Good first issue: teaches hermetic-test patterns + pytest fixtures; bounded reproduction.

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueGood for newcomersready-to-executeHas approved spec + impl plan; ready for /impl-executetestsTest coverage / test infrastructure

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions