Skip to content

fix(tests): scope test_ent183_skill_packages' sys.modules stubs to one import (#1898) - #2031

Open
dolho wants to merge 1 commit into
devfrom
fix/1898-sys-modules-poison
Open

fix(tests): scope test_ent183_skill_packages' sys.modules stubs to one import (#1898)#2031
dolho wants to merge 1 commit into
devfrom
fix/1898-sys-modules-poison

Conversation

@dolho

@dolho dolho commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

test_ent183_skill_packages.py wrote fake modules straight into sys.modules at import time and left them there. That is collection-time code, and pytest imports every test module before running a single test — so every file collected afterwards resolved services.settings_service (plus database, services.agent_client, utils.url_validation) to a four-function fake, captured names from it at module scope, and kept those bindings for the rest of the session.

Its own autouse sys.modules-restore fixture never had a chance. By the time a fixture first runs, the damage is a binding in another module's namespace, and restoring the module table doesn't reach it.

Two symptom shapes, both of which read as unrelated bugs:

ImportError: cannot import name 'resolve_github_pat' from
'services.settings_service' (unknown location)          # at COLLECTION (#1855)

…and silent wrong behaviour when the stub happens to define the name — the 7 failures in test_1081_physical_meter.py this issue reproduces.

Fix

The stubs now live inside a mock.patch.dict scoped to the single import that needs them. Nothing survives the with; the imported module keeps its stub bindings, which is all the stubbing was ever for.

A false start worth reading

My first version also evicted cached services.skill* modules, to force a fresh import against the stubs regardless of collection order. It broke 10 sibling tests in test_ent236_skills_lifecycle and test_ent237_skill_sources: evicting and re-importing produces a second module object, so their monkeypatching lands on a module the code under test isn't using.

I caught it by running the same selection with and without my change — 435 passed without, 10 failed with — not by reading the diff. Reverted, with the reasoning left in the file: the order-dependence it addressed is a real wart, but it isn't what #1898 is about, and scope creep in a test-isolation fix is how a test-isolation fix breaks tests.

Verification

Case Before After
The issue's reproduction (+ test_1081_physical_meter) 7 failed 74 passed
#1855's pair (+ test_ent125_resilient_system_deploy) collection error 79 passed
+ test_ent236_skills_lifecycle error 124 passed
The selection my false start regressed 435 passed 436 passed, 0 failed

3 guard tests, mutation-verified — reintroducing the bare module-scope assignment fails all three. The static guard's AST walk deliberately allows sys.modules assignment inside a with: a rule banning every assignment would ban the fix along with the bug.

One correction

When I triaged this I said it explained a test_subscription_auto_switch_pingpong flake I'd seen on #2025. I can't substantiate that. The error signature matched #1855's family, but the pairing doesn't reproduce pre-fix, and neither does a broader selection. test_ent236_skills_lifecycle is a confirmed victim (error → 124 passed); the pingpong one stays unexplained.

Related

#1855 is the same root cause with a different victim and should close as a duplicate once this lands.

Closes #1898

…e import (#1898)

The file wrote fake modules straight into `sys.modules` at import time and left
them there. That is collection-time code, and pytest imports every test module
before running a single test — so every file collected afterwards resolved
`services.settings_service` (plus `database`, `services.agent_client`,
`utils.url_validation`) to a four-function fake, captured names from it at its
own module scope, and kept those bindings for the rest of the session.

Its autouse `sys.modules`-restore fixture never had a chance: by the time a
fixture first runs, the damage is a BINDING in another module's namespace, and
restoring the module table does not reach it.

Two symptom shapes, both of which read as unrelated bugs:

  ImportError: cannot import name 'resolve_github_pat' from
  'services.settings_service' (unknown location)      # at COLLECTION (#1855)

and silent wrong behaviour when the stub happens to define the name — the 7
failures in `test_1081_physical_meter.py` this issue reproduces.

The stubs now live inside a `mock.patch.dict` scoped to the single import that
needs them. Nothing survives the `with`; the imported module keeps its stub
bindings, which is all the stubbing was ever for.

A first version also evicted cached `services.skill*` modules to force a fresh
import against the stubs regardless of collection order. That broke 10 sibling
tests in `test_ent236_skills_lifecycle` and `test_ent237_skill_sources`:
evicting and re-importing produces a SECOND module object, so their
monkeypatching lands on a module the code under test is not using. Reverted —
the wart it addressed is real but is not what this issue is about, and scope
creep in a test-isolation fix is how a test-isolation fix breaks tests.

Caught by comparing the same selection with and without the change (435 passed
without, 10 failed with) rather than by reading the diff, which is the only
reason it did not ship.

Verified: the issue's reproduction (74 passed, was 7 failed), #1855's pair (79
passed, was a collection error), `test_ent236_skills_lifecycle` (124 passed,
was an error), and the selection that regressed (436 passed, 0 failed).

3 guard tests, mutation-verified: reintroducing the bare module-scope
assignment fails all of them.

Closes #1898

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

⚠️ Nightly unit-suite check skipped — merge conflict against dev.

Resolve by running git merge dev locally and pushing the result. The next nightly run will re-test once the conflict is gone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant