fix(tests): scope test_ent183_skill_packages' sys.modules stubs to one import (#1898) - #2031
Open
dolho wants to merge 1 commit into
Open
fix(tests): scope test_ent183_skill_packages' sys.modules stubs to one import (#1898)#2031dolho wants to merge 1 commit into
dolho wants to merge 1 commit into
Conversation
…e import (#1898) The file wrote fake modules straight into `sys.modules` at import time and left them there. That is collection-time code, and pytest imports every test module before running a single test — so every file collected afterwards resolved `services.settings_service` (plus `database`, `services.agent_client`, `utils.url_validation`) to a four-function fake, captured names from it at its own module scope, and kept those bindings for the rest of the session. Its autouse `sys.modules`-restore fixture never had a chance: by the time a fixture first runs, the damage is a BINDING in another module's namespace, and restoring the module table does not reach it. Two symptom shapes, both of which read as unrelated bugs: ImportError: cannot import name 'resolve_github_pat' from 'services.settings_service' (unknown location) # at COLLECTION (#1855) and silent wrong behaviour when the stub happens to define the name — the 7 failures in `test_1081_physical_meter.py` this issue reproduces. The stubs now live inside a `mock.patch.dict` scoped to the single import that needs them. Nothing survives the `with`; the imported module keeps its stub bindings, which is all the stubbing was ever for. A first version also evicted cached `services.skill*` modules to force a fresh import against the stubs regardless of collection order. That broke 10 sibling tests in `test_ent236_skills_lifecycle` and `test_ent237_skill_sources`: evicting and re-importing produces a SECOND module object, so their monkeypatching lands on a module the code under test is not using. Reverted — the wart it addressed is real but is not what this issue is about, and scope creep in a test-isolation fix is how a test-isolation fix breaks tests. Caught by comparing the same selection with and without the change (435 passed without, 10 failed with) rather than by reading the diff, which is the only reason it did not ship. Verified: the issue's reproduction (74 passed, was 7 failed), #1855's pair (79 passed, was a collection error), `test_ent236_skills_lifecycle` (124 passed, was an error), and the selection that regressed (436 passed, 0 failed). 3 guard tests, mutation-verified: reintroducing the bare module-scope assignment fails all of them. Closes #1898 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Resolve by running |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
test_ent183_skill_packages.pywrote fake modules straight intosys.modulesat import time and left them there. That is collection-time code, and pytest imports every test module before running a single test — so every file collected afterwards resolvedservices.settings_service(plusdatabase,services.agent_client,utils.url_validation) to a four-function fake, captured names from it at module scope, and kept those bindings for the rest of the session.Its own autouse
sys.modules-restore fixture never had a chance. By the time a fixture first runs, the damage is a binding in another module's namespace, and restoring the module table doesn't reach it.Two symptom shapes, both of which read as unrelated bugs:
…and silent wrong behaviour when the stub happens to define the name — the 7 failures in
test_1081_physical_meter.pythis issue reproduces.Fix
The stubs now live inside a
mock.patch.dictscoped to the single import that needs them. Nothing survives thewith; the imported module keeps its stub bindings, which is all the stubbing was ever for.A false start worth reading
My first version also evicted cached
services.skill*modules, to force a fresh import against the stubs regardless of collection order. It broke 10 sibling tests intest_ent236_skills_lifecycleandtest_ent237_skill_sources: evicting and re-importing produces a second module object, so their monkeypatching lands on a module the code under test isn't using.I caught it by running the same selection with and without my change — 435 passed without, 10 failed with — not by reading the diff. Reverted, with the reasoning left in the file: the order-dependence it addressed is a real wart, but it isn't what #1898 is about, and scope creep in a test-isolation fix is how a test-isolation fix breaks tests.
Verification
+ test_1081_physical_meter)+ test_ent125_resilient_system_deploy)+ test_ent236_skills_lifecycle3 guard tests, mutation-verified — reintroducing the bare module-scope assignment fails all three. The static guard's AST walk deliberately allows
sys.modulesassignment inside awith: a rule banning every assignment would ban the fix along with the bug.One correction
When I triaged this I said it explained a
test_subscription_auto_switch_pingpongflake I'd seen on #2025. I can't substantiate that. The error signature matched #1855's family, but the pairing doesn't reproduce pre-fix, and neither does a broader selection.test_ent236_skills_lifecycleis a confirmed victim (error → 124 passed); the pingpong one stays unexplained.Related
#1855 is the same root cause with a different victim and should close as a duplicate once this lands.
Closes #1898