Skip to content

Non-deterministic unbounded memory growth (>35 GB, or minutes of thrash) type-checking a pandas-heavy library; same input sometimes completes at ~1.5 GB #4500

Description

@doronngrid

Summary

Summary

ty check on one library in our workspace has non-deterministic memory/time behavior — same source tree, same command, same machine, only the outcome changes:

  • Some runs complete in ~2 min at ~1.5 GB RSS, 0 diagnostics.
  • Some runs are OOM-killed by the OS (exit 137).
  • One run was observed in Activity Monitor climbing steadily past 35 GB RSS with no sign of leveling off, killed manually.
  • Some runs neither finish nor OOM: they run 7+ minutes, RSS oscillating between ~0.7 GB and ~6 GB, making no visible progress until killed.

It correlates with a single commit that added ~1,750 lines across two large, untyped, DataFrame-mutation-heavy modules.

Repro shape

We can't share the private repo, but the library is:

  • 41 .py files, ~14k LOC in src/, 7 first-party workspace deps + the usual third-party closure (pandas / numpy / pydantic / scipy).
  • Invocation: cd libs/<lib> && uvx ty check. Project has [tool.ty.rules] with 2 rules "ignore" and one [[tool.ty.overrides]] block scoping ~7 rules to "ignore" for a single large file.

The instability started with a commit adding two modules that are verbatim
ports of untyped stateful-pandas code:

  1. ~980-line module — a @DataClass with ~15 fields (several dict[Enum, pd.Series]), then a chain of functions each doing 20–40 df[SomeStrEnum.MEMBER] = <Series expr> assignments (SomeStrEnum(str, Enum)),
    e.g. df[Col.X] = (df[Col.Y] * df[Col.Z]).round(2), plus f"...{part}..." f-string keys.
  2. ~770-line module — one large class, ~30 pd.Series attributes, many @Property accessors, and float | Callable | None parameters narrowed with isinstance(x, collections.abc.Callable) / isinstance(x, float) (~6 sites), assigned to instance attributes that also receive np.inf and lambdas.

Isolation done so far

  • Excluding just those two files' bodies via [tool.ty.src] exclude (callers still resolve their signatures) drops the run from "never finishes / 35 GB" to ~2 s at <200 MB, deterministic across 5+ runs. That also let ty finish for the first time and report 36 real diagnostics that every prior run had crashed before reaching.
  • RAYON_NUM_THREADS=1 does NOT fix it — still peaks ~5 GB and is much slower.
  • An existing file of the same shape (~2,150 lines) is already scoped out via [[tool.ty.overrides]] for diagnostic-noise reasons; the blowup only became unstable when the two new files were added without being scoped out.

Version

ty 0.0.79 (b4cd792 2026-09-07)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-infoMore information is needed from the issue authorneeds-mreNeeds more information for reproductionnon-determinismFlaky, order-dependent, race-dependent, or otherwise non-deterministic behaviorperformancePotential performance improvement

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions