[ty] Avoid storing constraint nodes twice

[AlexWaygood]

Summary

Building large reachability and narrowing graphs stores each interior node twice: once in interiors and again as the key of the interning map. Store only the node ID in a hashbrown::HashTable, using interiors for hashing and equality. This reduces temporary memory during semantic-index construction while preserving the resulting nodes and their IDs.

Performance

Median peak RSS across four runs of profiling CLI builds on macOS arm64, using the existing microbenchmark inputs:

Benchmark	Before	After	Reduction
repeated_narrowed_assignments	71.7 MiB	59.5 MiB	17.1%
repeated_narrowed_assignments_suppressing_context_managers	117.6 MiB	97.6 MiB	17.0%
repeated_statement_calls_in_try	111.3 MiB	85.6 MiB	23.1%

CPU time fell by 3–11% in these three cases and was within ±4% in the remaining measured cases. These savings are in peak memory; the retained constraint graphs are unchanged.

[astral-sh-bot]

Typing conformance results

No changes detected ✅

Current numbers

The percentage of diagnostics emitted that were expected errors held steady at 97.84%. The percentage of expected errors that received a diagnostic held steady at 96.45%. The number of fully passing files held steady at 128/145.

[astral-sh-bot]

Memory usage report

Memory usage unchanged ✅

[astral-sh-bot]

ecosystem-analyzer results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

[codspeed]

Merging this PR will improve performance by 18.9%

⚡ 4 improved benchmarks
✅ 136 untouched benchmarks
⏩ 84 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Memory	ty_micro[repeated_statement_calls_in_try]	52 MB	35.9 MB	+44.63%
⚡	Memory	ty_micro[repeated_narrowed_assignments_suppressing_context_managers]	57.3 MB	45.3 MB	+26.49%
⚡	Simulation	ty_micro[repeated_statement_calls_in_try]	746.3 ms	711.1 ms	+4.94%
⚡	Memory	ty_micro[repeated_narrowed_assignments]	24.1 MB	23.1 MB	+4.13%

Tip

Curious why performance improved? Use the CodSpeed MCP and ask your agent.

---

_{Comparing alex/compact-flow-constraint-caches (5134676) with main (a159176)}

Footnotes

1. 84 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[AlexWaygood]

Codspeed and codex seem very happy about this. I don't know if this memory improvement is "real", though -- the structs optimised here are only temporary builders that are dropped pretty soon after they've served their purpose. The memory and performance improvements only show up on microbenchmarks, not on any real-world benchmarks.

[AlexWaygood]

Okay, I asked codex to dig in a bit more into how worthwhile this change really would be, and this seems more convincing to me.

Codex analysis

I’d merge it as a reduction in peak memory during indexing. The savings generalize beyond the existing benchmarks, including to a real pathological file. Broad speed improvements are much less convincing. I found no correctness blocker in the four-file diff.

1. The memory improvement is real

I compared current main (a1591769ad) against that same revision with this PR’s patch applied, avoiding unrelated differences from the PR branch being behind main. These are fresh-process, single-threaded profiling builds on macOS arm64, using five measured runs per version after warmup.

Input | Peak RSS before → after | Reduction -- | -- | -- Black’s src | 98.6 → 96.7 MiB | 1.9% typing_extensions’ src | 85.1 → 83.5 MiB | 1.8% Original reproducer: 6,000 statement calls | 43.8 → 43.4 MiB | 0.7% 320 conditional assignments | 71.6 → 58.8 MiB | 17.9% 640 conditional assignments | 197.2 → 141.2 MiB | 28.4% 640 conditional assignments under suppression | 221.3 → 160.0 MiB | 27.7% 800 locals and 800 calls inside try | 111.2 → 85.5 MiB | 23.1%

CPU improvements were approximately 1–4% for the two projects and conditional-assignment cases, and 11% for the try case. The smaller changes are insufficient evidence for promising a general speedup.

The improvements at twice the existing benchmark size are useful evidence against overfitting to its exact parameters.

2. It also helped an actual pathological source file

I traced the reachability graph limit back to Tauon’s enormous event loop, which motivated #23399. Running the original, pinned 49,912-line t_main.py gave:

• Peak RSS: 526.9 → 408.0 MiB, a 22.6% reduction.
• CPU time: 2.017 → 1.966 seconds, approximately 2.6% faster.

That comparison used three measured runs after warmup. I checked the file in isolation without Tauon’s full dependency environment, so this establishes a benefit on actual pathological source, not a complete Tauon project measurement. Diagnostics matched between versions.

For the generated Qt UI problem behind #3986, however, I would expect little additional benefit: its 6,000-call reproducer was nearly unchanged, as was the variant with an already-narrowed binding. The expensive repeated graph evaluation that originally motivated that issue was addressed by earlier changes.

3. Being temporary doesn’t make these allocations insignificant

Each node contains four u32 fields. Previously, the reverse lookup table stored the 16-byte node plus its 4-byte ID, in addition to the node vector. The new table stores just the ID: its bucket payload falls from 20 bytes to 4 bytes. This applies to every interned node, independently of the Python construct that produced it. Implementation.

Also, builders for the file’s scopes remain alive until the entire file has been visited and its use-def maps are finalized. These allocations can therefore overlap substantially and determine peak memory. Builder lifetime.

The unchanged retained-memory report is expected: the final graph representation is unchanged.

4. The limits are important

This preserves the same graph construction, node counts, operation caches, and node limits. It doesn’t improve asymptotic complexity or delay the precision fallback, because that fallback counts nodes rather than bytes. It makes large graphs cheaper to hold while constructing them. Existing limit.

Smaller tables can reduce allocation and memory-access costs, but lookups now introduce an extra indirection into the node vector. Consequently, a universal CPU improvement doesn’t follow from the implementation.

My assessment: the implementation is a general, reasonably small storage optimization. The benchmark headline overstates how broadly its gains apply, but the memory saving itself is useful—especially for unusually large control-flow graphs. A pinned Tauon project benchmark would provide stronger ongoing evidence than additional variations of the current synthetic loops.

I’d merge it as a reduction in peak memory during indexing. The savings generalize beyond the existing benchmarks, including to a real pathological file. Broad speed improvements are much less convincing. I found no correctness blocker in the four-file diff. 1. The memory improvement is real I compared current main (a159176) against that same revision with this PR’s patch applied, avoiding unrelated differences from the PR branch being behind main. These are fresh-process, single-threaded profiling builds on macOS arm64, using five measured runs per version after warmup. Input Peak RSS before → after Reduction Black’s src 98.6 → 96.7 MiB 1.9% typing_extensions’ src 85.1 → 83.5 MiB 1.8% Original reproducer: 6,000 statement calls 43.8 → 43.4 MiB 0.7% 320 conditional assignments 71.6 → 58.8 MiB 17.9% 640 conditional assignments 197.2 → 141.2 MiB 28.4% 640 conditional assignments under suppression 221.3 → 160.0 MiB 27.7% 800 locals and 800 calls inside try 111.2 → 85.5 MiB 23.1%

CPU improvements were approximately 1–4% for the two projects and conditional-assignment cases, and 11% for the try case. The smaller changes are insufficient evidence for promising a general speedup.
The improvements at twice the existing benchmark size are useful evidence against overfitting to its exact parameters.
2. It also helped an actual pathological source file
I traced the reachability graph limit back to Tauon’s enormous event loop, which motivated #23399. Running the original, pinned 49,912-line t_main.py gave:

• Peak RSS: 526.9 → 408.0 MiB, a 22.6% reduction.
• CPU time: 2.017 → 1.966 seconds, approximately 2.6% faster.
  That comparison used three measured runs after warmup. I checked the file in isolation without Tauon’s full dependency environment, so this establishes a benefit on actual pathological source, not a complete Tauon project measurement. Diagnostics matched between versions.
  For the generated Qt UI problem behind #3986, however, I would expect little additional benefit: its 6,000-call reproducer was nearly unchanged, as was the variant with an already-narrowed binding. The expensive repeated graph evaluation that originally motivated that issue was addressed by earlier changes.

3. Being temporary doesn’t make these allocations insignificant
   Each node contains four u32 fields. Previously, the reverse lookup table stored the 16-byte node plus its 4-byte ID, in addition to the node vector. The new table stores just the ID: its bucket payload falls from 20 bytes to 4 bytes. This applies to every interned node, independently of the Python construct that produced it. Implementation.
   Also, builders for the file’s scopes remain alive until the entire file has been visited and its use-def maps are finalized. These allocations can therefore overlap substantially and determine peak memory. Builder lifetime.
   The unchanged retained-memory report is expected: the final graph representation is unchanged.
4. The limits are important
   This preserves the same graph construction, node counts, operation caches, and node limits. It doesn’t improve asymptotic complexity or delay the precision fallback, because that fallback counts nodes rather than bytes. It makes large graphs cheaper to hold while constructing them. Existing limit.
   Smaller tables can reduce allocation and memory-access costs, but lookups now introduce an extra indirection into the node vector. Consequently, a universal CPU improvement doesn’t follow from the implementation.
   My assessment: the implementation is a general, reasonably small storage optimization. The benchmark headline overstates how broadly its gains apply, but the memory saving itself is useful—especially for unusually large control-flow graphs. A pinned Tauon project benchmark would provide stronger ongoing evidence than additional variations of the current synthetic loops.

[dcreager]

Interesting! I think this is a good pattern that we might want to apply elsewhere too. But I don't think it's worth prematurely generalizing in this PR.

[AlexWaygood]

Interesting! I think this is a good pattern that we might want to apply elsewhere too. But I don't think it's worth prematurely generalizing in this PR.

I looked into whether we could do this kind of thing in a few other places in #28647, but it doesn't look convincing right now :/