Skip to content

Free threading scaling bottleneck in sys.intern() #146041

Description

@colesbury

Bug report

Calling sys.intern() on a string that's not interned acquires the interpreter-wide interned_mutex. This can lead to scaling bottlenecks either due to explicit sys.intern() calls or indirectly because PyObject_SetAttr calls _PyUnicode_InternMortal.

We should avoid the lock acquisition if there is an interned copy already present in the interned_dict.

For example:

_ATTR_PREFIX = "bench"

@register_benchmark
def setattr_non_interned():
    prefix = _ATTR_PREFIX
    obj = MyObject()
    for _ in range(1000 * WORK_SCALE):
        setattr(obj, f"{prefix}_a", None)
        setattr(obj, f"{prefix}_b", None)
        setattr(obj, f"{prefix}_c", None)

This came up when looking at Django scaling bottlenecks.

Linked PRs

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

3.14bugs and security fixes3.15pre-release feature fixes, bugs and security fixesperformancePerformance or resource usagetopic-free-threadingtype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions