Skip to content

Escape logger is not thread-safe (missed escapes under threaded schedulers) #944

Description

@mwcraig

Follow-up from PR #942 (review item P3, #942 (review)).

The escape logger installed by ccdproc/conftest.py uses a single process-global _ReentrancyGuard shared by all three wrapped functions (np.asarray, np.asanyarray, np.ma.asanyarray). Under a threaded scheduler (e.g. dask's default threaded scheduler):

  • While thread A holds the guard (guard.active = True), a genuine escape occurring concurrently on thread B is silently skipped — the wrapper treats it as a re-entrant call.
  • The tally update in record_escape_log() (_ESCAPE_LOG_COUNTS[key] += 1 on a defaultdict) is not atomic on first insert.

Both failure modes drop real escapes, so the baseline ratchet (CCDPROC_ENFORCE_ESCAPE_BASELINE=1) can produce false negatives — never false alarms — meaning a new escape site reached only from worker threads could slip past CI.

Fix sketch:

  • Replace the shared guard with threading.local() so re-entrancy is tracked per thread.
  • Take a threading.Lock around the tally increment (or use a thread-safe counter).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions