Skip to content

Fix epoch CPU admission for hot-add#5411

Open
Alan-Jowett wants to merge 18 commits into
microsoft:mainfrom
Alan-Jowett:alanjo/hot-add
Open

Fix epoch CPU admission for hot-add#5411
Alan-Jowett wants to merge 18 commits into
microsoft:mainfrom
Alan-Jowett:alanjo/hot-add

Conversation

@Alan-Jowett

@Alan-Jowett Alan-Jowett commented Jul 2, 2026

Copy link
Copy Markdown
Member

Description

Fixes #3768.

This updates epoch CPU tracking to support systems where the maximum logical processor count exceeds the active processor count at startup.

  • size the epoch backing table for the maximum logical processor count while admitting only active CPUs into the consensus ring
  • use processor-change callbacks to initialize and admit hot-added CPUs in two phases
  • fail fast if current-CPU epoch APIs are reached before a CPU finishes admission
  • add unit coverage for startup active-CPU subsets, hot-add admission, and the unadmitted-CPU fail-fast case
  • document the active/admitted CPU model and hot-add behavior

Testing

Not run per request.

If new tests were added:

  • Unit tests are added.
  • Driver tests are added.
  • Fuzz tests are added.

Documentation

Updated docs/EpochBasedMemoryManagement.md.

Installation

No installer impact.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the runtime epoch-based memory reclamation subsystem to tolerate systems where the kernel-reported maximum logical CPU count exceeds the active CPU count at startup, and to support CPU hot-add by admitting CPUs into the epoch participant set in phases.

Changes:

  • Reworks ebpf_epoch CPU tracking to allocate per-CPU backing state for the maximum CPU count while only admitting active CPUs into the consensus/participant ring (with hot-add admission via processor-change callbacks).
  • Adds unit tests to cover startup active-CPU subsets, hot-add admission behavior, and an unadmitted-CPU fail-fast path (child-process based).
  • Adjusts libfuzzer harness/core lifecycle management and updates CI test filters and epoch documentation accordingly.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/libfuzzer/include/fuzz_helper_function.hpp Makes ebpf core init/term optional per helper instance and hardens cleanup paths.
tests/libfuzzer/core_helper_fuzzer/libfuzz_harness.cpp Centralizes ebpf core lifetime in the fuzz harness and disables per-helper core lifecycle management.
libs/runtime/unit/platform_unit_test.cpp Adds unit tests for startup active subset, hot-add admission, and fail-fast on unadmitted CPU (child process).
libs/runtime/ebpf_epoch.c Implements admitted-CPU ring + processor change callback path for initializing/admitting CPUs and routing epoch consensus only across admitted CPUs.
docs/EpochBasedMemoryManagement.md Documents the maximum-vs-admitted CPU model and hot-add admission behavior.
.github/workflows/cicd.yml Excludes the new fail-fast epoch tests from multiple CI jobs.
.azure/OneBranch.PullRequest.yml Excludes the new fail-fast epoch tests from multiple Azure pipeline jobs.

Comment thread libs/runtime/unit/platform_unit_test.cpp Outdated
Comment thread .github/workflows/cicd.yml
Comment thread .azure/OneBranch.PullRequest.yml
Comment thread libs/runtime/ebpf_epoch.c
Comment thread libs/runtime/ebpf_epoch.c Outdated
Comment thread libs/runtime/ebpf_epoch.c
@Alan-Jowett

Copy link
Copy Markdown
Member Author

@saxena-anurag You are 100% correct. I missed that the hot-add complete occurs after the CPU can be scheduled. That totally breaks my current assumption. Revisiting this.

@Alan-Jowett
Alan-Jowett marked this pull request as draft July 9, 2026 20:53
auto-merge was automatically disabled July 9, 2026 20:53

Pull request was converted to draft

@Alan-Jowett
Alan-Jowett marked this pull request as ready for review July 14, 2026 15:42
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Alan-Jowett and others added 16 commits July 14, 2026 12:27
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Add formal modeling infrastructure with TLA+ and TLAPS:
- TLA+ models: epoch, ring buffer, hash table
- TLAPS proofs for key safety properties
- CI job for TLC model checking
- Docker-based TLAPS proof verification
- TLATeX documentation generation
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d729ac7e-d3b0-4e13-97ba-f8aa6a2d02d9
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d729ac7e-d3b0-4e13-97ba-f8aa6a2d02d9
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 572a5c43-fdfe-45bd-9ac4-1e60e8530860
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 572a5c43-fdfe-45bd-9ac4-1e60e8530860
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 572a5c43-fdfe-45bd-9ac4-1e60e8530860
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 572a5c43-fdfe-45bd-9ac4-1e60e8530860
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 572a5c43-fdfe-45bd-9ac4-1e60e8530860
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Ensure the hot-add epoch tests always unblock and join the async synchronize thread before unwinding so assertion failures do not turn into hung CI jobs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 572a5c43-fdfe-45bd-9ac4-1e60e8530860
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Apply the repo formatter output for the new hot-add test synchronize helper.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 572a5c43-fdfe-45bd-9ac4-1e60e8530860
Signed-off-by: Alan Jowett <alanjo@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

ebpf_epoch_initiate fails on systems with KeQueryMaximumProcessorCountEx > KeQueryActiveProcessorCountEx

6 participants