Implement BTF-resolved function support by Alan-Jowett · Pull Request #5336 · microsoft/ebpf-for-windows

Alan-Jowett · 2026-06-02T20:55:27Z

Description

Implement end-to-end BTF-resolved function support across verification, native code generation, native/JIT/runtime loading, and the in-tree sample provider.

This updates the branch from the earlier registry-publication-only slice to the full feature set for #1901, including:

registry publication and deletion helpers for BTF-resolved function provider metadata,
verifier and service-side plumbing for BTF-resolved function metadata,
bpf2c metadata and proof-of-verification hash updates,
native, JIT, and interpreter runtime wiring for BTF-resolved dependencies,
sample BTF provider support under undocked\tests\sample\ext,
compatibility and regression fixes for native metadata loading, /analyze, sanitize, and unit-test coverage,
refreshed generated bpf2c expected outputs and updated BTF-related documentation/specs.

Fixes #1901.

Testing

Existing coverage was updated and expanded for the new BTF-resolved-function paths.

Unit tests are added.
Driver tests are added.

Built and exercised relevant existing test/build coverage, including tests\unit_tests, tests\bpf2c_tests, tests\sample, and full x64\Debug\unit_tests.exe.

Documentation

BTF-resolved function documentation and specs were updated to match the implemented design and runtime behavior.

Installation

No installer impact.

Copilot

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

Copilot

Pull request overview

Copilot reviewed 253 out of 253 changed files in this pull request and generated 5 comments.

Copilot

Pull request overview

Copilot reviewed 253 out of 253 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 253 out of 253 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 253 out of 253 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Move BTF-resolved-function provider ownership out of ebpf_native.c and into ebpf_program.c. Let programs own provider registration and state, and have native query program-owned address resolution during load and callback updates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5bf8544a-198f-4000-a991-78c35a2a41aa Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Keep BTF-resolved function change callbacks under the program lock during notification, and update the sample/test BTF contract so the output buffer argument is declared writable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5bf8544a-198f-4000-a991-78c35a2a41aa Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5bf8544a-198f-4000-a991-78c35a2a41aa Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Tighten BTF hash-entry cleanup and population so the execution-context analysis build passes without the prior false-positive failure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 5bf8544a-198f-4000-a991-78c35a2a41aa Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 77d49326-fb50-4f9c-be14-e2962aff3ed1 Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Clear the thread-local verifier program-info cache before preprocessing a new ELF in load_byte_code so BTF-resolved function metadata from a previous object cannot leak into a later load on the same thread. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 77d49326-fb50-4f9c-be14-e2962aff3ed1 Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Remove the unused BTF-resolved-function zero marker from the native metadata entry layout and update bpf2c output accordingly. Add verifier unit coverage that same-name BTF-resolved functions from different providers resolve correctly across separate compilation units, and create follow-up issue microsoft#5443 for subprogram coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 77d49326-fb50-4f9c-be14-e2962aff3ed1 Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 77d49326-fb50-4f9c-be14-e2962aff3ed1 Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Extract the shared BTF test store helpers used by both unit tests, restore the previous no-op bpf2c formatting to reduce review noise, and make the provider-ready flag assignment consistent with the other byte-sized writes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f2c90d32-f29b-42e3-b2c5-8f4df581d20a Signed-off-by: Alan Jowett <alanjo@microsoft.com>

Accept the pre-commit formatter's namespace brace placement in the shared BTF test helper header. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: f2c90d32-f29b-42e3-b2c5-8f4df581d20a Signed-off-by: Alan Jowett <alanjo@microsoft.com>

mikeagun · 2026-07-16T22:59:53Z

            *infos = info;
            info = nullptr;
        }
    } catch (std::runtime_error e) {


This noexcept function only catches std::runtime_error, but it looks like _preprocess_btf_resolved_functions introduces additional possible exceptions. Please add catch (const std::exception&) here to match them (and/or otherwise handle line 841 and any other lines that might throw other exceptions).

Other possible exception sources:

std::bad_alloc during BTF parsing

the libbtf::btf_type_data construction at line 840

unguarded get_kind_type<btf_kind_decl_tag> in cache_btf_resolved_functions (windows_platform_common.cpp:947, unlike the guarded calls at 928/975).

Other entry points (verify / verify_from_stream / from-file) also catch std::exception.

mikeagun · 2026-07-16T23:23:23Z

 #define EBPF_NATIVE_MAP_DATA_SIZE_1 EBPF_SIZE_INCLUDING_FIELD(map_data_t, array_data)
 size_t _ebpf_native_map_data_supported_size[] = {EBPF_NATIVE_MAP_DATA_SIZE_0, EBPF_NATIVE_MAP_DATA_SIZE_1};

 #define EBPF_NATIVE_PROGRAM_ENTRY_SIZE_0 EBPF_SIZE_INCLUDING_FIELD(program_entry_t, program_info_hash_type)


I think a BTF module can hit a kernel OOB when loaded on a runtime that predates BTF, because the program_entry doesn't declare the BTF ABI as required.

A BTF program's program_entry still stamps size = SIZE_0 (144) — e.g. btf_resolved_dll.c emits {1, 144, 160} — even though its code dereferences runtime_context->btf_resolved_function_data. So an older runtime:

accepts it ( size 144 == 144 ; the only version gate is the lower-bound _ebpf_minimum_version, no upper bound),

drops the trailing BTF fields in _ebpf_copy_program_entry ( min(160, sizeof=144) ), loading it as a plain native program,

then on invoke calls btf_resolved_function_data[0].address(...) past its smaller runtime_context → OOB + indirect call through garbage.

map_data_v2 already handles the equivalent case by adding a SIZE_1 so older runtimes reject it at load.

One option: do the same here — add EBPF_NATIVE_PROGRAM_ENTRY_SIZE_1 = SIZE_INCLUDING_FIELD(program_entry_t, btf_resolved_function_count), stamp it only for program entries that use BTF, and gate BTF-field use on size >= SIZE_1. (Bumping the program_entry version to 2 for BTF programs would also work via the validator's default: return false , but the size approach avoids rejecting non-BTF modules on older runtimes.)

mikeagun · 2026-07-17T01:45:44Z

+            return false;
+        }
+
+        size_t function_prototype_size = function_prototype_array[0].header.total_size;


Should we validate total_size here?

dthaler · 2026-07-17T04:47:34Z

+
+static void
+_ebpf_program_free_btf_hash_entries(
+    _In_ size_t btf_hash_entry_count, _Frees_ptr_opt_ ebpf_program_btf_hash_entry_t* btf_hash_entries)


Suggested change

_In_ size_t btf_hash_entry_count, _Frees_ptr_opt_ ebpf_program_btf_hash_entry_t* btf_hash_entries)

_In_ size_t btf_hash_entry_count, _In_reads_opt_(btf_hash_entry_count) _Frees_ptr_opt_ ebpf_program_btf_hash_entry_t* btf_hash_entries)

Can both annotations be added to btf_hash_entries?

dthaler · 2026-07-17T04:48:32Z

+}
+
+_Requires_lock_held_(program->lock) static ebpf_result_t
+    _ebpf_program_initialize_btf_provider_bindings(_Inout_ ebpf_program_t* program)


Suggested change

_ebpf_program_initialize_btf_provider_bindings(_Inout_ ebpf_program_t* program)

_ebpf_program_initialize_btf_provider_bindings(_Inout_ ebpf_program_t* program)

Why the indentation?

dthaler · 2026-07-17T04:49:16Z

+    ebpf_free(local_providers);
+    return result;
+}
 _IRQL_requires_max_(PASSIVE_LEVEL) static ebpf_result_t _ebpf_program_compute_program_information_hash(


Suggested change

_IRQL_requires_max_(PASSIVE_LEVEL) static ebpf_result_t _ebpf_program_compute_program_information_hash(

_IRQL_requires_max_(PASSIVE_LEVEL) static ebpf_result_t _ebpf_program_compute_program_information_hash(

nit: add blank line between functions

dthaler · 2026-07-17T04:51:33Z

+    state = ebpf_lock_lock(&program->lock);
+    _ebpf_program_clear_btf_resolved_function_entries(program);
+    ebpf_lock_unlock(&program->lock, state);
+}


Suggested change

}

EBPF_LOG_EXIT();

}

Need an exit to go with the entry in line 2715

Alan-Jowett force-pushed the issue1901 branch 3 times, most recently from 4eb9522 to 91aa968 Compare June 9, 2026 00:08

Alan-Jowett force-pushed the issue1901 branch 2 times, most recently from 0cbe258 to b6edd12 Compare June 11, 2026 22:01

Alan-Jowett changed the title ~~Add BTF registry publication store helper support~~ Implement BTF-resolved function support Jun 11, 2026

Alan-Jowett force-pushed the issue1901 branch from 2690bb8 to aa86352 Compare June 12, 2026 02:05

Alan-Jowett requested a review from Copilot June 12, 2026 02:07

Copilot AI reviewed Jun 12, 2026

Alan-Jowett requested a review from Copilot June 12, 2026 03:26

Copilot started reviewing on behalf of Alan-Jowett June 12, 2026 03:27 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread include/ebpf_extension_uuids.h Outdated

Comment thread libs/execution_context/ebpf_core.c Outdated

Comment thread libs/service/api_service.cpp Outdated

Comment thread libs/service/api_service.cpp Outdated

Comment thread tools/bpf2c/Convert-BpfToNative.ps1.template

Alan-Jowett requested a review from Copilot June 12, 2026 05:09

Copilot started reviewing on behalf of Alan-Jowett June 12, 2026 05:10 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread libs/execution_context/ebpf_core_jit.c Outdated

Comment thread libs/execution_context/ebpf_core_jit.c Outdated

Alan-Jowett requested a review from Copilot June 12, 2026 15:39

Copilot started reviewing on behalf of Alan-Jowett June 12, 2026 15:40 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread undocked/tests/sample/ext/drv/sample_ext_drv.c

Comment thread tests/bpf2c_tests/bpf_test.cpp

Alan-Jowett requested a review from Copilot June 12, 2026 16:21

Copilot started reviewing on behalf of Alan-Jowett June 12, 2026 16:22 View session

Copilot AI reviewed Jun 12, 2026

View reviewed changes

Comment thread tests/bpf2c_tests/bpf_test.cpp

Alan-Jowett commented Jun 13, 2026

View reviewed changes

Comment thread libs/execution_context/ebpf_protocol.h Outdated