feat: cross-language LTO to inline C TLS shim into Rust FFI#1982
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 18 commits intoJun 2, 2026
Conversation
Contributor
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1982 +/- ##
==========================================
- Coverage 73.40% 73.38% -0.03%
==========================================
Files 464 465 +1
Lines 77970 77985 +15
==========================================
- Hits 57236 57231 -5
- Misses 20734 20754 +20
🚀 New features to boost your workflow:
|
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 4eb82b7 | Docs | Datadog PR Page | Give us feedback! |
yannham
force-pushed
the
yannham/thread-ctx-inlining-exp
branch
4 times, most recently
from
9932cac to
9227252
Compare
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
yannham
commented
May 18, 2026
| let target_os = env::var("CARGO_CFG_TARGET_OS").unwrap(); | ||
| let target_arch = env::var("CARGO_CFG_TARGET_ARCH").unwrap(); | ||
|
|
||
| if target_os != "linux" { |
Contributor
Author
There was a problem hiding this comment.
This change is somehow unrelated, but using static cfg(target_os = "linux") is actually not the way to go in a build script (during cross-compilation typically, the target_os of the script and of the dynamic library could differ), so fixing it passing by.
Contributor
Author
|
@codex review |
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
ivoanjo
reviewed
May 21, 2026
ivoanjo
left a comment
ivoanjo
left a comment
Member
There was a problem hiding this comment.
I really like this optimization, left a few comments since I'm a bit... confused about some of the details :D
scottgerring
left a comment
scottgerring
left a comment
Member
There was a problem hiding this comment.
Nice - great to see the improvements to :D
yannham
force-pushed
the
yannham/thread-ctx-inlining-exp
branch
from
8f6c82e to
31a7e9c
Compare
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: waiting
mergequeue-status: removed
and removed
mergequeue-status: waiting
labels
hoolioh
approved these changes
Jun 2, 2026
hoolioh
left a comment
hoolioh
left a comment
Contributor
There was a problem hiding this comment.
LGTM. One caveat, probably the possibilities are scarce but I think it'd be worth to have an integration CI job that builds inline mode and asserts the invariants for the supported platforms. Since this depends on the external toolchains there could be situations in which the machinery could not work, we would silently default to not inlining the function and cause a regression. That said, I think it could be done in a following PR
Add an opt-in inline mode via the LIBDD_OTEL_THREAD_CTX_INLINE env var that uses cross-language LTO (clang + lld) to inline the C TLS shim directly into the Rust FFI functions, eliminating a function-call indirection on every TLS access. When the env var is set, build.rs validates that clang and a suitable LLD are available, compiles the C shim as LLVM bitcode (-flto=thin), and emits the linker flags needed for cross-language LTO. A companion build-optimized.sh wrapper sets the target-scoped RUSTFLAGS and env var. When the env var is absent, the previous behavior is preserved: the default cc compiles the shim with -mtls-dialect=gnu2 on x86-64, no LTO, no inlining. Also fixes #[cfg(target_os/arch)] in build scripts to use the correct CARGO_CFG_* env vars for cross-compilation correctness. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After a successful build, verify with nm that the C TLS shim symbol was actually inlined away. Warns and exits 1 if inlining failed, or warns (but succeeds) if nm is not available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deduplicate the function that locates the toolchain's bundled rust-lld by moving it into build_common, where both build scripts can reuse it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Scott Gerring <scottgerring@users.noreply.github.com>
Co-Authored-By: Claude <noreply@anthropic.com>
yannham
force-pushed
the
yannham/thread-ctx-inlining-exp
branch
from
1a5b30e to
4eb82b7
Compare
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
yannham/thread-ctx-inlining-exp
branch
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
that referenced
this pull request
Jun 19, 2026
# Release v36.0.0 This release bumps the workspace version `35.0.0 → 36.0.0`. Below are the commits in `v35.0.0..HEAD` that directly modify the C ABI surface consumed by downstream SDKs. ## Major | Commit | FFI crate(s) affected | |---|---| | `refactor(shm)!: Extract one_way_shared_memory to IPC and prepare libdd-remote-config for python` (#2121) | `datadog-sidecar-ffi` | | `refactor(span)!: use VecMap for meta, metrics and meta_struct for v04 spans` (#2043) | `datadog-sidecar-ffi`, `libdd-data-pipeline-ffi` | | `feat(data-pipeline)!: add fork safety hooks and cancellation token for trace exporter FFI` (#2051) | `libdd-data-pipeline-ffi`, `libdd-profiling-ffi` | ## Minor | Commit | FFI crate(s) affected | |---|---| | `feat(sidecar): add retry interval configuration` (#2106) | `datadog-sidecar-ffi` | | `feat(profiling): Add setting to omit local root span id from serialized pprof` (#2104) | `libdd-profiling-ffi` | | `feat(live-debugger): agentless intake forwarding` (#2075) | `datadog-live-debugger-ffi` | | `feat(sidecar): forward FFE exposures to EVP proxy` (#2026) | `datadog-sidecar-ffi` | | `feat(sidecar): forward FFE evaluation metrics to OTLP intake` (#2052) | `datadog-sidecar-ffi` | | `feat: cross-language LTO to inline C TLS shim into Rust FFI` (#1982) | `libdd-otel-thread-ctx-ffi` (build-only: build.rs / scripts / README) | ## Patch | Commit | FFI crate(s) affected | |---|---| | `fix(ffe): honor shared fixture result metadata` (#2109) | `datadog-ffe-ffi` | | `fix(sidecar): Dedup VecMap spans before serialization` (#2107) | `datadog-sidecar-ffi` | | `fix(crashtracking): authenticate peer granted socket ptrace access` (#2098) | `libdd-crashtracker-ffi`, `datadog-sidecar-ffi` | | `fix(remote-config): notification of multi-processing and runtime deduplication` (#2082) | `datadog-sidecar-ffi` | | `fix(sidecar): configure OTLP endpoint for FFE metrics` (#2076) | `datadog-sidecar-ffi` | | `refactor(datadog-remote-config): rename as libdd-remote-config` (#2085) | `datadog-sidecar-ffi` (incl. `cbindgen.toml`) | Co-authored-by: julio.gonzalez <julio.gonzalez@datadoghq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Add opt-in cross-language LTO so the C TLS shim (
libdd_get_otel_thread_ctx_v1) is inlined directly into the Rust FFI functions, eliminating a function-call indirection on every TLS access.The mode is controlled by the
LIBDD_OTEL_THREAD_CTX_INLINEenv var. Abuild-optimized.shwrapper script sets the env var and the target-scopedRUSTFLAGSneeded for cross-language LTO. Without the env var, the build behaves exactly as before.Motivation
The OTel thread-level context spec requires TLSDESC for the TLS variable, which forces us through a C shim since stable
rustcdoesn't expose TLS dialect control. This adds a function call on everyattach/detach. Cross-language LTO eliminates that call.Benchmark results (median of 3 batch runs on x86-64):
Additional Notes
clangandlld(the toolchain's bundledrust-lldis used automatically).build.rscannot set rustc codegen flags (-Clinker-plugin-lto,-Clinker=clang). Those must come fromRUSTFLAGS.#[cfg(target_os/arch)]in build scripts are replaced with the correctCARGO_CFG_*env vars for cross-compilation correctness.How to test the change?