Skip to content

fix(remote-config): notification of multi-processing and runtime deduplication#2082

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
bob/fix-rc-race
Jun 4, 2026
Merged

fix(remote-config): notification of multi-processing and runtime deduplication#2082
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
bob/fix-rc-race

Conversation

@bwoebi

@bwoebi bwoebi commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

We want the session_id as runtime_id key, but still deduplicate processes according to their runtime_id, otherwise notifcation will not reach all targeted processes.

Additionally submit the remote_config_generation, so that processes which start up and have read the present configuration, get notified when a new configuration has intermittently been read. (race condition)

@bwoebi bwoebi requested review from a team as code owners June 4, 2026 17:42
@datadog-official

datadog-official Bot commented Jun 4, 2026

Copy link
Copy Markdown

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

pr-name | pr_name_lint   View in Datadog   GitHub Actions

See error PR Title Validation Failed: subject and type may not be empty. Expected format: type(scope): description.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 48.60%
Overall Coverage: 73.43% (-0.07%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 3c76d63 | Docs | Datadog PR Page | Give us feedback!

@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

  • Base Branch: origin/main
  • PR Branch: origin/bob/fix-rc-race

Summary by Rule

Rule Base Branch PR Branch Change
expect_used 5 5 No change (0%)
unwrap_used 13 13 No change (0%)
Total 18 18 No change (0%)

Annotation Counts by File

File Base Branch PR Branch Change
datadog-remote-config/src/fetch/multitarget.rs 1 1 No change (0%)
datadog-sidecar/src/one_way_shared_memory.rs 1 1 No change (0%)
datadog-sidecar/src/service/blocking.rs 1 1 No change (0%)
datadog-sidecar/src/service/runtime_info.rs 1 1 No change (0%)
datadog-sidecar/src/service/sidecar_server.rs 6 6 No change (0%)
datadog-sidecar/src/shm_remote_config.rs 8 8 No change (0%)

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 21 21 No change (0%)
datadog-live-debugger 6 6 No change (0%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-remote-config 4 4 No change (0%)
datadog-sidecar 58 58 No change (0%)
libdd-common 13 13 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-data-pipeline 5 5 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 13 No change (0%)
libdd-telemetry 20 20 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 3 3 No change (0%)
libdd-trace-stats 1 1 No change (0%)
libdd-trace-utils 13 13 No change (0%)
Total 198 198 No change (0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

@bwoebi bwoebi force-pushed the bob/fix-rc-race branch from a3759f7 to 964e3ea Compare June 4, 2026 17:45

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a3759f79d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +487 to +488
instance_id.session_id.clone(),
instance_id.runtime_id.clone(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep sessions distinct when deduplicating runtimes

Passing instance_id.runtime_id here makes MultiTargetFetcher::add_runtime key its runtimes map only by the runtime id. When two forked/multiprocess sidecar sessions report the same runtime id for the same target, the second registration takes the occupied-target path and only increments a count, so its NotifyTarget/session id is never stored; later fetch notifications iterate that single runtime entry and only signal one process. This regresses the multi-process case that previously keyed registrations by the unique session id, so the fetcher needs to distinguish at least (session_id, runtime_id) while still deduplicating queues within one runtime.

Useful? React with 👍 / 👎.

@bwoebi bwoebi force-pushed the bob/fix-rc-race branch 2 times, most recently from 994d3d9 to 38d448a Compare June 4, 2026 17:59
@codecov-commenter

codecov-commenter commented Jun 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 48.59813% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.42%. Comparing base (a97e1d4) to head (3c76d63).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2082      +/-   ##
==========================================
- Coverage   73.55%   73.42%   -0.14%     
==========================================
  Files         475      475              
  Lines       78799    78919     +120     
==========================================
- Hits        57964    57948      -16     
- Misses      20835    20971     +136     
Components Coverage Δ
libdd-crashtracker 65.32% <ø> (-0.02%) ⬇️
libdd-crashtracker-ffi 37.68% <ø> (ø)
libdd-agent-client 83.79% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 86.08% <ø> (-1.06%) ⬇️
libdd-data-pipeline-ffi 73.02% <ø> (-4.02%) ⬇️
libdd-common 79.93% <ø> (ø)
libdd-common-ffi 74.41% <ø> (ø)
libdd-telemetry 73.37% <ø> (+0.02%) ⬆️
libdd-telemetry-ffi 31.36% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 74.90% <ø> (ø)
libdd-profiling 81.68% <ø> (-0.02%) ⬇️
libdd-profiling-ffi 64.79% <ø> (ø)
libdd-sampling 97.41% <ø> (ø)
datadog-sidecar 36.14% <29.03%> (-0.11%) ⬇️
datdog-sidecar-ffi 10.31% <0.00%> (-0.02%) ⬇️
spawn-worker 48.86% <ø> (ø)
libdd-tinybytes 93.80% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 87.30% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 89.38% <ø> (+0.09%) ⬆️
libdd-tracer-flare 86.57% <ø> (ø)
libdd-log 74.83% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

We want the session_id as runtime_id key, but still deduplicate *processes* according to their runtime_id, otherwise notifcation will not reach all targeted processes.

Additionally submit the remote_config_generation, so that processes which start up and have read the present configuration, get notified when a new configuration has intermittently been read. (race condition)

Signed-off-by: Bob Weinand <bob.weinand@datadoghq.com>
@bwoebi bwoebi force-pushed the bob/fix-rc-race branch from 38d448a to 3c76d63 Compare June 4, 2026 18:17
@bwoebi bwoebi changed the title Fix notification of multi-processing and runtime deduplication fix(remote-config): notification of multi-processing and runtime deduplication Jun 4, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 82.85 MB 82.85 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.63 MB 7.63 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.25 MB 10.25 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 93.93 MB 93.93 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 24.53 MB 24.53 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 83.96 KB 83.96 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 179.02 MB 179.02 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 919.85 MB 919.85 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.02 MB 8.02 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 83.96 KB 83.96 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 23.75 MB 23.75 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 47.38 MB 47.38 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 21.25 MB 21.25 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 85.29 KB 85.29 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 182.96 MB 182.96 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 912.59 MB 912.59 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.19 MB 6.19 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 85.29 KB 85.29 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 25.45 MB 25.45 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 45.04 MB 45.04 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 73.87 MB 73.87 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.52 MB 8.52 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 89.28 MB 89.28 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.35 MB 10.35 MB 0% (0 B) 👌

@morrisonlevi morrisonlevi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent some time with Codex trying to understand it. I looks okay to me based on my weak understanding. I think Session ID == Runtime ID for runtimes which correctly regenerated the ID on fork, and the Root/Parent Session ID are related to tracking the parent/child relationships. Based on the idea that Session ID == Runtime ID, this looks fine. I will certainly not claim to be an expert here!

@bwoebi

bwoebi commented Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jun 4, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-06-04 21:33:05 UTC ℹ️ Start processing command /merge


2026-06-04 21:33:11 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 48m (p90).


2026-06-04 22:15:57 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit a76412c into main Jun 4, 2026
98 of 101 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the bob/fix-rc-race branch June 4, 2026 22:15
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Jun 19, 2026
# Release v36.0.0 
This release bumps the workspace version `35.0.0 → 36.0.0`. Below are the commits in `v35.0.0..HEAD` that directly modify the C ABI surface consumed by downstream SDKs.

## Major

| Commit | FFI crate(s) affected |
|---|---|
| `refactor(shm)!: Extract one_way_shared_memory to IPC and prepare libdd-remote-config for python` (#2121) | `datadog-sidecar-ffi` |
| `refactor(span)!: use VecMap for meta, metrics and meta_struct for v04 spans` (#2043) | `datadog-sidecar-ffi`, `libdd-data-pipeline-ffi` |
| `feat(data-pipeline)!: add fork safety hooks and cancellation token for trace exporter FFI` (#2051) | `libdd-data-pipeline-ffi`, `libdd-profiling-ffi` |

## Minor

| Commit | FFI crate(s) affected |
|---|---|
| `feat(sidecar): add retry interval configuration` (#2106) | `datadog-sidecar-ffi` |
| `feat(profiling): Add setting to omit local root span id from serialized pprof` (#2104) | `libdd-profiling-ffi` |
| `feat(live-debugger): agentless intake forwarding` (#2075) | `datadog-live-debugger-ffi` |
| `feat(sidecar): forward FFE exposures to EVP proxy` (#2026) | `datadog-sidecar-ffi` |
| `feat(sidecar): forward FFE evaluation metrics to OTLP intake` (#2052) | `datadog-sidecar-ffi` |
| `feat: cross-language LTO to inline C TLS shim into Rust FFI` (#1982) | `libdd-otel-thread-ctx-ffi` (build-only: build.rs / scripts / README) |

## Patch

| Commit | FFI crate(s) affected |
|---|---|
| `fix(ffe): honor shared fixture result metadata` (#2109) | `datadog-ffe-ffi` |
| `fix(sidecar): Dedup VecMap spans before serialization` (#2107) | `datadog-sidecar-ffi` |
| `fix(crashtracking): authenticate peer granted socket ptrace access` (#2098) | `libdd-crashtracker-ffi`, `datadog-sidecar-ffi` |
| `fix(remote-config): notification of multi-processing and runtime deduplication` (#2082) | `datadog-sidecar-ffi` |
| `fix(sidecar): configure OTLP endpoint for FFE metrics` (#2076) | `datadog-sidecar-ffi` |
| `refactor(datadog-remote-config): rename as libdd-remote-config` (#2085) | `datadog-sidecar-ffi` (incl. `cbindgen.toml`) |



Co-authored-by: julio.gonzalez <julio.gonzalez@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants