Skip to content

feat(live-debugger): agentless intake forwarding#2075

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
mainfrom
tyler.finethy/agentless-debugger-proxy
Jun 8, 2026
Merged

feat(live-debugger): agentless intake forwarding#2075
gh-worker-dd-mergequeue-cf854d[bot] merged 4 commits into
mainfrom
tyler.finethy/agentless-debugger-proxy

Conversation

@tylfin

@tylfin tylfin commented Jun 3, 2026

Copy link
Copy Markdown
Member

What

Adds a transparent path for tracers to send Live Debugger and Symbol Database (SymDB) data directly to the Datadog intake when no agent is present, matching the behavior of the agent's debugger proxy routes (/debugger/v1/diagnostics, /debugger/v2/input, /symdb/v1/input). This extends the existing datadog-live-debugger crate and its FFI rather than adding a new crate.

Why

In agentless mode the tracer has no local agent to enrich and forward debugger/SymDB payloads to intake. This gives libdatadog the agent's forwarding behavior so tracers (e.g. dd-trace) can ship the same payloads transparently, choosing agent vs. agentless based on whether an API key is present.

Changes

Core (datadog-live-debugger/src/sender.rs)

  • debugger_intake_endpoint(site, api_key) builds an agentless endpoint targeting debugger-intake.{site} (mirrors profiling's agentless()).
  • Factored the per-track path derivation into derive_endpoint_path(); agentless requests use /api/v2/debugger, agent requests use the proxy paths. file:// endpoints (tests) are left untouched.
  • Config gained symdb_endpoint plus dual-ship endpoint vectors, with set_symdb_endpoint, add_additional_debugger_endpoint, and add_additional_symdb_endpoint.
  • send() and send_symdb() send the primary endpoint first, then fan out best-effort to any additional endpoints (mirroring the agent's *_additional_endpoints, where non-primary responses are discarded). Sending the primary first keeps a slow or stalled additional endpoint from delaying the intake the caller depends on.
  • send_symdb() forwards raw bytes verbatim with DD-EVP-ORIGIN: agent-symdb and tags carried in the X-Datadog-Additional-Tags header (matching symdb.go).
  • PayloadSender::new split into new / new_to_endpoint; existing callers (e.g. datadog-sidecar) are unaffected.

FFI (datadog-live-debugger-ffi/src/sender.rs)

  • ddog_live_debugger_endpoint_from_site_and_api_key for the agentless endpoint (freed via the existing ddog_endpoint_drop).
  • A config-builder surface (..._sender_config_new / ..._set_endpoint / ..._set_symdb_endpoint / ..._add_additional_endpoint / ..._add_additional_symdb_endpoint / ..._drop) and ddog_live_debugger_spawn_sender_with_config. The existing ddog_live_debugger_spawn_sender is unchanged.
  • ddog_live_debugger_send_symdb_data to enqueue a raw SymDB body (zero-copy across the boundary via OwnedCharSlice).
  • The sender routine logs the track name via a static &str instead of formatting one per send, dropping an allocation on the hot path.

DebuggerType is intentionally left unchanged (SymDB is handled as a separate path) because datadog-sidecar does an exhaustive match on it.

Testing

  • cargo check on both crates and datadog-sidecar
  • cargo +nightly-2026-02-08 fmt --check, cargo clippy --all-targets --all-features -D warnings
  • cargo test -p datadog-live-debugger -p datadog-live-debugger-ffi (14 pass, 4 new)
  • cbindgen header verified; cargo ffi-test (only pre-existing crashtracking/ffe env failures, unrelated to this change)
  • No new dependencies; Cargo.lock untouched.

@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

  • Base Branch: origin/main
  • PR Branch: origin/tyler.finethy/agentless-debugger-proxy

Summary by Rule

Rule Base Branch PR Branch Change
unwrap_used 4 2 ✅ -2 (-50.0%)
Total 4 2 ✅ -2 (-50.0%)

Annotation Counts by File

File Base Branch PR Branch Change
datadog-live-debugger/src/sender.rs 4 2 ✅ -2 (-50.0%)

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 21 21 No change (0%)
datadog-live-debugger 6 4 ✅ -2 (-33.3%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-remote-config 4 4 No change (0%)
datadog-sidecar 58 58 No change (0%)
libdd-common 13 13 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-data-pipeline 5 5 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 13 No change (0%)
libdd-telemetry 20 20 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 3 3 No change (0%)
libdd-trace-stats 1 1 No change (0%)
libdd-trace-utils 13 13 No change (0%)
Total 198 196 ✅ -2 (-1.0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 32.17%
Overall Coverage: 73.33% (-0.17%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a38ce35 | Docs | Datadog PR Page | Give us feedback!

@codecov-commenter

codecov-commenter commented Jun 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 32.16783% with 194 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.32%. Comparing base (2a6c295) to head (a38ce35).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2075      +/-   ##
==========================================
- Coverage   73.49%   73.32%   -0.17%     
==========================================
  Files         475      475              
  Lines       78839    79085     +246     
==========================================
+ Hits        57944    57993      +49     
- Misses      20895    21092     +197     
Components Coverage Δ
libdd-crashtracker 65.35% <ø> (+0.01%) ⬆️
libdd-crashtracker-ffi 37.68% <ø> (ø)
libdd-agent-client 83.79% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 86.08% <ø> (ø)
libdd-data-pipeline-ffi 73.02% <ø> (ø)
libdd-common 79.93% <ø> (ø)
libdd-common-ffi 74.41% <ø> (ø)
libdd-telemetry 73.37% <ø> (+0.02%) ⬆️
libdd-telemetry-ffi 31.36% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 76.36% <ø> (ø)
libdd-profiling 81.69% <ø> (ø)
libdd-profiling-ffi 64.79% <ø> (ø)
libdd-sampling 97.41% <ø> (ø)
datadog-sidecar 36.04% <ø> (-0.21%) ⬇️
datdog-sidecar-ffi 9.33% <ø> (-1.00%) ⬇️
spawn-worker 48.86% <ø> (ø)
libdd-tinybytes 93.80% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 87.30% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 89.29% <ø> (ø)
libdd-tracer-flare 86.57% <ø> (ø)
libdd-log 74.83% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dd-octo-sts

dd-octo-sts Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 83.49 MB 83.49 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.70 MB 7.70 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 94.58 MB 94.58 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.34 MB 10.34 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 24.73 MB 24.73 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 86.89 KB 86.89 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 179.89 MB 179.91 MB +0% (+16.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 923.17 MB 923.17 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.09 MB 8.09 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 86.89 KB 86.89 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 23.91 MB 23.91 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 47.72 MB 47.72 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 21.43 MB 21.43 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 88.26 KB 88.26 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 183.87 MB 183.86 MB -0% (-8.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 915.93 MB 915.93 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.25 MB 6.25 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 88.26 KB 88.26 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 25.63 MB 25.63 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 45.35 MB 45.35 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 74.45 MB 74.45 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.58 MB 8.58 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 89.86 MB 89.86 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.43 MB 10.43 MB 0% (0 B) 👌

@tylfin tylfin requested review from andreimatei, bwoebi and ojung June 4, 2026 17:51
@tylfin tylfin marked this pull request as ready for review June 4, 2026 17:58
@tylfin tylfin requested review from a team as code owners June 4, 2026 17:58

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7f3440d544

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread datadog-live-debugger/src/sender.rs Outdated
Comment thread datadog-live-debugger/src/sender.rs
Comment thread datadog-live-debugger-ffi/src/sender.rs Outdated
tylfin added 3 commits June 4, 2026 14:22
Add a transparent path for tracers to send Live Debugger and Symbol
Database (SymDB) data directly to the Datadog intake when no agent is
present, matching what the agent's debugger proxy routes do.

- Build agentless intake endpoints from a site and API key, targeting
  debugger-intake.{site} for diagnostics, snapshots and SymDB uploads.
- Forward SymDB payloads with the appropriate origin and tags, mirroring
  the agent's /symdb/v1/input route.
- Dual-ship payloads to additional intake endpoints when configured.
- Expose a sender config builder and SymDB send entry point over the FFI;
  the existing sender entry point is unchanged.
Set DD-EVP-ORIGIN on the SymDB request only when an API key is present,
matching the debugger tracks. The origin is only meaningful on the direct
intake; in agent mode the agent sets it while proxying. No behavior change
in agentless mode.
Send the primary debugger and SymDB endpoints before fanning out to
best-effort additional endpoints so a slow or stalled extra endpoint no
longer delays the intake the caller depends on. Drop a per-send string
allocation on the FFI send path.
@tylfin tylfin force-pushed the tyler.finethy/agentless-debugger-proxy branch from 7f3440d to e29975a Compare June 4, 2026 18:26
Comment thread datadog-live-debugger/src/sender.rs Outdated

@bwoebi bwoebi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from the sequential await, looks good to me.

Send the primary and additional dual-ship endpoints concurrently in
send() and send_symdb() instead of awaiting them sequentially, so a slow
extra endpoint no longer delays the others. Keep the primary's result as
the returned one. Avoid a per-send heap allocation by collecting endpoints
into a stack-inlined SmallVec, and derive the agentless SymDB route from
the diagnostics constant so the intentional path reuse stays in lockstep.
@tylfin tylfin requested a review from a team as a code owner June 4, 2026 19:29
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit cfb46bd into main Jun 8, 2026
89 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the tyler.finethy/agentless-debugger-proxy branch June 8, 2026 19:17
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Jun 19, 2026
# Release v36.0.0 
This release bumps the workspace version `35.0.0 → 36.0.0`. Below are the commits in `v35.0.0..HEAD` that directly modify the C ABI surface consumed by downstream SDKs.

## Major

| Commit | FFI crate(s) affected |
|---|---|
| `refactor(shm)!: Extract one_way_shared_memory to IPC and prepare libdd-remote-config for python` (#2121) | `datadog-sidecar-ffi` |
| `refactor(span)!: use VecMap for meta, metrics and meta_struct for v04 spans` (#2043) | `datadog-sidecar-ffi`, `libdd-data-pipeline-ffi` |
| `feat(data-pipeline)!: add fork safety hooks and cancellation token for trace exporter FFI` (#2051) | `libdd-data-pipeline-ffi`, `libdd-profiling-ffi` |

## Minor

| Commit | FFI crate(s) affected |
|---|---|
| `feat(sidecar): add retry interval configuration` (#2106) | `datadog-sidecar-ffi` |
| `feat(profiling): Add setting to omit local root span id from serialized pprof` (#2104) | `libdd-profiling-ffi` |
| `feat(live-debugger): agentless intake forwarding` (#2075) | `datadog-live-debugger-ffi` |
| `feat(sidecar): forward FFE exposures to EVP proxy` (#2026) | `datadog-sidecar-ffi` |
| `feat(sidecar): forward FFE evaluation metrics to OTLP intake` (#2052) | `datadog-sidecar-ffi` |
| `feat: cross-language LTO to inline C TLS shim into Rust FFI` (#1982) | `libdd-otel-thread-ctx-ffi` (build-only: build.rs / scripts / README) |

## Patch

| Commit | FFI crate(s) affected |
|---|---|
| `fix(ffe): honor shared fixture result metadata` (#2109) | `datadog-ffe-ffi` |
| `fix(sidecar): Dedup VecMap spans before serialization` (#2107) | `datadog-sidecar-ffi` |
| `fix(crashtracking): authenticate peer granted socket ptrace access` (#2098) | `libdd-crashtracker-ffi`, `datadog-sidecar-ffi` |
| `fix(remote-config): notification of multi-processing and runtime deduplication` (#2082) | `datadog-sidecar-ffi` |
| `fix(sidecar): configure OTLP endpoint for FFE metrics` (#2076) | `datadog-sidecar-ffi` |
| `refactor(datadog-remote-config): rename as libdd-remote-config` (#2085) | `datadog-sidecar-ffi` (incl. `cbindgen.toml`) |



Co-authored-by: julio.gonzalez <julio.gonzalez@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants