Skip to content

feat(data-pipeline)!: CSS Trace Filters#1985

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 54 commits into
mainfrom
oscarld/APMSP-2763-css-trace-filtering
Jun 16, 2026
Merged

feat(data-pipeline)!: CSS Trace Filters#1985
gh-worker-dd-mergequeue-cf854d[bot] merged 54 commits into
mainfrom
oscarld/APMSP-2763-css-trace-filtering

Conversation

@Eldolfin

@Eldolfin Eldolfin commented May 13, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Implements the CSS trace-level filtering mechanism, that is applied before stats computation.

Motivation

It's un unmet requirement from the spec.

Additional Notes

Please Ctrl+F FIXME when reviewing, I let some questions in the code that I'd like answers for before merging

TODO:

  • unit tests
  • system-tests Add trace filter tests [APMSP-2763] system-tests#6952
  • normalization before filtering
  • regex keys (currently only regex values are checked)
  • new spec or CSS spec RFC
  • telemetry ? Count dropped traces with the reason
  • trim spaces when parsing config
  • Move implementation to trace-utils
  • stop using v4 span but generic spans instead for php
  • move arcswap to the trace exporter (remove update_conf)
  • use test agent for snapshot tests

Wont do

  • advanced resource normalization (sql, redis etc...). No other implementation does that
  • 100% normalization edge cases (empty service field + non-normalized name + filter depending on that) Too specific edge case. Ignoring it for now
  • check on meta_struct too Not in the spec

APMSP-2763

@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

  • Base Branch: origin/main
  • PR Branch: origin/oscarld/APMSP-2763-css-trace-filtering

Summary by Rule

Rule Base Branch PR Branch Change
todo 2 2 No change (0%)
unwrap_used 4 4 No change (0%)
Total 6 6 No change (0%)

Annotation Counts by File

File Base Branch PR Branch Change
libdd-data-pipeline/src/telemetry/mod.rs 1 1 No change (0%)
libdd-data-pipeline/src/trace_exporter/mod.rs 2 2 No change (0%)
libdd-trace-normalization/src/normalize_utils.rs 1 1 No change (0%)
libdd-trace-utils/src/trace_utils.rs 2 2 No change (0%)

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 21 21 No change (0%)
datadog-live-debugger 4 4 No change (0%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-sidecar 46 46 No change (0%)
libdd-common 13 13 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-data-pipeline 5 5 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 13 No change (0%)
libdd-remote-config 3 3 No change (0%)
libdd-telemetry 20 20 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 3 3 No change (0%)
libdd-trace-stats 1 1 No change (0%)
libdd-trace-utils 12 12 No change (0%)
Total 182 182 No change (0%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

@codecov-commenter

codecov-commenter commented May 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.94799% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.71%. Comparing base (a1da9fc) to head (4ea70a8).
⚠️ Report is 16 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1985      +/-   ##
==========================================
+ Coverage   73.44%   73.71%   +0.26%     
==========================================
  Files         465      476      +11     
  Lines       77949    79647    +1698     
==========================================
+ Hits        57248    58709    +1461     
- Misses      20701    20938     +237     
Components Coverage Δ
libdd-crashtracker 65.34% <ø> (-0.14%) ⬇️
libdd-crashtracker-ffi 37.68% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 87.55% <97.45%> (+0.41%) ⬆️
libdd-data-pipeline-ffi 75.24% <ø> (-1.79%) ⬇️
libdd-common 79.93% <ø> (+0.04%) ⬆️
libdd-common-ffi 74.41% <ø> (ø)
libdd-telemetry 73.34% <ø> (-0.03%) ⬇️
libdd-telemetry-ffi 31.36% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 76.36% <ø> (+0.14%) ⬆️
libdd-profiling 81.68% <ø> (-0.02%) ⬇️
libdd-profiling-ffi 64.79% <ø> (ø)
libdd-sampling 97.48% <ø> (+0.06%) ⬆️
datadog-sidecar 36.35% <ø> (+1.73%) ⬆️
datdog-sidecar-ffi 11.44% <ø> (+2.82%) ⬆️
spawn-worker 48.86% <ø> (ø)
libdd-tinybytes 93.80% <ø> (ø)
libdd-trace-normalization 81.87% <92.85%> (+0.16%) ⬆️
libdd-trace-obfuscation 87.30% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 89.15% <43.33%> (-0.06%) ⬇️
libdd-tracer-flare 86.57% <ø> (-0.31%) ⬇️
libdd-log 74.83% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented May 13, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 94.60%
Overall Coverage: 73.13% (+0.13%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: cc7156b | Docs | Datadog PR Page | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.70 MB 7.76 MB +.81% (+64.21 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 83.67 MB 84.02 MB +.41% (+355.70 KB) 🔍
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 94.77 MB 95.12 MB +.36% (+356.85 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.34 MB 10.36 MB +.13% (+13.92 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 24.83 MB 24.93 MB +.41% (+104.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 87.33 KB 87.33 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 180.86 MB 181.48 MB +.34% (+640.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 925.02 MB 928.01 MB +.32% (+2.98 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.09 MB 8.12 MB +.41% (+34.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 87.33 KB 87.33 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 23.94 MB 24.03 MB +.39% (+96.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 47.78 MB 47.95 MB +.37% (+181.86 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 21.52 MB 21.61 MB +.43% (+96.50 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 88.71 KB 88.71 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 184.88 MB 185.54 MB +.35% (+672.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 917.97 MB 920.94 MB +.32% (+2.96 MB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.24 MB 6.27 MB +.42% (+27.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 88.71 KB 88.71 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 25.66 MB 25.76 MB +.42% (+112.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 45.41 MB 45.59 MB +.39% (+182.94 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 74.59 MB 74.90 MB +.41% (+320.75 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.58 MB 8.61 MB +.32% (+28.21 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 90.02 MB 90.33 MB +.34% (+322.07 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.44 MB 10.48 MB +.35% (+38.43 KB) 🔍

@Eldolfin Eldolfin changed the title feat(data-pipeline): CSS Trace Filters feat!(data-pipeline): CSS Trace Filters May 15, 2026
@Eldolfin Eldolfin changed the title feat!(data-pipeline): CSS Trace Filters feat(data-pipeline)!: CSS Trace Filters May 15, 2026
@Eldolfin Eldolfin marked this pull request as ready for review May 27, 2026 11:23
@Eldolfin Eldolfin requested review from a team as code owners May 27, 2026 11:23

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4e0ff40bf4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-data-pipeline/src/trace_exporter/mod.rs Outdated
@gh-worker-ownership-write-b05516 gh-worker-ownership-write-b05516 Bot removed the request for review from a team June 10, 2026 15:55
Comment thread libdd-trace-utils/src/trace_filter.rs Outdated
Comment thread libdd-trace-utils/src/trace_filter.rs Outdated
// 3. Require filtering: If filter_tags.require or filter_tags_regex.require contain any
// filters, all of them must match tags on the root span. If any required filter doesn't
// match, reject the trace.
fn should_drop<'a>(&self, root_span: &'a impl Span<'a>) -> bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please make this pub? I need this, not filter_traces (having individual root spans I want to filter).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filter_traces makes sense when you have an already serialized batch of data where the root span is just one span of many. But not when they're still linked lists or some native data structure given by the tracers.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the logic to get root span and drop the trace can be handled by each caller and the TraceFilter only returns the decision for a given root span. This allow the Span trait to stay minimal.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VianneyRuhlmann Yep, that's what I'd propose.

@Eldolfin Eldolfin Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes more sense I agree. I fixed it by making filter_traces(traces) only work for traces with v04 span. 👍🏽

Eldolfin added 12 commits June 11, 2026 11:54
… add missing license header

The dedup() call added to send_trace_chunks_inner was reversing VecMap key
order unconditionally, changing the attribute ordering in V1 msgpack output
and causing snapshot mismatch. The V1 encoder already calls defensive_dedup()
internally; the extra dedup is unnecessary. Also adds missing Apache 2.0
license header to test_trace_filters.rs.
New test agent no longer propagates _dd.p.tid and _dd.p.dm from chunk
level to child spans.
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit 2842d90 into main Jun 16, 2026
91 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the oscarld/APMSP-2763-css-trace-filtering branch June 16, 2026 14:21
iunanua added a commit that referenced this pull request Jun 19, 2026
# Release proposal for libdd-remote-config and its dependencies

This PR contains version bumps based on public API changes and commits
since last release.

## libdd-common
**Next version:** `5.0.0`
**Semver bump:** `major`
**Tag:** `libdd-common-v5.0.0`

### Commits

- chore(profiling): Use SECURITY_ANONYMOUS when connecting to named pipe
server (#2134)
- fix: Fix http PathAndQuery Uri Parsing (#2122)
- chore(common)!: replace native-certs with platform-verifier (#2078)
- feat(data-pipeline)!: CSS Trace Filters (#1985)
- fix(libdd-common): Add fallback logic for resolving Azure Functions
instance name [SVLS-8931] (#2077)
- test: fix timeouts on heavily contended scenarios (#2093)

## libdd-remote-config
**Next version:** `1.0.0`
**Semver bump:** `major`
**Tag:** `libdd-remote-config-v1.0.0`

**Warning:** this is an initial release. Please verify that the version
and commits included are correct.

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: iunanua <18325288+iunanua@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants