Add scenario TRACING_CONFIG_NONDEFAULT in system-tests by cbeauchesne · Pull Request #8602 · DataDog/dd-trace-java

cbeauchesne · 2025-03-21T08:46:37Z

What Does This Do

Add TRACING_CONFIG_NONDEFAULT, TRACING_CONFIG_NONDEFAULT_2 and TRACING_CONFIG_NONDEFAULT_3 in CI

Motivation

Two days ago, a possible regression has been pushed to master (nightly job). Those scenario will prevent this (and other) in the future.

Additional Notes

Wait for DataDog/system-tests#4355 before merging this

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

pr-commenter · 2025-03-21T11:22:20Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	cbeauchesne/TRACING_CONFIG_NONDEFAULT_2
git_commit_date	1742520091	1742552994
git_commit_sha	`fe9f968`	27900d69e0
release_version	1.48.0-SNAPSHOT~fe9f968edb	1.48.0-SNAPSHOT~27900d69e0

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1742556009	1742556009
ci_job_id	858552601	858552601
ci_pipeline_id	59566419	59566419
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-kucxac5a-project-304-concurrent-0-kgzjpndo 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-kucxac5a-project-304-concurrent-0-kgzjpndo 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 5 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.48.0-SNAPSHOT~27900d69e0, baseline=1.48.0-SNAPSHOT~fe9f968edb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.044 s) : 0, 1044427
Total [baseline] (8.656 s) : 0, 8656185
Agent [candidate] (1.044 s) : 0, 1044243
Total [candidate] (8.655 s) : 0, 8654767
section iast
Agent [baseline] (1.17 s) : 0, 1170392
Total [baseline] (9.227 s) : 0, 9227257
Agent [candidate] (1.173 s) : 0, 1173424
Total [candidate] (9.223 s) : 0, 9223366
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.178 s) : 0, 1177943
Total [baseline] (9.214 s) : 0, 9213942
Agent [candidate] (1.181 s) : 0, 1181470
Total [candidate] (9.214 s) : 0, 9213710
section iast_TELEMETRY_OFF
Agent [baseline] (1.169 s) : 0, 1168934
Total [baseline] (9.272 s) : 0, 9272489
Agent [candidate] (1.167 s) : 0, 1166550
Total [candidate] (9.211 s) : 0, 9211067

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	iast	1.17 s	125.965 ms (12.1%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.178 s	133.516 ms (12.8%)
Agent	iast_TELEMETRY_OFF	1.169 s	124.507 ms (11.9%)
Total	tracing	8.656 s	-
Total	iast	9.227 s	571.072 ms (6.6%)
Total	iast_HARDCODED_SECRET_DISABLED	9.214 s	557.757 ms (6.4%)
Total	iast_TELEMETRY_OFF	9.272 s	616.304 ms (7.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	iast	1.173 s	129.181 ms (12.4%)
Agent	iast_HARDCODED_SECRET_DISABLED	1.181 s	137.227 ms (13.1%)
Agent	iast_TELEMETRY_OFF	1.167 s	122.307 ms (11.7%)
Total	tracing	8.655 s	-
Total	iast	9.223 s	568.599 ms (6.6%)
Total	iast_HARDCODED_SECRET_DISABLED	9.214 s	558.943 ms (6.5%)
Total	iast_TELEMETRY_OFF	9.211 s	556.3 ms (6.4%)

gantt
    title insecure-bank - break down per module: candidate=1.48.0-SNAPSHOT~27900d69e0, baseline=1.48.0-SNAPSHOT~fe9f968edb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (718.808 ms) : 0, 718808
BytebuddyAgent [candidate] (719.382 ms) : 0, 719382
GlobalTracer [baseline] (240.13 ms) : 0, 240130
GlobalTracer [candidate] (239.914 ms) : 0, 239914
AppSec [baseline] (55.029 ms) : 0, 55029
AppSec [candidate] (54.643 ms) : 0, 54643
Remote Config [baseline] (710.634 µs) : 0, 711
Remote Config [candidate] (689.408 µs) : 0, 689
Telemetry [baseline] (13.749 ms) : 0, 13749
Telemetry [candidate] (13.515 ms) : 0, 13515
section iast
BytebuddyAgent [baseline] (836.937 ms) : 0, 836937
BytebuddyAgent [candidate] (839.003 ms) : 0, 839003
GlobalTracer [baseline] (229.782 ms) : 0, 229782
GlobalTracer [candidate] (230.546 ms) : 0, 230546
AppSec [baseline] (55.152 ms) : 0, 55152
AppSec [candidate] (55.903 ms) : 0, 55903
Remote Config [baseline] (608.584 µs) : 0, 609
Remote Config [candidate] (598.188 µs) : 0, 598
Telemetry [baseline] (8.642 ms) : 0, 8642
Telemetry [candidate] (8.698 ms) : 0, 8698
IAST [baseline] (23.39 ms) : 0, 23390
IAST [candidate] (22.735 ms) : 0, 22735
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (842.171 ms) : 0, 842171
BytebuddyAgent [candidate] (844.882 ms) : 0, 844882
GlobalTracer [baseline] (230.672 ms) : 0, 230672
GlobalTracer [candidate] (231.826 ms) : 0, 231826
AppSec [baseline] (56.473 ms) : 0, 56473
AppSec [candidate] (56.351 ms) : 0, 56351
Remote Config [baseline] (622.865 µs) : 0, 623
Remote Config [candidate] (607.489 µs) : 0, 607
Telemetry [baseline] (8.741 ms) : 0, 8741
Telemetry [candidate] (8.719 ms) : 0, 8719
IAST [baseline] (23.188 ms) : 0, 23188
IAST [candidate] (22.92 ms) : 0, 22920
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (834.673 ms) : 0, 834673
BytebuddyAgent [candidate] (833.326 ms) : 0, 833326
GlobalTracer [baseline] (230.121 ms) : 0, 230121
GlobalTracer [candidate] (229.803 ms) : 0, 229803
AppSec [baseline] (55.59 ms) : 0, 55590
AppSec [candidate] (55.137 ms) : 0, 55137
Remote Config [baseline] (628.087 µs) : 0, 628
Remote Config [candidate] (600.615 µs) : 0, 601
Telemetry [baseline] (8.643 ms) : 0, 8643
Telemetry [candidate] (8.555 ms) : 0, 8555
IAST [baseline] (23.291 ms) : 0, 23291
IAST [candidate] (23.158 ms) : 0, 23158

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.48.0-SNAPSHOT~27900d69e0, baseline=1.48.0-SNAPSHOT~fe9f968edb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039039
Total [baseline] (10.432 s) : 0, 10432456
Agent [candidate] (1.052 s) : 0, 1052450
Total [candidate] (10.466 s) : 0, 10465507
section appsec
Agent [baseline] (1.182 s) : 0, 1181913
Total [baseline] (10.72 s) : 0, 10719512
Agent [candidate] (1.183 s) : 0, 1183375
Total [candidate] (10.714 s) : 0, 10714147
section iast
Agent [baseline] (1.174 s) : 0, 1174185
Total [baseline] (11.033 s) : 0, 11033147
Agent [candidate] (1.176 s) : 0, 1175990
Total [candidate] (10.997 s) : 0, 10996950
section profiling
Agent [baseline] (1.259 s) : 0, 1259303
Total [baseline] (10.842 s) : 0, 10842009
Agent [candidate] (1.258 s) : 0, 1257790
Total [candidate] (10.84 s) : 0, 10839778

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.039 s	-
Agent	appsec	1.182 s	142.875 ms (13.8%)
Agent	iast	1.174 s	135.146 ms (13.0%)
Agent	profiling	1.259 s	220.264 ms (21.2%)
Total	tracing	10.432 s	-
Total	appsec	10.72 s	287.056 ms (2.8%)
Total	iast	11.033 s	600.691 ms (5.8%)
Total	profiling	10.842 s	409.553 ms (3.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	appsec	1.183 s	130.925 ms (12.4%)
Agent	iast	1.176 s	123.54 ms (11.7%)
Agent	profiling	1.258 s	205.34 ms (19.5%)
Total	tracing	10.466 s	-
Total	appsec	10.714 s	248.64 ms (2.4%)
Total	iast	10.997 s	531.443 ms (5.1%)
Total	profiling	10.84 s	374.271 ms (3.6%)

gantt
    title petclinic - break down per module: candidate=1.48.0-SNAPSHOT~27900d69e0, baseline=1.48.0-SNAPSHOT~fe9f968edb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.778 ms) : 0, 716778
BytebuddyAgent [candidate] (726.935 ms) : 0, 726935
GlobalTracer [baseline] (238.846 ms) : 0, 238846
GlobalTracer [candidate] (240.636 ms) : 0, 240636
AppSec [baseline] (54.561 ms) : 0, 54561
AppSec [candidate] (55.039 ms) : 0, 55039
Remote Config [baseline] (693.845 µs) : 0, 694
Remote Config [candidate] (693.5 µs) : 0, 694
Telemetry [baseline] (12.193 ms) : 0, 12193
Telemetry [candidate] (12.991 ms) : 0, 12991
section appsec
BytebuddyAgent [baseline] (735.818 ms) : 0, 735818
BytebuddyAgent [candidate] (735.831 ms) : 0, 735831
GlobalTracer [baseline] (236.069 ms) : 0, 236069
GlobalTracer [candidate] (235.789 ms) : 0, 235789
AppSec [baseline] (174.338 ms) : 0, 174338
AppSec [candidate] (176.151 ms) : 0, 176151
Remote Config [baseline] (666.929 µs) : 0, 667
Remote Config [candidate] (662.507 µs) : 0, 663
Telemetry [baseline] (8.264 ms) : 0, 8264
Telemetry [candidate] (8.265 ms) : 0, 8265
IAST [baseline] (21.621 ms) : 0, 21621
IAST [candidate] (21.335 ms) : 0, 21335
section iast
BytebuddyAgent [baseline] (838.191 ms) : 0, 838191
BytebuddyAgent [candidate] (841.558 ms) : 0, 841558
GlobalTracer [baseline] (231.506 ms) : 0, 231506
GlobalTracer [candidate] (230.011 ms) : 0, 230011
AppSec [baseline] (56.17 ms) : 0, 56170
AppSec [candidate] (55.526 ms) : 0, 55526
Remote Config [baseline] (607.76 µs) : 0, 608
Remote Config [candidate] (596.51 µs) : 0, 597
Telemetry [baseline] (8.81 ms) : 0, 8810
Telemetry [candidate] (8.668 ms) : 0, 8668
IAST [baseline] (22.975 ms) : 0, 22975
IAST [candidate] (23.608 ms) : 0, 23608
section profiling
BytebuddyAgent [baseline] (708.147 ms) : 0, 708147
BytebuddyAgent [candidate] (707.602 ms) : 0, 707602
GlobalTracer [baseline] (349.141 ms) : 0, 349141
GlobalTracer [candidate] (349.569 ms) : 0, 349569
AppSec [baseline] (54.466 ms) : 0, 54466
AppSec [candidate] (53.668 ms) : 0, 53668
Remote Config [baseline] (717.854 µs) : 0, 718
Remote Config [candidate] (696.313 µs) : 0, 696
Telemetry [baseline] (8.867 ms) : 0, 8867
Telemetry [candidate] (8.903 ms) : 0, 8903
ProfilingAgent [baseline] (96.48 ms) : 0, 96480
ProfilingAgent [candidate] (95.914 ms) : 0, 95914
Profiling [baseline] (96.506 ms) : 0, 96506
Profiling [candidate] (95.938 ms) : 0, 95938

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2025-03-21T10:50:29	2025-03-21T10:58:15
git_branch	master	cbeauchesne/TRACING_CONFIG_NONDEFAULT_2
git_commit_date	1742520091	1742552994
git_commit_sha	`fe9f968`	27900d69e0
release_version	1.48.0-SNAPSHOT~fe9f968edb	1.48.0-SNAPSHOT~27900d69e0
start_time	2025-03-21T10:50:15	2025-03-21T10:58:01

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1742555095	1742555095
ci_job_id	858552602	858552602
ci_pipeline_id	59566419	59566419
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-kucxac5a-project-304-concurrent-1-r2aaz76j 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-kucxac5a-project-304-concurrent-1-r2aaz76j 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 17 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~27900d69e0, baseline=1.48.0-SNAPSHOT~fe9f968edb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.379 ms) : 1360, 1398
.   : milestone, 1379,
appsec (1.733 ms) : 1709, 1757
.   : milestone, 1733,
appsec_no_iast (1.74 ms) : 1715, 1765
.   : milestone, 1740,
code_origins (1.682 ms) : 1655, 1709
.   : milestone, 1682,
iast (1.523 ms) : 1500, 1547
.   : milestone, 1523,
profiling (1.562 ms) : 1537, 1587
.   : milestone, 1562,
tracing (1.495 ms) : 1471, 1519
.   : milestone, 1495,
section candidate
no_agent (1.359 ms) : 1339, 1378
.   : milestone, 1359,
appsec (1.745 ms) : 1722, 1768
.   : milestone, 1745,
appsec_no_iast (1.764 ms) : 1739, 1788
.   : milestone, 1764,
code_origins (1.691 ms) : 1664, 1717
.   : milestone, 1691,
iast (1.536 ms) : 1511, 1561
.   : milestone, 1536,
profiling (1.515 ms) : 1491, 1538
.   : milestone, 1515,
tracing (1.514 ms) : 1490, 1539
.   : milestone, 1514,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.379 ms [1.36 ms, 1.398 ms]	-
appsec	1.733 ms [1.709 ms, 1.757 ms]	353.836 µs (25.7%)
appsec_no_iast	1.74 ms [1.715 ms, 1.765 ms]	361.373 µs (26.2%)
code_origins	1.682 ms [1.655 ms, 1.709 ms]	303.374 µs (22.0%)
iast	1.523 ms [1.5 ms, 1.547 ms]	144.189 µs (10.5%)
profiling	1.562 ms [1.537 ms, 1.587 ms]	182.661 µs (13.2%)
tracing	1.495 ms [1.471 ms, 1.519 ms]	115.787 µs (8.4%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.359 ms [1.339 ms, 1.378 ms]	-
appsec	1.745 ms [1.722 ms, 1.768 ms]	386.276 µs (28.4%)
appsec_no_iast	1.764 ms [1.739 ms, 1.788 ms]	404.838 µs (29.8%)
code_origins	1.691 ms [1.664 ms, 1.717 ms]	331.759 µs (24.4%)
iast	1.536 ms [1.511 ms, 1.561 ms]	177.192 µs (13.0%)
profiling	1.515 ms [1.491 ms, 1.538 ms]	155.798 µs (11.5%)
tracing	1.514 ms [1.49 ms, 1.539 ms]	155.695 µs (11.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.48.0-SNAPSHOT~27900d69e0, baseline=1.48.0-SNAPSHOT~fe9f968edb
    dateFormat X
    axisFormat %s
section baseline
no_agent (391.34 µs) : 372, 411
.   : milestone, 391,
iast (523.403 µs) : 501, 546
.   : milestone, 523,
iast_FULL (732.352 µs) : 710, 754
.   : milestone, 732,
iast_GLOBAL (566.962 µs) : 545, 589
.   : milestone, 567,
iast_HARDCODED_SECRET_DISABLED (516.898 µs) : 495, 539
.   : milestone, 517,
iast_INACTIVE (470.92 µs) : 449, 493
.   : milestone, 471,
iast_TELEMETRY_OFF (507.861 µs) : 486, 530
.   : milestone, 508,
tracing (466.207 µs) : 445, 487
.   : milestone, 466,
section candidate
no_agent (386.815 µs) : 367, 407
.   : milestone, 387,
iast (517.318 µs) : 496, 539
.   : milestone, 517,
iast_FULL (737.964 µs) : 716, 760
.   : milestone, 738,
iast_GLOBAL (573.788 µs) : 551, 597
.   : milestone, 574,
iast_HARDCODED_SECRET_DISABLED (517.874 µs) : 496, 540
.   : milestone, 518,
iast_INACTIVE (479.66 µs) : 457, 502
.   : milestone, 480,
iast_TELEMETRY_OFF (504.214 µs) : 483, 526
.   : milestone, 504,
tracing (464.01 µs) : 443, 485
.   : milestone, 464,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	391.34 µs [371.579 µs, 411.102 µs]	-
iast	523.403 µs [500.811 µs, 545.996 µs]	132.063 µs (33.7%)
iast_FULL	732.352 µs [710.321 µs, 754.382 µs]	341.012 µs (87.1%)
iast_GLOBAL	566.962 µs [544.721 µs, 589.203 µs]	175.622 µs (44.9%)
iast_HARDCODED_SECRET_DISABLED	516.898 µs [494.923 µs, 538.873 µs]	125.558 µs (32.1%)
iast_INACTIVE	470.92 µs [448.68 µs, 493.16 µs]	79.58 µs (20.3%)
iast_TELEMETRY_OFF	507.861 µs [485.732 µs, 529.99 µs]	116.521 µs (29.8%)
tracing	466.207 µs [445.141 µs, 487.273 µs]	74.866 µs (19.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	386.815 µs [366.716 µs, 406.915 µs]	-
iast	517.318 µs [495.507 µs, 539.13 µs]	130.503 µs (33.7%)
iast_FULL	737.964 µs [715.962 µs, 759.966 µs]	351.148 µs (90.8%)
iast_GLOBAL	573.788 µs [550.664 µs, 596.911 µs]	186.972 µs (48.3%)
iast_HARDCODED_SECRET_DISABLED	517.874 µs [496.243 µs, 539.504 µs]	131.058 µs (33.9%)
iast_INACTIVE	479.66 µs [457.272 µs, 502.048 µs]	92.844 µs (24.0%)
iast_TELEMETRY_OFF	504.214 µs [482.551 µs, 525.877 µs]	117.399 µs (30.4%)
tracing	464.01 µs [442.768 µs, 485.253 µs]	77.195 µs (20.0%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	cbeauchesne/TRACING_CONFIG_NONDEFAULT_2
git_commit_date	1742520091	1742552994
git_commit_sha	`fe9f968`	27900d69e0
release_version	1.48.0-SNAPSHOT~fe9f968edb	1.48.0-SNAPSHOT~27900d69e0

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1742555589	1742555589
ci_job_id	858552603	858552603
ci_pipeline_id	59566419	59566419
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-yaepk1w-project-304-concurrent-0-fz4h0w9w 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-yaepk1w-project-304-concurrent-0-fz4h0w9w 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant	appsec	appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~27900d69e0, baseline=1.48.0-SNAPSHOT~fe9f968edb
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.495 s) : 15495000, 15495000
.   : milestone, 15495000,
appsec (14.85 s) : 14850000, 14850000
.   : milestone, 14850000,
iast (19.11 s) : 19110000, 19110000
.   : milestone, 19110000,
iast_GLOBAL (18.229 s) : 18229000, 18229000
.   : milestone, 18229000,
profiling (14.874 s) : 14874000, 14874000
.   : milestone, 14874000,
tracing (14.996 s) : 14996000, 14996000
.   : milestone, 14996000,
section candidate
no_agent (15.533 s) : 15533000, 15533000
.   : milestone, 15533000,
appsec (15.124 s) : 15124000, 15124000
.   : milestone, 15124000,
iast (18.595 s) : 18595000, 18595000
.   : milestone, 18595000,
iast_GLOBAL (18.039 s) : 18039000, 18039000
.   : milestone, 18039000,
profiling (15.619 s) : 15619000, 15619000
.   : milestone, 15619000,
tracing (14.851 s) : 14851000, 14851000
.   : milestone, 14851000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.495 s [15.495 s, 15.495 s]	-
appsec	14.85 s [14.85 s, 14.85 s]	-645.0 ms (-4.2%)
iast	19.11 s [19.11 s, 19.11 s]	3.615 s (23.3%)
iast_GLOBAL	18.229 s [18.229 s, 18.229 s]	2.734 s (17.6%)
profiling	14.874 s [14.874 s, 14.874 s]	-621.0 ms (-4.0%)
tracing	14.996 s [14.996 s, 14.996 s]	-499.0 ms (-3.2%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.533 s [15.533 s, 15.533 s]	-
appsec	15.124 s [15.124 s, 15.124 s]	-409.0 ms (-2.6%)
iast	18.595 s [18.595 s, 18.595 s]	3.062 s (19.7%)
iast_GLOBAL	18.039 s [18.039 s, 18.039 s]	2.506 s (16.1%)
profiling	15.619 s [15.619 s, 15.619 s]	86.0 ms (0.6%)
tracing	14.851 s [14.851 s, 14.851 s]	-682.0 ms (-4.4%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.48.0-SNAPSHOT~27900d69e0, baseline=1.48.0-SNAPSHOT~fe9f968edb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.359 ms) : 2315, 2403
.   : milestone, 2359,
iast (2.14 ms) : 2084, 2196
.   : milestone, 2140,
iast_GLOBAL (2.174 ms) : 2118, 2230
.   : milestone, 2174,
profiling (1.992 ms) : 1947, 2036
.   : milestone, 1992,
tracing (1.972 ms) : 1929, 2015
.   : milestone, 1972,
section candidate
no_agent (1.485 ms) : 1473, 1496
.   : milestone, 1485,
appsec (2.364 ms) : 2320, 2408
.   : milestone, 2364,
iast (2.135 ms) : 2079, 2190
.   : milestone, 2135,
iast_GLOBAL (2.172 ms) : 2116, 2228
.   : milestone, 2172,
profiling (2.008 ms) : 1962, 2054
.   : milestone, 2008,
tracing (1.959 ms) : 1916, 2002
.   : milestone, 1959,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.461 ms, 1.484 ms]	-
appsec	2.359 ms [2.315 ms, 2.403 ms]	887.038 µs (60.3%)
iast	2.14 ms [2.084 ms, 2.196 ms]	667.767 µs (45.4%)
iast_GLOBAL	2.174 ms [2.118 ms, 2.23 ms]	701.594 µs (47.7%)
profiling	1.992 ms [1.947 ms, 2.036 ms]	519.432 µs (35.3%)
tracing	1.972 ms [1.929 ms, 2.015 ms]	499.599 µs (33.9%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.485 ms [1.473 ms, 1.496 ms]	-
appsec	2.364 ms [2.32 ms, 2.408 ms]	879.247 µs (59.2%)
iast	2.135 ms [2.079 ms, 2.19 ms]	649.75 µs (43.8%)
iast_GLOBAL	2.172 ms [2.116 ms, 2.228 ms]	687.371 µs (46.3%)
profiling	2.008 ms [1.962 ms, 2.054 ms]	523.213 µs (35.2%)
tracing	1.959 ms [1.916 ms, 2.002 ms]	474.347 µs (31.9%)

github-actions · 2025-03-21T14:01:27Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

cbeauchesne added 2 commits March 21, 2025 09:45

Add scenario TRACING_CONFIG_NONDEFAULT in system-tests

7c015b9

Update system-tests to 69a5e874384dd256e2e3f42fc1c95807a67efbe6

cef2912

cbeauchesne force-pushed the cbeauchesne/TRACING_CONFIG_NONDEFAULT_2 branch from 27900d6 to cef2912 Compare March 21, 2025 10:31

cbeauchesne marked this pull request as ready for review March 21, 2025 14:01

cbeauchesne requested a review from a team as a code owner March 21, 2025 14:01

cbeauchesne requested a review from dougqh March 21, 2025 14:01

amarziali added type: feature Enhancements and improvements comp: testing Testing tag: no release notes Changes to exclude from release notes labels Mar 21, 2025

cbeauchesne enabled auto-merge (squash) March 21, 2025 14:11

amarziali approved these changes Mar 21, 2025

View reviewed changes

amarziali added comp: testing Testing and removed comp: testing Testing labels Mar 21, 2025

cbeauchesne merged commit efdcea8 into master Mar 21, 2025

cbeauchesne deleted the cbeauchesne/TRACING_CONFIG_NONDEFAULT_2 branch March 21, 2025 14:15

github-actions Bot added this to the 1.48.0 milestone Mar 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add scenario TRACING_CONFIG_NONDEFAULT in system-tests#8602

Add scenario TRACING_CONFIG_NONDEFAULT in system-tests#8602
cbeauchesne merged 2 commits into
masterfrom
cbeauchesne/TRACING_CONFIG_NONDEFAULT_2

cbeauchesne commented Mar 21, 2025 •

edited

Loading

Uh oh!

pr-commenter Bot commented Mar 21, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Mar 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

cbeauchesne commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter Bot commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

github-actions Bot commented Mar 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cbeauchesne commented Mar 21, 2025 •

edited

Loading

pr-commenter Bot commented Mar 21, 2025 •

edited

Loading