Add process autodiscovery infrastructure and support to krakend

[vitkyrka]

What does this PR do?

Adds process autodiscovery infrastructure, and support to the krakend integration. When krakend runs as a process in environments without containers, the agent's process autodiscovery listener matches it via a CEL expression and the existing configuration discovery flow finds the metrics endpoint and schedules the check automatically.

The essential change to add the support to krakend is the change to auto_conf.yaml (regenerated from spec.yaml). The other changes are infrastructure to allow process autodiscovery support to be added to integrations and to be E2E tested while keeping the amount of changes required in each integration relatively small.

The main challenge is cleanly re-using the existing container-based E2E infrastructure for process autodiscovery testing without running krakend as a true host process (since that would require a lot of extra work in each integration's tests to set up the respective services without using containers).

The chosen approach is to keep the existing Docker Compose setup unchanged and instead hide the containers from the agent by setting DD_AUTOCONFIG_EXCLUDE_FEATURES=docker, which makes the agent's process autodiscovery component handle the processes as normal host processes. This environment variable is pass to the agent check invocation. The Docker environment for the agent also needs a few extra caps and config options for the processes tests, but these options should not affect the container-based tests.

Motivation

https://datadoghq.atlassian.net/browse/DSCVR-493

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[datadog-prod-us1-4]

   

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 2 Pipeline jobs failed

PR All | test / j06ca546 / SNMP     

PR | test / check     

🧪 1 Test failed in 1 job

PR All | run  

All test failures are known flaky — job may pass on retry.

❄️ Known flaky: test_e2e_snmp_listener from test_e2e_snmp_listener.py  

Needed at least 1 candidates for 'datadog.snmp.check_duration', got 0
Expected:
        MetricStub(name='datadog.snmp.check_duration', type=0, value=None, tags=['autodiscovery_subnet:172.18.0.0/28', 'device_vendor:apc', 'firmware_version:2.0.3-test', 'loader:python', 'model:APC Smart-UPS 600', 'serial_num:test_serial', 'snmp_device:172.18.0.1', 'snmp_profile:apc_ups', 'ups_name:testIdentName'], hostname=None, device=None, flush_first_value=None)
Difference to closest:
        Expected tag snmp_device:172.18.0.1
        Found snmp_device:172.18.0.2

Similar submitted:
Score   Most similar
1.00    MetricStub(name='datadog.snmp.check_duration', type=0, value=0.21939873695373535, tags=['autodiscovery_subnet:172.18.0.0/28', 'device_vendor:apc', 'firmware_version:2.0.3-test', 'loader:python', 'model:APC Smart-UPS 600', 'serial_num:test_serial', 'snmp_device:172.18.0.2', 'snmp_profile:apc_ups', 'ups_name:testIdentName'], hostname='runnervmjbsos', device=None, flush_first_value=False)
...

Not introduced in this PR.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 57.14%
• Overall Coverage: 88.13% (+0.00%)

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: cbbf8e2 | Docs | Datadog PR Page | Give us feedback!}

[vitkyrka]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: baaed78950

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c04e675036

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[vitkyrka]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• Add process autodiscovery infrastructure and support to krakend #24238 👈 (View in Graphite)
• ddev: support passing env vars to ddev env agent #24289
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[dd-octo-sts]

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and code coverage settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
license-headers	Validate Python files have proper license headers	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run