renamed folder and added resources to match others by technovangelist · Pull Request #2 · DataDog/integrations-core

technovangelist · 2015-12-22T22:35:48Z

No description provided.

renamed folder and added resources to match others

…13243) * Implement Multi-instance counters without relying on Windows PdhEnumObject(refresh=TRUE) API. Key points: * Multi-instance counter is created with wildcards for the instance name component. If include_fast configuration is specified its list is used exactly AS IS as instance components during counters creation. This include_fast is a new configuration which is complementary to include and exclude filters. * Depending on the number of instances wildcard counters are at least 2x-10x faster than per instance counters (7.34+ release). When the number of counters specified for an object (not to confuse with counter instances) is more than one the wildcard counters are even faster. * However, if a counter has a large number of counter instances (let’s say more than a thousand), in previous implementations since 7.34 with a very selective include and exclude filter can still be fast. Wildcard counters by default will retrieve all counters from the PDH layer and only then apply include and exclude regex filters, and accordingly may be slower, especially if there are very large instance counts (let’s say 5k+). This is where include_fast configuration should be used instead by passing seleciver wildcard filters to the PDH. * New performance counters objects and their counters become available only after their provider is installed (e.g. via lodctr.exe) and reversely performance counters objects and their counters disappear after their provider is uninstalled (e.g. unlodctr.exe). Accordingly, by default, a call PdhEnumObject(refresh=TRUE) is not needed since Agent configuration requires specifying both object and counters and likelihood of performance counter provider install or uninstall is relatively small. If the specified objects had been installed after Agent service started, the Agent needs to be restarted. Alternatively, one can specify windows_counter_refresh_interval configuration (in seconds) to detect and start using new performance counters objects and their counters with acknowledgments of the following caveats: Caveats 1) PdhEnumObject(refresh=TRUE) in general is very slow and resource intensive (100k+ alloc and free) and its duration may reach few seconds. 2) PdhEnumObject(refresh=TRUE) certainly causes at least a few warnings and errors in Windows Event Log if the Agent user is not a Local System user of Local Administrators. Unless you can ignore such nuances errors you should change Agent user configuration. 3) There are anecdotal evidences that on some environments and for some performance counters objects PdhEnumObject(refresh=TRUE) call may trigger a memory or resource leak. * Implement Multi-instance counters without relying on Windows PdhEnumObject(refresh=TRUE) API. Removed non-default edge case support of dynamic addition/removal of configured performance counters. Even if a provider of a performance counter object, specified in our configuration, is not running, as long as its provider has been installed/deployed on the box, that is all needed for this integration. When the counter provider starts, Agent will discover its new instances and report them as metrics correspondingly. However, if the provider is installed AFTER the agent starts running, this implementation will not be able to automatically discover and use it. In some way it is a step back because this ability had been added in 7.34 among few other very important changes including detection of appearance and disappearance of performance counter instances, support for non-unique instances and localization support. In this release detection of newly registered performance counters providers (and their objects and counters) is pulled out for the following reasons: * It does not appear that the feature was needed by customers * It adds non-insignificant complexity to the code * It relies on very slow and resource hungry PdhEnumObjects(refresh=TRUE) function. * It pretty much requires the agent process to be run as administrator or local system user (otherwise will generate a handful of error messages in the event log every time refresh run). Microsoft support confirmed it even though it is not documented. It is difficult to explain to customers. * The logic of dynamically adding and removing counters handles to facilitate support appears to may cause memory leaks in some counters (documented at least for ASP.NET e.g.). Perhaps in future if customers would really want that and are ready to deal with the caveats above we can potentially add it back. * Implement Multi-instance counters without relying on Windows PdhEnumObject(refresh=TRUE) API. * Implement Multi-instance counters without relying on Windows PdhEnumObject(refresh=TRUE) API. Made few minor adjustments to fix automated tests * Temporary added direct call to Windows PdhGetFormattedCounterArrayW() API instead of win32pdh.PdhGetFormattedCounterArray()) to handle duplicate instances. By default this implementation will be invoked but it can be overridden via duplicate_instances_exist: False configuration which is 5-10x faster. We are requesting win32pdh.PdhGetFormattedCounterArray() enhancement to handle duplicates as well. * Fix linting * Attempt to fix spec/model error * Attempt to fix spec/model error (Attempt #2) * Fix Python lint report * Address code review comments, chiefly that PdhValidatePath was not locale-aware. * Code review related changes * Clarify performance comments and made a tiny improvement * Fix Python lint error * Move updates to templates into a separate PR

….GetFormattedCounterArray() (#13897) * Fix IIS Check memory leaked due to a bug in win32pdh.GetFormattedCounterArray() The leak was introduced in 7.42 and only IIS Check has been affected in attempt to optimize it using win32pdh fast function which inadvertently had a memory leak. * Fix linting problem * Fix linting #2 * Remove base check comment Per code review to avoid polluting changelog with no code changes

Refs PR #23883

… on server To ensure that this no risk with probing the ports of the container for configuration discovery, add a test which instanciates the integration which each of the generated configurations in turn and checks that the container logs don't have any obvious crashes. https://datadoghq.atlassian.net/wiki/spaces/DSCVR/pages/6671862234/Configuration+Discovery+for+Agent+Integrations#Probing-causes-problems While the logic for detecting "bad" effects is best-effort, the test also the logs from this scenario to be seen easier during development. For example, for krakend: ``` $ ddev env test --base --new-env krakend py3.13-2.10 -- -k test_e2e_discovery_candidates_do_not_destabilize_container -rP --log-level=DEBUG ... _________ test_e2e_discovery_candidates_do_not_destabilize_container __________ ------------------------------ Captured log call ------------------------------- DEBUG root:docker.py:90 Probing candidate #1: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:9090/metrics'}]} DEBUG root:docker.py:90 Probing candidate #2: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8080/metrics'}]} DEBUG root:docker.py:94 Error probing candidate #2: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8080/metrics'}]} DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:46 | 404 | 2.803µs | 172.17.129.1 | GET "/metrics" DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:47 | 404 | 2.753µs | 172.17.129.1 | GET "/metrics" DEBUG root:docker.py:90 Probing candidate #3: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8090/metrics'}]} DEBUG root:docker.py:94 Error probing candidate #3: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8090/metrics'}]} DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:49 | 200 | 51.868µs | ::1 | GET "/__health" ```

* Add HTTPXWrapper backed by httpx2 behind use_httpx flag * Address Round 1 review findings * Tighten comments from human review * Use httpx2 directly instead of aliasing as httpx * Rename to httpx2 * Rename HTTPXWrapper symbols to HTTPX2Wrapper * Round 2 finding #3: move test_stream_connection_lines to OM scraper test dir * Round 2 finding #7: drop unreachable __del__ AttributeError * Round 2 finding #4: drop dead DEFAULT_REMAPPED_FIELDS constant * Round 2 finding #9: simplify lifecycle import-failure test * Round 2 finding #8: assert stream-kwarg drop log via caplog * Round 2 finding #5: extract _build_http_client helper * Round 2 finding #10: case-fold headers/extra_headers merge * Round 2 finding #6: close response on adapter wrap failure * Round 2 finding #1: map httpx2 exceptions in iter_lines/iter_content * Round 3 fix #4: tighten ImportError assertion to exc.value.name * Round 3 fix #2: iter_lines bytes-through fidelity * Round 3 fix #3: derive passthrough from REQUEST_KWARGS * Round 3 fix #1: timeout=None guard + scalar _make_timeout consistency * Round 4 fix #1: map mid-stream httpx2 errors to HTTPConnectionError * Round 4 fix #4: document init header dict-shape parity with RequestsWrapper * Round 4 fix #3: add encoding/url/cookies/elapsed to HTTPResponseProtocol * Round 4 fix #2: replace bytes accumulator with bytearray in iter_lines * Round 5 finding #5: tighten HTTPResponseProtocol.cookies to Mapping[str, str] * Round 5 finding #1: collapse _map_httpx2_exception branches via httpx2.NetworkError * Round 5 finding #2: lock in iter_lines decode CRLF parity test * Round 5 finding #4: lock in headers={} benign no-op behavior * Round 5 finding #3: document options['headers'] mirror constraint * Round 5 finding #6: use getattr guard in test_lifecycle teardown * Round 5 finding #7: rename misleading iter_lines bytes-path test * Round 5 finding #8: drop tautological protocol attrs test * Round 6 finding #1: collapse follow_redirects routing * Round 6 finding #2: assert request kwargs invariant at import * Round 7 finding #1: http-response-protocol-encoding-setter * Round 7 finding #2: follow-redirects-positive-test-coverage * Round 8 finding #1: wrap-content-text-json-exceptions * Round 8 finding #4: parametrize-warning-path-tests * Map self.options['headers'] mutations to httpx2 requests * Improve HTTPX2Wrapper TypeError with per-kwarg remediation hints * Tighten test_stream_kwarg_still_silently_dropped to assert response, not call count * Add allow_redirects hint to UNKNOWN_KWARG_HINTS * Drop dead _client.headers writes from set_header * Accept per-request auth kwarg in HTTPX2Wrapper * Type-hint _build_http_client instance parameter Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Document SSL folds into HTTPConnectionError in _map_httpx2_exception Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

… on server To ensure that this no risk with probing the ports of the container for configuration discovery, add a test which instanciates the integration which each of the generated configurations in turn and checks that the container logs don't have any obvious crashes. https://datadoghq.atlassian.net/wiki/spaces/DSCVR/pages/6671862234/Configuration+Discovery+for+Agent+Integrations#Probing-causes-problems While the logic for detecting "bad" effects is best-effort, the test also the logs from this scenario to be seen easier during development. For example, for krakend: ``` $ ddev env test --base --new-env krakend py3.13-2.10 -- -k test_e2e_discovery_candidates_do_not_destabilize_container -rP --log-level=DEBUG ... _________ test_e2e_discovery_candidates_do_not_destabilize_container __________ ------------------------------ Captured log call ------------------------------- DEBUG root:docker.py:90 Probing candidate #1: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:9090/metrics'}]} DEBUG root:docker.py:90 Probing candidate #2: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8080/metrics'}]} DEBUG root:docker.py:94 Error probing candidate #2: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8080/metrics'}]} DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:46 | 404 | 2.803µs | 172.17.129.1 | GET "/metrics" DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:47 | 404 | 2.753µs | 172.17.129.1 | GET "/metrics" DEBUG root:docker.py:90 Probing candidate #3: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8090/metrics'}]} DEBUG root:docker.py:94 Error probing candidate #3: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8090/metrics'}]} DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:49 | 200 | 51.868µs | ::1 | GET "/__health" ```

@strategy

#24126) * Remove __discovery_provides__ attribute and dead-code tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add discovery config spec validation and model generation to ddev. Adds a discovery section to the spec schema (port_hints, strategy field), extends the example and model consumers to render/generate discovery config, adds spec.py validation for discovery spec correctness, and covers the new paths with tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * add annotations import * Add krakend discovery config support with e2e discovery test helpers. Adds krakend config_models/discovery.py and auto_conf.yaml for port-based config autodiscovery. Adds get_e2e_discovery_metadata() to ddev docker helpers and a dd_agent_check_discovery pytest fixture so integrations can run discovery-path E2E tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * regen * add test to force probe of all ports/configs to check no side effects on server To ensure that this no risk with probing the ports of the container for configuration discovery, add a test which instanciates the integration which each of the generated configurations in turn and checks that the container logs don't have any obvious crashes. https://datadoghq.atlassian.net/wiki/spaces/DSCVR/pages/6671862234/Configuration+Discovery+for+Agent+Integrations#Probing-causes-problems While the logic for detecting "bad" effects is best-effort, the test also the logs from this scenario to be seen easier during development. For example, for krakend: ``` $ ddev env test --base --new-env krakend py3.13-2.10 -- -k test_e2e_discovery_candidates_do_not_destabilize_container -rP --log-level=DEBUG ... _________ test_e2e_discovery_candidates_do_not_destabilize_container __________ ------------------------------ Captured log call ------------------------------- DEBUG root:docker.py:90 Probing candidate #1: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:9090/metrics'}]} DEBUG root:docker.py:90 Probing candidate #2: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8080/metrics'}]} DEBUG root:docker.py:94 Error probing candidate #2: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8080/metrics'}]} DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:46 | 404 | 2.803µs | 172.17.129.1 | GET "/metrics" DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:47 | 404 | 2.753µs | 172.17.129.1 | GET "/metrics" DEBUG root:docker.py:90 Probing candidate #3: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8090/metrics'}]} DEBUG root:docker.py:94 Error probing candidate #3: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8090/metrics'}]} DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:49 | 200 | 51.868µs | ::1 | GET "/__health" ``` * Spec-driven discovery redesign: tooling + krakend PoC Move from_ports from a runtime function in datadog_checks_base to a codegen strategy in datadog_checks_dev, and introduce a registry-driven model consumer that generates candidates() using Pydantic models. Key changes: - New discovery registry package with @strategy decorator and REGISTRY - core_strategies.py registers from_ports as a codegen strategy - _build_discovery_file is now registry-driven (no hardcoded from_ports) - Generated candidates() use InstanceConfig/SharedConfig model_dump (D1) - Remove ad_identifiers from spec discovery stanza (D2: hand-maintained) - Remove auto_conf.yaml generation from example consumer (D2) - Add discovery_strategies.py and discovery_overrides.py to CUSTOM_FILES - Regenerate krakend discovery.py with model-backed candidate output - Add hand-maintained krakend/auto_conf.yaml with ad_identifiers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Use model_validate with context for discovery candidate generation Direct InstanceConfig/SharedConfig construction fails because the field validators access info.context['configured_fields'], which is None without explicit context. Switch to model_validate with the required context dict so that all defaults from defaults.py are applied correctly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * T3/T4: extract expand_template_items, restructure discovery validation Extract expand_template_items from the template expansion loop in options_validator so it can be shared with discovery strategy lists. Add handle_discovery which runs after options_validator so discovery validation sees resolved options. Discovery now supports enabled:false kill-switch, local: strategy names, and candidate field cross-check against resolved instance options. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Refactor options_validator to call expand_template_items Remove the inline template expansion loop from options_validator and replace it with a call to expand_template_items, which was extracted in T3 precisely so both callers share one implementation. The only behavioral difference from the original was a per-item overrides dict; aligning expand_template_items to use shared overrides (accumulated across all items) preserves the existing semantics. hide_template was redundant: template.update(option) already carries any hidden:true from intermediate templates into the expanded item, so setdefault('hidden', False) produces the same result in all cases. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Make spec-driven discovery tooling fully registry-driven Align the discovery tooling with the implementation plan: a strategy now owns both its contract (provides/inputs) and its codegen (emit_context) in the dev-side registry, and the generator collects each strategy's runtime imports and emits the model-backed candidate body. No strategy name or input is special-cased in the generator or validator. - registry: add Input and Strategy(provides, inputs, runtime_imports, emit_context); add guarded SERVICE_FIELDS/PORT_FIELDS constants - core_strategies: from_ports emits only the candidate loop and ctx - model_consumer: registry-driven generator that builds candidates through the config models, supports local: strategies, and emits the discovery_strategies.py/discovery_overrides.py first-render stubs - spec: derive candidate placeholders from provides + the field constants, validate strategy inputs generically, honour enabled:false as a kill switch, resolve discovery-level templates - base: add a guard test asserting the dev field constants match the Service/Port pydantic models so the hand-copied fact cannot drift - fix the datadog_checks_dev discovery changelog entry PR number Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * krakend: regenerate discovery models and drop stray auto_conf.yaml - add the regenerated discovery_overrides.py custom-file stub - remove the misplaced package-root auto_conf.yaml; the Agent-facing opt-in file lives in data/auto_conf.yaml - point the discovery changelog entry at the correct PR number Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fix discovery strategy validation * Wire discovery_overrides.py seam into generated discovery.py Generated discovery.py now delegates to the discovery_overrides.py custom file via the same pattern validators.py already uses: import the always- co-generated module and resolve candidates() through getattr with a fallback to the generated generator. Behaviorally identical while the stub is empty; live once an integration fills in candidates(service, default), with no further tooling PR. Regenerated krakend's discovery.py accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * krakend: regenerate discovery stub docs from current constants The discovery_strategies.py / discovery_overrides.py stubs are write-once CUSTOM_FILES, so they kept the wording from an earlier render and never re-synced with the updated documentation constants. Delete and regenerate them so the in-tree stubs match the current model_consumer constants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * krakend: match changelog text to phase 2 spec Use the wording prescribed by the phase 2 plan (T9): "Add configuration discovery support." Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fix template-expansion regressions in discovery config tooling Two issues surfaced by review of the discovery config tooling refactor: - expand_template_items dropped wrapper attributes (e.g. `hidden: true`) when a template resolved to a list, so the common `- template: instances/pdh_legacy` + `hidden: true` pattern used by aspdotnet, dotnetclr, hyperv, exchange_server, and active_directory stopped hiding the expanded options. Propagate the wrapper's leftover attributes to each expanded item via setdefault, restoring the pre-refactor behaviour. - The discovery model consumer emitted candidate templates inside a single-quoted literal, producing invalid discovery.py when a template contained a single quote or backslash. Use repr() to emit a properly escaped literal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add test for hidden propagation on list-template expansion Locks in the fix: a `hidden: true` wrapper on a list template applies to every expanded item, while an item's own explicit `hidden` value wins. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Replace chr(10) workaround with a plain newline join chr(10) was the pre-3.12 workaround for using a backslash inside an f-string expression. Build the joined message first, matching the existing style in expand_template_items. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Restore master template-expansion behavior in spec tooling The discovery-config refactor changed two aspects of option-template expansion in `expand_template_items`, both of which broke config/models validation for existing integrations: - Overrides were scoped per-item, so an override targeting a nested template (e.g. `extra_metrics.value.example` in the Windows perf-counter specs) was attempted prematurely on the parent template, reported as an error, and discarded before the nested template was spliced in and expanded. Restore the shared-override semantics: `apply_overrides` pops each override on success, so unresolved ones are retried against later items, and only truly-unused overrides are reported. - `hidden` propagation was scoped to a template's own expanded items, dropping the historical cross-sibling leak where a template wrapper's `hidden: true` carries onto following plain options until the next template. haproxy relies on this to hide its legacy (non-OpenMetrics) option block. Reintroduce it behind a `propagate_hidden` flag (on for options, off for discovery strategies, which have no `hidden` concept). Rendered example/model output is now byte-identical to master across all 263 integration specs. Adds a regression test for the nested-template override case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * remove test_discovery_stanza_does_not_emit_auto_conf * Stop reserving auto_conf.yaml name in discovery handling Discovery no longer generates auto_conf.yaml (it is hand-maintained per integration), so reserving the example name produced a false-positive collision for the legitimate "discovery stanza + hand-maintained auto_conf.yaml" combination. Drop the reservation, its plumbing through handle_discovery, and the test that pinned the false positive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Drop redundant hidden setdefault in options_validator `expand_template_items(propagate_hidden=True)` now sets `hidden` on every resolved option during expansion, so the `option.setdefault('hidden', False)` in options_validator was a dead no-op that misleadingly read as the source of the default. Remove it; rendered example/model output is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add discovery/openmetrics_from_ports and auto_conf/discovery spec templates Adds two reusable spec tooling templates: - discovery/openmetrics_from_ports: a from_ports strategy candidate that generates an openmetrics_endpoint URL from the discovered host and port. Integrations override port_hints to set the expected port. - auto_conf/discovery: the common options block for fleet-discovery auto_conf.yaml files (discovery: {}, init_config, instances: []). Each integration sets ad_identifiers independently and includes this template for the rest. Converts the krakend spec to use both templates and regenerates the krakend auto_conf.yaml and config_models/discovery.py from the spec. Also fixes ModelConsumer._process_section to return early for leaf options (no sub-options key) so that fleet-discovery auto_conf.yaml files, which use leaf instances: [] rather than a full instances section, do not crash validate models. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * modify auto_config * Narrow model consumer leaf-instances guard to instances section only The previous fix skipped any section without an `options` key before even checking whether it was init_config or instances. Tighten it to only apply inside the `instances` branch, where auto_conf.yaml files using the fleet-discovery pattern emit a leaf `instances: []` with no model schema. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Restrict leaf-instances guard to auto_conf.yaml files only Pass the spec file name into _process_section and apply the no-options early-return only when processing auto_conf.yaml, where a leaf instances: [] is valid for fleet discovery. Any other file with a leaf instances section still hits the normal code path and fails visibly. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Use by_alias=True when dumping discovery candidate configs Without this, options with hyphenated names (e.g. slowlog-max-len) are emitted as Python field names (slowlog_max_len) in the generated candidate dict. The Agent passes this dict verbatim as the raw instance config, so checks that look up the original hyphenated key would not find it. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Skip SharedConfig import/usage when spec has no init_config section When a spec declares discovery but omits init_config, no shared.py is generated. The discovery file unconditionally imported SharedConfig, causing ModuleNotFoundError at import time and silently preventing any candidates from being produced. Pass has_shared=False to _build_discovery_file in that case; the generated file emits `shared = {}` instead and omits the import. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * regen --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> aa23ee0

@strategy

DataDog#24126) * Remove __discovery_provides__ attribute and dead-code tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Add discovery config spec validation and model generation to ddev. Adds a discovery section to the spec schema (port_hints, strategy field), extends the example and model consumers to render/generate discovery config, adds spec.py validation for discovery spec correctness, and covers the new paths with tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * add annotations import * Add krakend discovery config support with e2e discovery test helpers. Adds krakend config_models/discovery.py and auto_conf.yaml for port-based config autodiscovery. Adds get_e2e_discovery_metadata() to ddev docker helpers and a dd_agent_check_discovery pytest fixture so integrations can run discovery-path E2E tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * regen * add test to force probe of all ports/configs to check no side effects on server To ensure that this no risk with probing the ports of the container for configuration discovery, add a test which instanciates the integration which each of the generated configurations in turn and checks that the container logs don't have any obvious crashes. https://datadoghq.atlassian.net/wiki/spaces/DSCVR/pages/6671862234/Configuration+Discovery+for+Agent+Integrations#Probing-causes-problems While the logic for detecting "bad" effects is best-effort, the test also the logs from this scenario to be seen easier during development. For example, for krakend: ``` $ ddev env test --base --new-env krakend py3.13-2.10 -- -k test_e2e_discovery_candidates_do_not_destabilize_container -rP --log-level=DEBUG ... _________ test_e2e_discovery_candidates_do_not_destabilize_container __________ ------------------------------ Captured log call ------------------------------- DEBUG root:docker.py:90 Probing candidate DataDog#1: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:9090/metrics'}]} DEBUG root:docker.py:90 Probing candidate DataDog#2: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8080/metrics'}]} DEBUG root:docker.py:94 Error probing candidate DataDog#2: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8080/metrics'}]} DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:46 | 404 | 2.803µs | 172.17.129.1 | GET "/metrics" DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:47 | 404 | 2.753µs | 172.17.129.1 | GET "/metrics" DEBUG root:docker.py:90 Probing candidate DataDog#3: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8090/metrics'}]} DEBUG root:docker.py:94 Error probing candidate DataDog#3: {'init_config': {}, 'instances': [{'openmetrics_endpoint': 'http://172.17.129.3:8090/metrics'}]} DEBUG root:docker.py:103 New log line: [GIN] 2026/06/17 - 15:31:49 | 200 | 51.868µs | ::1 | GET "/__health" ``` * Spec-driven discovery redesign: tooling + krakend PoC Move from_ports from a runtime function in datadog_checks_base to a codegen strategy in datadog_checks_dev, and introduce a registry-driven model consumer that generates candidates() using Pydantic models. Key changes: - New discovery registry package with @strategy decorator and REGISTRY - core_strategies.py registers from_ports as a codegen strategy - _build_discovery_file is now registry-driven (no hardcoded from_ports) - Generated candidates() use InstanceConfig/SharedConfig model_dump (D1) - Remove ad_identifiers from spec discovery stanza (D2: hand-maintained) - Remove auto_conf.yaml generation from example consumer (D2) - Add discovery_strategies.py and discovery_overrides.py to CUSTOM_FILES - Regenerate krakend discovery.py with model-backed candidate output - Add hand-maintained krakend/auto_conf.yaml with ad_identifiers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Use model_validate with context for discovery candidate generation Direct InstanceConfig/SharedConfig construction fails because the field validators access info.context['configured_fields'], which is None without explicit context. Switch to model_validate with the required context dict so that all defaults from defaults.py are applied correctly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * T3/T4: extract expand_template_items, restructure discovery validation Extract expand_template_items from the template expansion loop in options_validator so it can be shared with discovery strategy lists. Add handle_discovery which runs after options_validator so discovery validation sees resolved options. Discovery now supports enabled:false kill-switch, local: strategy names, and candidate field cross-check against resolved instance options. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Refactor options_validator to call expand_template_items Remove the inline template expansion loop from options_validator and replace it with a call to expand_template_items, which was extracted in T3 precisely so both callers share one implementation. The only behavioral difference from the original was a per-item overrides dict; aligning expand_template_items to use shared overrides (accumulated across all items) preserves the existing semantics. hide_template was redundant: template.update(option) already carries any hidden:true from intermediate templates into the expanded item, so setdefault('hidden', False) produces the same result in all cases. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Make spec-driven discovery tooling fully registry-driven Align the discovery tooling with the implementation plan: a strategy now owns both its contract (provides/inputs) and its codegen (emit_context) in the dev-side registry, and the generator collects each strategy's runtime imports and emits the model-backed candidate body. No strategy name or input is special-cased in the generator or validator. - registry: add Input and Strategy(provides, inputs, runtime_imports, emit_context); add guarded SERVICE_FIELDS/PORT_FIELDS constants - core_strategies: from_ports emits only the candidate loop and ctx - model_consumer: registry-driven generator that builds candidates through the config models, supports local: strategies, and emits the discovery_strategies.py/discovery_overrides.py first-render stubs - spec: derive candidate placeholders from provides + the field constants, validate strategy inputs generically, honour enabled:false as a kill switch, resolve discovery-level templates - base: add a guard test asserting the dev field constants match the Service/Port pydantic models so the hand-copied fact cannot drift - fix the datadog_checks_dev discovery changelog entry PR number Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * krakend: regenerate discovery models and drop stray auto_conf.yaml - add the regenerated discovery_overrides.py custom-file stub - remove the misplaced package-root auto_conf.yaml; the Agent-facing opt-in file lives in data/auto_conf.yaml - point the discovery changelog entry at the correct PR number Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fix discovery strategy validation * Wire discovery_overrides.py seam into generated discovery.py Generated discovery.py now delegates to the discovery_overrides.py custom file via the same pattern validators.py already uses: import the always- co-generated module and resolve candidates() through getattr with a fallback to the generated generator. Behaviorally identical while the stub is empty; live once an integration fills in candidates(service, default), with no further tooling PR. Regenerated krakend's discovery.py accordingly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * krakend: regenerate discovery stub docs from current constants The discovery_strategies.py / discovery_overrides.py stubs are write-once CUSTOM_FILES, so they kept the wording from an earlier render and never re-synced with the updated documentation constants. Delete and regenerate them so the in-tree stubs match the current model_consumer constants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * krakend: match changelog text to phase 2 spec Use the wording prescribed by the phase 2 plan (T9): "Add configuration discovery support." Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fix template-expansion regressions in discovery config tooling Two issues surfaced by review of the discovery config tooling refactor: - expand_template_items dropped wrapper attributes (e.g. `hidden: true`) when a template resolved to a list, so the common `- template: instances/pdh_legacy` + `hidden: true` pattern used by aspdotnet, dotnetclr, hyperv, exchange_server, and active_directory stopped hiding the expanded options. Propagate the wrapper's leftover attributes to each expanded item via setdefault, restoring the pre-refactor behaviour. - The discovery model consumer emitted candidate templates inside a single-quoted literal, producing invalid discovery.py when a template contained a single quote or backslash. Use repr() to emit a properly escaped literal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add test for hidden propagation on list-template expansion Locks in the fix: a `hidden: true` wrapper on a list template applies to every expanded item, while an item's own explicit `hidden` value wins. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Replace chr(10) workaround with a plain newline join chr(10) was the pre-3.12 workaround for using a backslash inside an f-string expression. Build the joined message first, matching the existing style in expand_template_items. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Restore master template-expansion behavior in spec tooling The discovery-config refactor changed two aspects of option-template expansion in `expand_template_items`, both of which broke config/models validation for existing integrations: - Overrides were scoped per-item, so an override targeting a nested template (e.g. `extra_metrics.value.example` in the Windows perf-counter specs) was attempted prematurely on the parent template, reported as an error, and discarded before the nested template was spliced in and expanded. Restore the shared-override semantics: `apply_overrides` pops each override on success, so unresolved ones are retried against later items, and only truly-unused overrides are reported. - `hidden` propagation was scoped to a template's own expanded items, dropping the historical cross-sibling leak where a template wrapper's `hidden: true` carries onto following plain options until the next template. haproxy relies on this to hide its legacy (non-OpenMetrics) option block. Reintroduce it behind a `propagate_hidden` flag (on for options, off for discovery strategies, which have no `hidden` concept). Rendered example/model output is now byte-identical to master across all 263 integration specs. Adds a regression test for the nested-template override case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * remove test_discovery_stanza_does_not_emit_auto_conf * Stop reserving auto_conf.yaml name in discovery handling Discovery no longer generates auto_conf.yaml (it is hand-maintained per integration), so reserving the example name produced a false-positive collision for the legitimate "discovery stanza + hand-maintained auto_conf.yaml" combination. Drop the reservation, its plumbing through handle_discovery, and the test that pinned the false positive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Drop redundant hidden setdefault in options_validator `expand_template_items(propagate_hidden=True)` now sets `hidden` on every resolved option during expansion, so the `option.setdefault('hidden', False)` in options_validator was a dead no-op that misleadingly read as the source of the default. Remove it; rendered example/model output is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Add discovery/openmetrics_from_ports and auto_conf/discovery spec templates Adds two reusable spec tooling templates: - discovery/openmetrics_from_ports: a from_ports strategy candidate that generates an openmetrics_endpoint URL from the discovered host and port. Integrations override port_hints to set the expected port. - auto_conf/discovery: the common options block for fleet-discovery auto_conf.yaml files (discovery: {}, init_config, instances: []). Each integration sets ad_identifiers independently and includes this template for the rest. Converts the krakend spec to use both templates and regenerates the krakend auto_conf.yaml and config_models/discovery.py from the spec. Also fixes ModelConsumer._process_section to return early for leaf options (no sub-options key) so that fleet-discovery auto_conf.yaml files, which use leaf instances: [] rather than a full instances section, do not crash validate models. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * modify auto_config * Narrow model consumer leaf-instances guard to instances section only The previous fix skipped any section without an `options` key before even checking whether it was init_config or instances. Tighten it to only apply inside the `instances` branch, where auto_conf.yaml files using the fleet-discovery pattern emit a leaf `instances: []` with no model schema. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Restrict leaf-instances guard to auto_conf.yaml files only Pass the spec file name into _process_section and apply the no-options early-return only when processing auto_conf.yaml, where a leaf instances: [] is valid for fleet discovery. Any other file with a leaf instances section still hits the normal code path and fails visibly. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Use by_alias=True when dumping discovery candidate configs Without this, options with hyphenated names (e.g. slowlog-max-len) are emitted as Python field names (slowlog_max_len) in the generated candidate dict. The Agent passes this dict verbatim as the raw instance config, so checks that look up the original hyphenated key would not find it. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * Skip SharedConfig import/usage when spec has no init_config section When a spec declares discovery but omits init_config, no shared.py is generated. The discovery file unconditionally imported SharedConfig, causing ModuleNotFoundError at import time and silently preventing any candidates from being produced. Pass has_shared=False to _build_discovery_file in that case; the generated file emits `shared = {}` instead and omits the import. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * regen --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

renamed folder and added resources to match others

605b0de

technovangelist added a commit that referenced this pull request Dec 22, 2015

Merge pull request #2 from DataDog/dockerupdate

3fecd69

renamed folder and added resources to match others

technovangelist merged commit 3fecd69 into master Dec 22, 2015

gmmeyer deleted the dockerupdate branch December 30, 2016 20:15

gmmeyer added a commit that referenced this pull request Feb 8, 2017

elastic should be #2

c01f16c

ChristineTChen added a commit that referenced this pull request Mar 16, 2018

attempt to fix #2

465dec7

ChristineTChen added a commit that referenced this pull request Mar 19, 2018

attempt to fix #2

818e4a6

dujuku mentioned this pull request Mar 24, 2021

Fix wrong timestamp for DBM beta feature #9024

Merged

4 tasks

ofek mentioned this pull request Nov 12, 2021

Add new base class for monitoring Windows performance counters #10504

Merged

jmeunier28 mentioned this pull request Jan 14, 2022

add new metric for tables size for mysql #10674

Merged

4 tasks

mdkent mentioned this pull request Sep 16, 2022

Allow the disabling of binary log size collection #12960

Closed

4 tasks

iglendd added a commit that referenced this pull request Nov 14, 2022

Attempt to fix spec/model error (Attempt #2)

6d8e553

iglendd added a commit that referenced this pull request Nov 14, 2022

Attempt to fix spec/model error (Attempt #2)

1443c42

iglendd added a commit that referenced this pull request Feb 7, 2023

Fix linting #2

db4b4fc

meklon-imhio mentioned this pull request Mar 3, 2023

[BUG] Error monitoring MySQL - list index out of range #13942

Closed

mahipdeora25 mentioned this pull request Jan 14, 2026

DynamicD First PR #22328

Merged

mwdd146980 added a commit that referenced this pull request Jun 3, 2026

Address finding #2: iter_content uses httpx2 iter_text for unicode

0147459

Refs PR #23883

mwdd146980 added a commit that referenced this pull request Jun 5, 2026

Round 3 fix #2: iter_lines bytes-through fidelity

dbadf00

mwdd146980 added a commit that referenced this pull request Jun 5, 2026

Round 4 fix #2: replace bytes accumulator with bytearray in iter_lines

16d619d

mwdd146980 added a commit that referenced this pull request Jun 5, 2026

Round 5 finding #2: lock in iter_lines decode CRLF parity test

4a77e98

mwdd146980 added a commit that referenced this pull request Jun 5, 2026

Round 6 finding #2: assert request kwargs invariant at import

e0cb6ad

mwdd146980 added a commit that referenced this pull request Jun 5, 2026

Round 7 finding #2: follow-redirects-positive-test-coverage

6520a52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

renamed folder and added resources to match others#2

renamed folder and added resources to match others#2
technovangelist merged 1 commit into
masterfrom
dockerupdate

technovangelist commented Dec 22, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

technovangelist commented Dec 22, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant