Skip to content

Agnosticize integrations HTTP error handling (batch 3 of 3)#24548

Merged
mwdd146980 merged 5 commits into
mwdd146980/http-agnostic-protocol-surfacefrom
mwdd146980/agnosticize-integrations-batch-c
Jul 17, 2026
Merged

Agnosticize integrations HTTP error handling (batch 3 of 3)#24548
mwdd146980 merged 5 commits into
mwdd146980/http-agnostic-protocol-surfacefrom
mwdd146980/agnosticize-integrations-batch-c

Conversation

@mwdd146980

@mwdd146980 mwdd146980 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Removes backend-specific requests references from the final set of integrations in this sweep, leaving each on the HTTP-agnostic surface only. Per integration it narrows self.http catch sites to datadog_checks.base.utils.http_exceptions.* (no dual-catch), swaps bare from requests import ... to agnostic names (including requests.Response type hints to HTTPResponseProtocol), drops now-unused requests imports, and updates the interception-path tests (agnostic exceptions, MockHTTPResponse, and the mock_http/mock_openmetrics_http fixtures).

Integrations covered (batch 3 of 3): mesos_slave, n8n, nifi, nutanix, octopus_deploy, prefect, proxmox, rabbitmq, sonarqube, sonatype_nexus, spark, squid, teamcity, traefik_mesh, yarn.

The backend stays requests. Behavior is unchanged. This PR is based directly on #24516 for now. When #24516 is approved, it will be merged into #22676, and then this PR will be stacked on #22676.

Motivation

Checks and their tests should depend only on the agnostic HTTP surface (http_exceptions.*), never on requests-specific APIs, so the backend can later be swapped without touching consumers.

Verification

  • Touched production packages are free of requests imports/attribute use.
  • Catch sites reference only http_exceptions.*. No (RequestException, HTTPError) dual-catch remains.
  • Per-integration unit tests were run and pass in the source work. ddev test --lint is green across the touched integrations.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

@datadog-official

datadog-official Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests

⚠️ Warnings

🚦 29 Pipeline jobs failed

PR | test / test (linux, ubuntu-22.04, nifi, Apache NiFi (py3.13-2.8), py3.13-2.8) / Apache NiFi (py3.13-2.8)-py3.13-2.8   View in Datadog   GitHub Actions

PR | test / test (linux, ubuntu-22.04, nutanix, Nutanix (py3.13), py3.13) / Nutanix (py3.13)-py3.13   View in Datadog   GitHub Actions

PR | test / test (linux, ubuntu-22.04, sonatype_nexus, sonatype_nexus (py3.13), py3.13) / sonatype_nexus (py3.13)-py3.13   View in Datadog   GitHub Actions

View all 29 failed jobs.

🧪 1 Test failed in 1 job

PR | run   GitHub Actions

test_get_version from test_unit.py   View in Datadog
No module named &#39;datadog_checks.base.utils.http_testing&#39;

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7929b0c | Docs | Datadog PR Page | Give us feedback!

@mwdd146980
mwdd146980 force-pushed the mwdd146980/agnosticize-integrations-batch-c branch from 0c5647f to 8d6f525 Compare July 14, 2026 16:15
@mwdd146980
mwdd146980 force-pushed the mwdd146980/agnosticize-integrations-batch-b branch from 8677a25 to 633eb98 Compare July 14, 2026 16:15
@mwdd146980
mwdd146980 changed the base branch from mwdd146980/agnosticize-integrations-batch-b to mwdd146980/http-agnostic-protocol-surface July 14, 2026 16:15
@mwdd146980 mwdd146980 changed the title Agnosticize integrations HTTP error handling (batch C) Agnosticize integrations HTTP error handling (batch 3 of 3) Jul 14, 2026
@mwdd146980
mwdd146980 marked this pull request as ready for review July 14, 2026 19:23
@mwdd146980
mwdd146980 requested a review from a team as a code owner July 14, 2026 19:23

@NouemanKHAL NouemanKHAL left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the code!

Blocker: changelog entries should either by a .added or omitted in this PR and only generated when merging to master as this is a dev change, we only want a changelog for the user-facing changes (when merging to master)

We'd also probably want ti have a single changelog entry for a migration to an agnostic HTTP layer, for now I see different changelog entries such as:

  • Migrate off requests-specific exceptions to the HTTP-agnostic exception library.
  • Use library-agnostic HTTP exceptions when handling API request failures.
  • Use library-agnostic HTTP exceptions when handling request timeouts.
  • Use library-agnostic HTTP exceptions when a token request returns an unexpected status.

Which are a bit too specific for a user-facing changelog.

Narrow self.http catch sites to backend-agnostic http_exceptions and swap
interception-path tests to the agnostic surface for: mesos_slave, n8n,
nifi, nutanix, octopus_deploy. Keeps the existing n8n/nutanix changelog
entries already present on the base.
Narrow self.http catch sites to backend-agnostic http_exceptions and swap
interception-path tests to the agnostic surface for: proxmox, rabbitmq,
sonarqube, sonatype_nexus, spark, squid, teamcity, traefik_mesh, yarn,
and complete prefect.
…C review)

The agnostic ResponseWrapper.json() raises a stdlib json.JSONDecodeError
(ValueError), not an HTTPError, so a 200 with a non-JSON body escaped
`except HTTPError` in check() and skipped the connect service check that
master emitted (master caught it via the broad RequestException). Also
catch json.JSONDecodeError.
Address non-blocking review nits on the new nifi test helpers:

- Drop the leading underscore from the module-level helpers introduced in
  this PR (_dispatch, _mock_http_responses -> dispatch, mock_http_responses)
  per the repo naming convention for test-file helpers. Pre-existing
  underscore helpers are left untouched (out of scope).
- Route test_overlapping_process_groups_no_duplicates through
  mock_http_responses instead of a hand-rolled side_effect, matching its
  siblings. Behavior is identical (same URL-substring dispatch).
@mwdd146980
mwdd146980 force-pushed the mwdd146980/agnosticize-integrations-batch-c branch from db61d14 to 0d15c84 Compare July 17, 2026 13:18
This PR merges into a feature branch, not master, so changelog
entries aren't needed until the feature branch itself ships.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mwdd146980
mwdd146980 requested a review from NouemanKHAL July 17, 2026 13:19
@mwdd146980
mwdd146980 merged commit cffde7e into mwdd146980/http-agnostic-protocol-surface Jul 17, 2026
45 of 75 checks passed
@mwdd146980
mwdd146980 deleted the mwdd146980/agnosticize-integrations-batch-c branch July 17, 2026 13:22
@dd-octo-sts

dd-octo-sts Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment