Raise agnostic HTTP exceptions and narrow integration catch sites#24310
Merged
mwdd146980 merged 23 commits intoJul 13, 2026
Merged
Conversation
Contributor
|
…r boundary Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mwdd146980
force-pushed
the
mwdd146980/floor-exception-translation
branch
from
July 2, 2026 22:04
e2668ca to
35c85f5
Compare
…exception-translation
Cover the translator directly (full mapping table with an exact-type ordering guard), the streaming seam (iter_content/iter_lines), the buffered seam (content/text/json via a failing-response stand-in), and JSON parse convergence to the stdlib json.JSONDecodeError. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The wrapper now raises HTTPConnectionError, which the requests-only catch in stream_connection_lines no longer matched, silently disabling ignore_connection_errors for OM v2. Narrow the catch to the agnostic type and drop the dead requests import. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ix poll docstrings Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… raw response Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dary ResponseWrapper overrode iter_content/iter_lines but not __iter__, so a direct `for chunk in response` forwarded through wrapt to the raw requests.Response, whose __iter__ delegates to the raw iter_content and re-raised untranslated requests exceptions. Override __iter__ to route through the translating iter_content, keeping the wrapper the sole translation choke point. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-2 review of PR #24310 found the OpenMetrics v2 stream scraper narrowed its `except ConnectionError` to `except HTTPConnectionError`. The translator maps requests' ConnectTimeout to HTTPTimeoutError (a sibling, not a subclass), so connect timeouts under ignore_connection_errors=True escaped instead of being swallowed. Catch both agnostic types and parametrize the test over them. Also address the round's minor findings: - route ResponseWrapper.raise_for_status through _translate_requests_exception via a new optional response= param, dropping duplicated extraction - add type hints to the two new module-level translation helpers - drop the unlinked Step 0.1 subpage reference from the translator docstring - finish the half-migrated poll() docstrings (requests.Response -> response) - add the 24310.changed changelog entry Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-3 review polish for PR #24310: - annotate the response kwarg on _translate_requests_exception (ResponseWrapper | None) - declare __iter__ on HTTPResponseProtocol so the frozen contract matches the ResponseWrapper.__iter__ seam this PR added Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HTTPResponseProtocol gained __iter__ in this PR; give the mock the same iter_content(128) delegation so a test iterating a mock_http response doesn't raise TypeError. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The (HTTPConnectionError, HTTPTimeoutError) catch in stream_connection_lines also swallows a mid-stream ReadTimeout under ignore_connection_errors, since the agnostic hierarchy has no connect-vs-read timeout distinction. Document this as intentional per round-5 review feedback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
24310 merges into the httpx-migration-base feature branch (PR 22676), which merges to master. Rename 24310.changed to 22676.changed so the behavior-change entry appears under the PR that actually reaches master. It coexists with the existing 22676.added (additive infra); together they reflect the full batch, and the .changed entry preserves the major version bump. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fold 22676.added into 22676.changed so the batch merging to master carries a single entry. The behavior change (agnostic exceptions) makes the whole batch a major bump, so the consolidated entry is typed changed and also covers the added agnostic mocks/proto/exceptions and test migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3 tasks
The base RequestsWrapper now raises the backend-agnostic exceptions in datadog_checks.base.utils.http_exceptions instead of requests.exceptions, so these checks must catch the agnostic types to keep working after the httpx migration. Narrow the production catch sites and update the coupled test assertions accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route the seven bare requests calls (three instance methods and the four auth/catalog classmethods, via a threaded http client) through self.http, so they raise backend-agnostic exceptions after the httpx migration. Narrow the ~10 catch sites to the agnostic types and drop import requests. ssl_verify is an init_config option here, so bridge it to the instance-level tls_verify the shared client resolves. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…exception-translation
The agnostic reachability-failure tuple (HTTPStatusError, HTTPTimeoutError, HTTPConnectionError) was repeated at six catch sites. Extract it into the API_TRANSPORT_ERRORS module constant so the next migration batch has a single place to update. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
request_auth_token, get_token_for_project, and request_project_list dereference http immediately and are always called with a real client. Drop the http=None default so a missing argument fails fast with a TypeError at the call site instead of an opaque AttributeError deep in the method. The from_config entry points keep the default because they can raise IncompleteConfig before http is used. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The standalone openstack integration is legacy and close to deprecation. Per team decision, leave it exactly as it is rather than migrate it onto the shared HTTP client. This backs out all of this PR's openstack changes (self.http routing, the ssl_verify to tls_verify bridge, the required-http signatures, and the agnostic catch-site narrowing), restoring the file to the feature-branch base. openstack keeps its bare requests calls and catches requests exceptions, so it is unaffected by the wrapper exception translation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route the OpenStack API and Keystone auth requests through the shared HTTP client and catch the backend-agnostic HTTPStatusError, HTTPTimeoutError, and HTTPConnectionError instead of requests exceptions. Behavior is preserved: the Keystone auth request keeps its legacy default of skipping TLS verification unless ssl_verify is set, carried through as an explicit per-call verify override rather than inheriting the shared client's verify-by-default. The dormant unscoped-catalog fallback is left in place untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mwdd146980
marked this pull request as ready for review
July 9, 2026 15:36
Rewrite the httpx floor changelog entries in plain language centered on the actual change (backend-agnostic HTTP exceptions), dropping the internal migration references and the exception class-name lists. The base entry keeps its .changed classification since raising different exception types is breaking for anything catching requests errors off self.http. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
triviajon
approved these changes
Jul 9, 2026
triviajon
left a comment
Contributor
There was a problem hiding this comment.
Migrating to generic HTTP erros in the openmetrics scraper LGTM
| self.log.error("Invalid SSL settings for requesting %s endpoint", endpoint) | ||
| raise | ||
| except IOError: | ||
| except (IOError, HTTPRequestError): |
Contributor
Author
There was a problem hiding this comment.
yes, basically because transport failures are translated as HTTPRequestError or a subclass here and that no longer subclasses IOError, but it's still possible for an IOError to be raised (e.g. FileNotFoundError)
…exception-translation # Conflicts: # teleport/changelog.d/22676.fixed
3 tasks
NouemanKHAL
approved these changes
Jul 13, 2026
Contributor
Validation ReportAll 21 validations passed. Show details
|
mwdd146980
merged commit Jul 13, 2026
2eaab66
into
mwdd146980/httpx-migration-base
344 of 347 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Establishes the backend-agnostic exception floor for the httpx migration: the shared HTTP wrapper in
datadog_checks_baseraises library-neutral exception types, and the integrations that catch HTTP errors are moved onto those types in the same change.Note on the openstack integration: This integration specifically has some bugs that AI reviewers may flag, but given that it is legacy/deprecated in favor of the openstack_controller integration, we've decided to preserve existing behavior.
Motivation
Integrations today catch
requests.exceptions.*directly, which couples every check to the HTTP library and blocks any incremental move to httpx. Translating errors once, in the wrapper, turns the underlying library into an implementation detail: integrations catch stable, neutral exceptions and won't need to change again when the transport is swapped.The base change and the integration updates are inseparable. The moment the wrapper stops emitting
requests.exceptions.*, any catch site still keyed to those types silently stops matching, for example a health check that quietly fails to report CRITICAL. Shipping them together, and on themwdd146980/httpx-migration-basefeature branch rather than master, guarantees no Agent release is ever exposed to that half-migrated state.Verification
Base and per-integration unit suites pass;
openstackwas additionally verified locally. Docker-gated suites rely on CI. Runddev test -fs datadog_checks_basefor the base package.Review checklist (to be filled by reviewers)
qa/requiredif this PR needs QA validation, orqa/skip-qaif it does not. Exactly one of the two is required.backport/<branch-name>label to the PR and it will automatically open a backport PR once this one is merged