URL Length check for locate_data#3356
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3356 +/- ##
=======================================
Coverage 70.07% 70.07%
=======================================
Files 232 232
Lines 19893 19893
=======================================
Hits 13940 13940
Misses 5953 5953 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
This looks good to me, and I like the idea of breaking up queries that are too long. As long as HEASARC is OK with getting a bunch of split up GET requests, this seems like a good solution. Just to check - there isn't a different API that accepts large POST requests, right? That would be a simpler fallback if so. |
In fact there is, I've updated the PR to bypass the |
|
Sounds all this good, flip the PR status out from draft when you think this is ready for in depth review/merge. |
Fixes #3276.
Considering a standard URL character limit of ~2000 for GET requests, this PR checks the number of targets being queried. Assuming a maximum ID length of 16 (based on the heuristic of not having seen IDs approaching this length personally...), the query could support ~100 IDs at a time before running into the character limit for the URL. This PR splits a query into multiple chunks to avoid this limit and combines the result into a single table.
This PR is created as a draft as the assumptions I've made could merit further discussion, and I'm also unsure if this is the desired behavior rather than simply failing when running into a query that is too long.