Skip to content

Fix XCom migration failing for NaN/Infinity float values (#62686)#62760

Merged
jedcunningham merged 1 commit into
apache:v3-1-testfrom
astronomer:backport_xcom_migration_nan_inf
Mar 3, 2026
Merged

Fix XCom migration failing for NaN/Infinity float values (#62686)#62760
jedcunningham merged 1 commit into
apache:v3-1-testfrom
astronomer:backport_xcom_migration_nan_inf

Conversation

@jedcunningham

Copy link
Copy Markdown
Member

XCom values containing float('nan'), float('inf'), or float('-inf') caused the database migration to silently corrupt data or fail outright when upgrading. Three bugs were present across backends:

  • Consecutive tokens (e.g. [NaN, NaN]) were only partially replaced, leaving bare NaN/Infinity in the output and breaking the JSON cast.
  • Infinity and -Infinity were not handled at all — only NaN was.
  • Bare top-level values (a single NaN or Infinity, not inside a list or dict) were not matched and passed through unconverted.

MySQL also had two bugs in the replacement query that caused it to produce the wrong output (one of these was pre-existing from #57866).

Backport of #62686
(cherry picked from commit 7a301e6)

XCom values containing float('nan'), float('inf'), or float('-inf')
caused the database migration to silently corrupt data or fail
outright when upgrading. Three bugs were present across backends:

- Consecutive tokens (e.g. [NaN, NaN]) were only partially replaced,
  leaving bare NaN/Infinity in the output and breaking the JSON cast.
- Infinity and -Infinity were not handled at all — only NaN was.
- Bare top-level values (a single NaN or Infinity, not inside a list
  or dict) were not matched and passed through unconverted.

MySQL also had two bugs in the replacement query that caused it to produce
the wrong output (one of these was pre-existing from apache#57866).

Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
(cherry picked from commit 7a301e6)
@jedcunningham jedcunningham merged commit 17910dd into apache:v3-1-test Mar 3, 2026
65 checks passed
@jedcunningham jedcunningham deleted the backport_xcom_migration_nan_inf branch March 3, 2026 06:42
@vatsrahul1001 vatsrahul1001 added the type:bug-fix Changelog: Bug Fixes label Mar 3, 2026
@vatsrahul1001 vatsrahul1001 added this to the Airflow 3.1.8 milestone Mar 3, 2026
vatsrahul1001 added a commit that referenced this pull request Mar 4, 2026
…2760)

XCom values containing float('nan'), float('inf'), or float('-inf')
caused the database migration to silently corrupt data or fail
outright when upgrading. Three bugs were present across backends:

- Consecutive tokens (e.g. [NaN, NaN]) were only partially replaced,
  leaving bare NaN/Infinity in the output and breaking the JSON cast.
- Infinity and -Infinity were not handled at all — only NaN was.
- Bare top-level values (a single NaN or Infinity, not inside a list
  or dict) were not matched and passed through unconverted.

MySQL also had two bugs in the replacement query that caused it to produce
the wrong output (one of these was pre-existing from #57866).


(cherry picked from commit 7a301e6)

Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:db-migrations PRs with DB migration kind:documentation type:bug-fix Changelog: Bug Fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants