Skip to content

fix(backfill): populate partition_date on partitioned backfill runs#65998

Merged
Lee-W merged 3 commits into
apache:mainfrom
astronomer:calendar-backfilled-partitions
May 3, 2026
Merged

fix(backfill): populate partition_date on partitioned backfill runs#65998
Lee-W merged 3 commits into
apache:mainfrom
astronomer:calendar-backfilled-partitions

Conversation

@Lee-W

@Lee-W Lee-W commented Apr 28, 2026

Copy link
Copy Markdown
Member

Why

The Calendar view filters historical runs of partitioned Dags by partition_date, but the partitioned-backfill code path only sets partition_key, leaving partition_date NULL. As a result, backfilled partition runs were silently excluded from the calendar. The scheduler already populates both fields when creating scheduled partition runs — this was an oversight on the backfill side.

What

  • Forward partition_date=info.partition_date through create_dagrun in create_backfill_dag_run_partitioned, mirroring what the scheduler already does.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: [Claude] following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@Lee-W Lee-W requested review from XD-DENG and ashb as code owners April 28, 2026 06:26
@Lee-W Lee-W moved this to In Review in AIP-76 Asset Partitioning Apr 28, 2026
@Lee-W Lee-W self-assigned this Apr 29, 2026
Comment thread airflow-core/tests/unit/models/test_backfill.py Outdated
Lee-W added 3 commits April 29, 2026 23:13
Calendar view filters partitioned Dags by partition_date, but the
backfill path only set partition_key — so backfilled partitions
were dropped from the calendar.
@Lee-W Lee-W force-pushed the calendar-backfilled-partitions branch from 9057787 to af4c3ae Compare April 29, 2026 15:17
@Lee-W Lee-W requested a review from uranusjr April 30, 2026 09:07
@Lee-W Lee-W merged commit 9b18d23 into apache:main May 3, 2026
78 checks passed
@Lee-W Lee-W deleted the calendar-backfilled-partitions branch May 3, 2026 07:09
@github-project-automation github-project-automation Bot moved this from In Review to Done in AIP-76 Asset Partitioning May 3, 2026
Lee-W added a commit to astronomer/airflow that referenced this pull request May 5, 2026
shahar1 pushed a commit that referenced this pull request May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

3 participants