fix(release-controller): stop resubmitting election proposals for already-elected versions - #2023
Merged
pietrodimarco-dfinity merged 2 commits intoMay 26, 2026
Conversation
…eady-elected versions The release-controller resubmitted three different NNS election proposals for commit c7552446... (141776 -> failed, 141782 -> executed, then duplicates 141879 and 141880) because two latent bugs combined badly with a stale `ignored_proposals` entry left in `release-index.yaml` after the replacement proposal was submitted: 1. `get_election_proposals_by_version` in `public_dashboard.py` and `dre_cli.py` had a nested loop and a blind `d[version] = proposal` that kept *whichever proposal was iterated last* per version. Given that the dashboard API returns proposals newest-first, that meant the *oldest* (here the rejected 141776) won. After `_filtered_proposals_retriever` then stripped 141776, the version looked unproposed every cycle, so each pod restart submitted a new one. 2. The proposal-submission phase already fetched the elected (`blessed`) versions to compute `unelect_versions`, but it never used that set to short-circuit the submission, so the governance canister was left to refuse every duplicate on its own. Both fixes: - Replace the buggy per-version aggregation with `proposals_by_version`, a free helper that keeps the highest-id proposal per version and applies `ignored_proposals` filtering at the raw-list level *before* aggregation, so a stale ignored id can't accidentally hide an otherwise-valid proposal that targets the same version. - Add a `release_commit in blessed` short-circuit to the `proposal submission` phase that warns loudly and skips submission when the target version is already elected -- a safety net for any future operator mistake of the same shape. Also drop the now-stale `141776` entry from `release-index.yaml` and update the README so the example uses a synthetic id and the caveats describe the new fallback-to-next-highest-id behaviour.
pietrodimarco-dfinity
enabled auto-merge (squash)
May 26, 2026 11:08
NikolaMilosa
approved these changes
May 26, 2026
Contributor
Author
|
Reopening to retrigger CI after self-hosted runner outage cleared. |
pietrodimarco-dfinity
deleted the
pmarco/release-controller-fix-duplicate-submission
branch
May 26, 2026 13:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The release-controller fired three different NNS election proposals for commit
c7552446...(141776 -> failed, 141782 -> executed, then duplicates 141879 and 141880, both rejected by governance) because two latent bugs combined badly with a staleignored_proposalsentry that nobody removed once 141782 was executed:get_election_proposals_by_versioninpublic_dashboard.pyanddre_cli.pyhad a nested loop and a blindd[version] = proposalassignment that kept whichever proposal was iterated last per version. The dashboard API returns proposals newest-first, so the oldest (the rejected 141776) won. After_filtered_proposals_retrieverthen stripped 141776, the version looked unproposed on every cycle, and each pod restart submitted yet another one.blessed) versions to computeunelect_versions, but it never used that set to short-circuit the submission, so the governance canister was left to refuse every duplicate on its own.What
dre_cli.proposals_by_version, a free helper that:ignored_proposalsfiltering at the raw-list level before aggregation, so a stale ignored id can't hide a perfectly good non-ignored proposal targeting the same version -- the wrapper falls back to the next-highest-id automatically.Reconciler._filtered_proposals_retrieverto take a raw-list retriever (get_past_election_proposals) and call the new helper, so the filter happens at the right layer.release_commit in blessedshort-circuit to theproposal submissionphase that warns loudly and skips submission when the target version is already elected -- safety net for any future operator mistake of the same shape.141776entry fromrelease-index.yamland update the README's example to a synthetic id, plus document the new fallback-to-next-highest-id behaviour in the caveats.Tests
New tests in
tests/test_reconciler_integration.py:test_dashboard_get_election_proposals_by_version_keeps_highest_id_for_duplicates-- guards the dashboard aggregator.test_dre_cli_get_election_proposals_by_version_keeps_highest_id_for_duplicates-- guards the dre_cli aggregator.test_reconciler_filtered_proposals_retriever_falls_back_to_next_highest_id-- guards the filter-before-aggregation ordering (this is the user's exact scenario).test_reconciler_skips_submission_when_version_already_in_blessed_set-- guards the safety net.Updated tests:
test_reconciler_filtered_proposals_retriever_drops_ignored_ids/..._is_noop_without_ids-- migrated to the new raw-list retriever signature.test_reconciler_reconciles_without_error_already_submitted_proposals-- expected forum-post id updated from138814to138817to reflect the now-correct highest-id-wins behaviour.Local runs: