feat(batch): add support to SQS FIFO queues (SqsFifoPartialProcessor) - #1934
Merged
rubenfonseca merged 6 commits intoFeb 20, 2023
Merged
Conversation
rubenfonseca
force-pushed
the
rf/batch-sqs-fifo
branch
from
February 16, 2023 12:35
08f4601 to
7686381
Compare
rubenfonseca
marked this pull request as ready for review
February 16, 2023 13:37
rubenfonseca
requested review from
heitorlessa
and removed request for
a team
February 16, 2023 13:37
heitorlessa
suggested changes
Feb 17, 2023
heitorlessa
left a comment
Contributor
There was a problem hiding this comment.
This is great!! Made some initial comments and spotted two issues. After lunch, I'll look over docs and tests.
Thank you @rubenfonseca <3
Codecov ReportBase: 97.45% // Head: 97.43% // Decreases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## develop #1934 +/- ##
===========================================
- Coverage 97.45% 97.43% -0.02%
===========================================
Files 144 146 +2
Lines 6629 6660 +31
Branches 475 478 +3
===========================================
+ Hits 6460 6489 +29
- Misses 132 134 +2
Partials 37 37
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
heitorlessa
reviewed
Feb 17, 2023
heitorlessa
reviewed
Feb 17, 2023
heitorlessa
reviewed
Feb 17, 2023
heitorlessa
reviewed
Feb 17, 2023
heitorlessa
reviewed
Feb 17, 2023
| # GIVEN | ||
| first_record = SQSRecord(sqs_event_factory("success")) | ||
| second_record = SQSRecord(sqs_event_factory("fail")) | ||
| # this would normally suceed, but since it's a FIFO queue, it will be marked as failure |
Contributor
There was a problem hiding this comment.
loved this comment. thank you for your great attention to detail @rubenfonseca
Co-authored-by: Heitor Lessa <lessa@amazon.nl> Signed-off-by: Ruben Fonseca <fonseka@gmail.com>
heitorlessa
reviewed
Feb 20, 2023
heitorlessa
approved these changes
Feb 20, 2023
Signed-off-by: Heitor Lessa <lessa@amazon.nl>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue number: #1927
Summary
Changes
This PR adds support for SQS FIFO queues on the Batch Processing utility.
From docs:
User experience
Before this change, using the
BatchProcessorwith an SQS FIFO would result into undefined behaviour, as we were still continuing to process records after the first failure.Using the new specialized class
SqsFifoPartialProcessormakes the utility safe to use with SQS FIFO.Checklist
If your change doesn't seem to apply, please leave them unchecked.
Is this a breaking change?
RFC issue number:
Checklist:
Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.