fix(amber): correct WorkerSpec input port to stop flaky timeouts#6037
fix(amber): correct WorkerSpec input port to stop flaky timeouts#6037Ma77Ball wants to merge 3 commits into
Conversation
Automated Reviewer SuggestionsBased on the
|
✅ No material benchmark regressions detected🟢 5 better · 🔴 0 worse · ⚪ 10 noise (<±5%) · 0 without baseline
Baseline detailsLatest main
Raw CSVconfig_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,338.22,200,128000,591,0.361,16163.87,22946.85,22946.85
1,100,10,64,20,1610.60,2000,1280000,1242,0.758,78658.20,98797.54,98797.54
2,1000,10,64,20,14231.84,20000,12800000,1405,0.858,706209.65,776519.38,776519.38 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6037 +/- ##
============================================
+ Coverage 54.50% 56.87% +2.37%
+ Complexity 2915 1901 -1014
============================================
Files 1108 843 -265
Lines 42807 34646 -8161
Branches 4604 3480 -1124
============================================
- Hits 23332 19705 -3627
+ Misses 18119 13986 -4133
+ Partials 1356 955 -401
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Updated error handling comment for clarity. Signed-off-by: Matthew B. <mgball@uci.edu>
| // Re-throw Errors (e.g. failed assertions) after replying; only | ||
| // Exceptions are returned to the sender and recovered from. | ||
| if (err.isInstanceOf[Error]) throw err |
There was a problem hiding this comment.
how come non errors can reach here? that's a bigger problem...
with your change, this "non-error" is omitted silently...
can we at least log warning when it happens?
There was a problem hiding this comment.
Dropped the throw since it can take down the actor loop. The Error was an AssertionError from the test's mismatched addPort args; it now just logs a warning so it's not silent.
Yicong-Huang
left a comment
There was a problem hiding this comment.
The src code seems to have a bigger issue. we should not make source code less robust to satisfy test cases...
|
Reverted the re-throw. Fixed the real root cause: the test passed mismatched |
What changes were proposed in this PR?
WorkerSpec's input-portAssignPortRequestnow passes an emptystorageUris(List()) matching its emptypartitionings, satisfyingInputManager.addPort'ssizeinvariant, so the input port registers and the three worker tests pass deterministically.AsyncRPCServer.invokeMethodnow re-throwsErrors (e.g. failed assertions) after replying to the sender, instead of swallowing them; normalExceptions are still returned to the sender unchanged, so genuine invariant violations surface loudly rather than degrading into flaky timeouts.Any related issues, documentation, discussions?
Closes: #6036
How was this PR tested?
sbt "WorkflowExecutionService/testOnly *WorkerSpec"; expectTests: succeeded 3, failed 0.assertion failed. Before this fix the count is 3 (the assertion fired and was swallowed every run); after it is 0. Verified locally under Java 17.AsyncRPCServerchange is left to the amber CI job to exercise across all RPC handlers.Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8 in compliance with ASF