Skip to content

test: peer tests for size-balanced partitioner#16

Merged
cigrainger merged 1 commit into
mainfrom
cg/partitioner-test-coverage
Mar 24, 2026
Merged

test: peer tests for size-balanced partitioner#16
cigrainger merged 1 commit into
mainfrom
cg/partitioner-test-coverage

Conversation

@cigrainger

Copy link
Copy Markdown
Contributor

Summary

  • Add :peer-based distributed tests for the size-balanced partitioner
  • 5 new tests covering end-to-end correctness with real BEAM peer nodes and actual Parquet files on disk

Tests added

Test What it verifies
Skewed files produce correct results 1x5000-row + 5x20-row files → SUM/COUNT/MIN/MAX match expected
Filter + group_by on skewed data Distributed result matches local result exactly
AVG + STDDEV on skewed data Numerical accuracy (Welford merge) with highly uneven partitions
DuckLake file list simulation {:ducklake_files, paths} source distributes and reads correctly
Single large file with multiple workers Replicated read still produces correct aggregation

Test plan

  • mix test --include distributed test/dux/partitioner_peer_test.exs — 5 tests, 0 failures
  • mix check — 578 non-distributed tests pass, Credo clean

🤖 Generated with Claude Code

End-to-end distributed tests using :peer for the size-balanced
partitioner: skewed file correctness, filter+group_by, AVG+STDDEV
numerical accuracy, DuckLake file list simulation, and single-file
replication edge case.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@cigrainger
cigrainger merged commit 1e0bdf2 into main Mar 24, 2026
3 checks passed
@cigrainger
cigrainger deleted the cg/partitioner-test-coverage branch March 24, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant