|
15 | 15 | // specific language governing permissions and limitations |
16 | 16 | // under the License. |
17 | 17 |
|
| 18 | +//! HyperLogLog sketch implementation |
| 19 | +//! |
| 20 | +//! This module provides the main [`HllSketch`] struct, which is the primary interface |
| 21 | +//! for creating and using HLL sketches for cardinality estimation. |
| 22 | +//! |
| 23 | +//! # Adaptive Mode System |
| 24 | +//! |
| 25 | +//! The sketch automatically transitions between three internal modes based on cardinality: |
| 26 | +//! |
| 27 | +//! - **List mode**: Stores individual coupons in a compact list for small cardinalities. |
| 28 | +//! Used when fewer than ~32 unique values have been seen. |
| 29 | +//! |
| 30 | +//! - **Set mode**: Uses a hash set with open addressing for medium cardinalities. |
| 31 | +//! Provides better performance than list mode while still being space-efficient. |
| 32 | +//! The set grows dynamically until it reaches K/8 entries. |
| 33 | +//! |
| 34 | +//! - **HLL mode**: Uses the full HLL array (Array4, Array6, or Array8) for large cardinalities. |
| 35 | +//! Provides constant memory usage and accurate estimates for billions of unique values. |
| 36 | +//! |
| 37 | +//! Mode transitions are automatic and transparent to the user. Each promotion preserves |
| 38 | +//! all previously observed values and maintains estimation accuracy. |
| 39 | +//! |
| 40 | +//! # Serialization |
| 41 | +//! |
| 42 | +//! Sketches can be serialized and deserialized while preserving all state, including: |
| 43 | +//! - Current mode and HLL type |
| 44 | +//! - All observed values (coupons or register values) |
| 45 | +//! - HIP accumulator state for accurate estimation |
| 46 | +//! - Out-of-order flag for merged/deserialized sketches |
| 47 | +//! |
| 48 | +//! The serialization format is compatible with Apache DataSketches implementations |
| 49 | +//! in Java and C++, enabling cross-platform sketch exchange. |
| 50 | +
|
18 | 51 | use std::hash::Hash; |
19 | 52 | use std::io; |
20 | 53 |
|
|
0 commit comments