bench: add read benchmarks against zarr-python

[kylebarron]

Written by Claude:

---

Adds a bench/ directory with one script that answers a single question: is the native zarrs binding actually faster than zarr-python?

What it does

bench/bench_read.py writes one array with zarr-python, then reads the whole array repeatedly with three implementations reading those same bytes:

Row	What it is
zarr-python	Stock zarr-python with the pure-Python codec pipeline
zarr-python+zarrs	zarr-python with the zarrs Rust codec pipeline plugin
zarrista	zarrista's array[...], then .to_numpy()

The middle row is the point. Without it you cannot tell whether a speed increase comes from Rust codecs or from a native end-to-end binding.

--shards selects the scenario: give it for a sharded array, omit it for a plain chunked one.

Results

Apple M-series laptop, 10 threads, release build, 8.4 MB uint16 array.

Sharded (--shards 512,512):

implementation           best (ms)   median (ms)   median MB/s    vs zarr-python
zarr-python                  81.97         83.49           100             1.00x
zarr-python+zarrs             2.60          2.66          3157            31.42x
zarrista                      2.34          2.40          3502            34.86x

Plain chunked:

implementation           best (ms)   median (ms)   median MB/s    vs zarr-python
zarr-python                 193.82        197.89            42             1.00x
zarr-python+zarrs            21.28         21.61           388             9.16x
zarrista                     10.78         11.19           749            17.68x

The two Rust rows are close on the sharded array, but zarrista is about twice as fast as the zarrs codec pipeline on the plain chunked array.

Notes on method

• Every implementation does one read checked against the source data before any timing is reported.
• .to_numpy() is inside zarrista's timed region, since the other two rows already return a NumPy array.
• --threads pins zarr-python's threading.max_workers and rayon's RAYON_NUM_THREADS to the same value. Rayon reads that variable only when it first builds its global pool, so the script sets it before importing zarrista or zarrs. Every extension import is therefore function-local.
• zarrista must be built with --release. A debug build makes the numbers meaningless, and Python cannot detect the build profile at run time. bench/README.md states this first, and the printed header repeats it.

Not covered

In-memory store (zarrista.MemoryStore() has no API to ingest external bytes, so the two libraries could not read one set of bytes), partial/strided reads, writes, and remote object stores.

Starting point was @d-v-b's gist from zarr-developers/zarr-python#4064.

🤖 Generated with Claude Code