Python 3.12 modernizations: itertools.batched, collections.abc.Buffer, math.prod, pathlib

changes/3417.bugfix.md

@@ -0,0 +1,4 @@
+Fixed `BytesCodec.from_dict` so that `BytesCodec` instances roundtrip to / from
+their dict representation. `BytesCodec.from_dict` now interprets a missing
+`endian` configuration as `endian=None` (matching what `BytesCodec.to_dict`
+emits), instead of falling back to the system's native byte order.

changes/3987.feature.md

@@ -0,0 +1 @@
+Two new fields on `ArrayConfig` control how the sharding codec coalesces partial-shard reads: `sharding_coalesce_max_gap_bytes` (default 1 MiB) and `sharding_coalesce_max_bytes` (default 16 MiB). When reading multiple chunks from the same shard, nearby byte ranges are merged into a single request to the store if separated by no more than `sharding_coalesce_max_gap_bytes` and the merged read stays within `sharding_coalesce_max_bytes`. Defaults are seeded from the matching `array.sharding_coalesce_max_gap_bytes` / `array.sharding_coalesce_max_bytes` keys in [`zarr.config`][] at array-creation time, and can be overridden per array by passing `config={...}` to [`zarr.create_array`][].

changes/4074.bugfix.md

@@ -0,0 +1,7 @@
+Fixed several storage and codec bugs:
+
+- Reading a value with a `SuffixByteRequest` larger than the value now correctly returns the whole value (matching HTTP `bytes=-N` suffix-range semantics), instead of silently returning incorrect data for `MemoryStore`.
+- `LoggingStore.get_partial_values` and `FsspecStore.get_partial_values` no longer return empty results when `key_ranges` is passed as a one-shot iterable (e.g. a generator).
+- `Store.getsize_prefix` no longer over-counts sibling keys that merely share a string prefix (e.g. `getsize_prefix("foo")` no longer includes keys under `foobar/`).
+- `ZipStore.close()` no longer raises `AttributeError` when the store was created but never opened (including when used as a context manager without any I/O).
+- `codecs_from_list` now raises a descriptive `TypeError` when a `BytesBytesCodec` immediately follows an `ArrayArrayCodec`, instead of a misleading "Required ArrayBytesCodec was not found" `ValueError`.

docs/user-guide/config.md

@@ -35,6 +35,7 @@ Configuration options include the following:
 - Async and threading options, e.g. `async.concurrency` and `threading.max_workers`
 - Selections of implementations of codecs, codec pipelines and buffers
 - Enabling GPU support with `zarr.config.enable_gpu()`. See GPU support for more.
+- Control request merging when reading multiple chunks from the same shard with `array.sharding_coalesce_max_gap_bytes` and `array.sharding_coalesce_max_bytes`. Reads of nearby chunks are coalesced into a single request to the store when separated by at most `sharding_coalesce_max_gap_bytes` and the resulting merged read is no larger than `sharding_coalesce_max_bytes`.
 
 For selecting custom implementations of codecs, pipelines, buffers and ndbuffers,
 first register the implementations in the registry and then select them in the config.

packages/zarr-metadata/CHANGELOG.md

@@ -2,6 +2,34 @@
 
 <!-- towncrier release notes start -->
 
+## 0.3.0 (2026-06-19)
+
+### Deprecations and Removals
+
+- Introduces a new `JSONValue` type that models python objects that serialize directly to JSON. This type is used to annotate the contents of `attributes` and `fill_value` fields, replacing the use of the overly wide `object` type. This is technically a breaking change. ([#4037](https://github.com/zarr-developers/zarr-python/issues/4037))
+- Promoted a curated "front door" of names to the top-level `zarr_metadata`
+  namespace, so consumers can write e.g. `from zarr_metadata import
+  ArrayMetadataV3, ShardingIndexLocation, BLOSC_CNAME` instead of importing from
+  deep submodule paths. The front door covers every metadata-document TypedDict,
+  each codec/chunk-grid/chunk-key-encoding canonical type, the full data-type
+  trio for every dtype, and every constant + `Literal` pair. Deep submodule paths
+  continue to work unchanged.
+
+  Several promoted names were given clearer, less ambiguous spellings than their
+  deep-module names, since they now appear bare at the top level:
+  `Endian`/`ENDIAN` → `Endianness`/`ENDIANNESS`,
+  `IndexLocation`/`INDEX_LOCATION` → `ShardingIndexLocation`/`SHARDING_INDEX_LOCATION`,
+  `RoundingMode`/`ROUNDING_MODE` → `CastRoundingMode`/`CAST_ROUNDING_MODE`,
+  `OutOfRangeMode`/`OUT_OF_RANGE_MODE` → `CastOutOfRangeMode`/`CAST_OUT_OF_RANGE_MODE`,
+  `DateTimeUnit` → `NumpyTimeUnit`,
+  `NamedConfig` → `NamedConfigV3`, and
+  `MetadataFieldV3` → `MetadataV3` (matching the name `zarrs` uses for this
+  `name`-or-`{name, configuration}` shape).
+
+  Also added the `NUMPY_TIME_UNIT` runtime constant (a `Final` tuple paired with
+  the `NumpyTimeUnit` Literal) in `zarr_metadata.v3.data_type.numpy_timedelta64`. ([#4083](https://github.com/zarr-developers/zarr-python/issues/4083))
+
+
 ## 0.2.0 (2026-05-19)
 
 ### Bugfixes

packages/zarr-metadata/src/zarr_metadata/__init__.py

@@ -1,7 +1,9 @@
 from importlib.metadata import version
 
-from zarr_metadata._common import JSONValue, NamedConfig
+from zarr_metadata._common import JSONValue, NamedConfigV3
 from zarr_metadata.v2.array import (
+    ARRAY_DIMENSION_SEPARATOR_V2,
+    ARRAY_ORDER_V2,
     ArrayDimensionSeparatorV2,
     ArrayMetadataV2,
     ArrayMetadataV2Partial,
@@ -13,35 +15,320 @@
 from zarr_metadata.v2.codec import CodecMetadataV2
 from zarr_metadata.v2.consolidated import ConsolidatedMetadataV2
 from zarr_metadata.v2.group import GroupMetadataV2, GroupMetadataV2Partial, ZGroupMetadata
-from zarr_metadata.v3._common import MetadataFieldV3
+from zarr_metadata.v3._common import MetadataV3
 from zarr_metadata.v3.array import ArrayMetadataV3, ArrayMetadataV3Partial, ExtensionFieldV3
+from zarr_metadata.v3.chunk_grid.rectilinear import (
+    RECTILINEAR_CHUNK_GRID_NAME,
+    RectilinearChunkGridMetadata,
+    RectilinearChunkGridName,
+)
+from zarr_metadata.v3.chunk_grid.regular import (
+    REGULAR_CHUNK_GRID_NAME,
+    RegularChunkGridMetadata,
+    RegularChunkGridName,
+)
+from zarr_metadata.v3.chunk_key_encoding.default import (
+    DEFAULT_CHUNK_KEY_ENCODING_NAME,
+    DEFAULT_CHUNK_KEY_ENCODING_SEPARATOR,
+    DefaultChunkKeyEncodingMetadata,
+    DefaultChunkKeyEncodingName,
+    DefaultChunkKeyEncodingSeparator,
+)
+from zarr_metadata.v3.chunk_key_encoding.v2 import (
+    V2_CHUNK_KEY_ENCODING_NAME,
+    V2_CHUNK_KEY_ENCODING_SEPARATOR,
+    V2ChunkKeyEncodingMetadata,
+    V2ChunkKeyEncodingName,
+    V2ChunkKeyEncodingSeparator,
+)
+from zarr_metadata.v3.codec.blosc import (
+    BLOSC_CNAME,
+    BLOSC_CODEC_NAME,
+    BLOSC_SHUFFLE,
+    BloscCName,
+    BloscCodecMetadata,
+    BloscCodecName,
+    BloscShuffle,
+)
+from zarr_metadata.v3.codec.bytes import (
+    BYTES_CODEC_NAME,
+    ENDIANNESS,
+    BytesCodecMetadata,
+    BytesCodecName,
+    Endianness,
+)
+from zarr_metadata.v3.codec.cast_value import (
+    CAST_OUT_OF_RANGE_MODE,
+    CAST_ROUNDING_MODE,
+    CAST_VALUE_CODEC_NAME,
+    CastOutOfRangeMode,
+    CastRoundingMode,
+    CastValueCodecMetadata,
+    CastValueCodecName,
+)
+from zarr_metadata.v3.codec.crc32c import CRC32C_CODEC_NAME, Crc32cCodecMetadata, Crc32cCodecName
+from zarr_metadata.v3.codec.gzip import GZIP_CODEC_NAME, GzipCodecMetadata, GzipCodecName
+from zarr_metadata.v3.codec.scale_offset import (
+    SCALE_OFFSET_CODEC_NAME,
+    ScaleOffsetCodecMetadata,
+    ScaleOffsetCodecName,
+)
+from zarr_metadata.v3.codec.sharding_indexed import (
+    SHARDING_INDEX_LOCATION,
+    SHARDING_INDEXED_CODEC_NAME,
+    ShardingIndexedCodecMetadata,
+    ShardingIndexedCodecName,
+    ShardingIndexLocation,
+)
+from zarr_metadata.v3.codec.transpose import (
+    TRANSPOSE_CODEC_NAME,
+    TransposeCodecMetadata,
+    TransposeCodecName,
+)
+from zarr_metadata.v3.codec.zstd import ZSTD_CODEC_NAME, ZstdCodecMetadata, ZstdCodecName
 from zarr_metadata.v3.consolidated import ConsolidatedMetadataV3
+from zarr_metadata.v3.data_type.bool import (
+    BOOL_DATA_TYPE_NAME,
+    BoolDataTypeName,
+    BoolFillValue,
+)
+from zarr_metadata.v3.data_type.bytes import (
+    BYTES_DATA_TYPE_NAME,
+    BytesDataTypeName,
+    BytesFillValue,
+)
+from zarr_metadata.v3.data_type.complex64 import (
+    COMPLEX64_DATA_TYPE_NAME,
+    Complex64DataTypeName,
+    Complex64FillValue,
+)
+from zarr_metadata.v3.data_type.complex128 import (
+    COMPLEX128_DATA_TYPE_NAME,
+    Complex128DataTypeName,
+    Complex128FillValue,
+)
+from zarr_metadata.v3.data_type.float16 import (
+    FLOAT16_DATA_TYPE_NAME,
+    Float16DataTypeName,
+    Float16FillValue,
+)
+from zarr_metadata.v3.data_type.float32 import (
+    FLOAT32_DATA_TYPE_NAME,
+    Float32DataTypeName,
+    Float32FillValue,
+)
+from zarr_metadata.v3.data_type.float64 import (
+    FLOAT64_DATA_TYPE_NAME,
+    Float64DataTypeName,
+    Float64FillValue,
+)
+from zarr_metadata.v3.data_type.int8 import (
+    INT8_DATA_TYPE_NAME,
+    Int8DataTypeName,
+    Int8FillValue,
+)
+from zarr_metadata.v3.data_type.int16 import (
+    INT16_DATA_TYPE_NAME,
+    Int16DataTypeName,
+    Int16FillValue,
+)
+from zarr_metadata.v3.data_type.int32 import (
+    INT32_DATA_TYPE_NAME,
+    Int32DataTypeName,
+    Int32FillValue,
+)
+from zarr_metadata.v3.data_type.int64 import (
+    INT64_DATA_TYPE_NAME,
+    Int64DataTypeName,
+    Int64FillValue,
+)
+from zarr_metadata.v3.data_type.numpy_datetime64 import (
+    NUMPY_DATETIME64_DATA_TYPE_NAME,
+    NumpyDatetime64DataTypeName,
+    NumpyDatetime64FillValue,
+)
+from zarr_metadata.v3.data_type.numpy_timedelta64 import (
+    NUMPY_TIME_UNIT,
+    NUMPY_TIMEDELTA64_DATA_TYPE_NAME,
+    NumpyTimedelta64DataTypeName,
+    NumpyTimedelta64FillValue,
+    NumpyTimeUnit,
+)
+from zarr_metadata.v3.data_type.raw import RawBytesDataTypeName, RawBytesFillValue
+from zarr_metadata.v3.data_type.string import (
+    STRING_DATA_TYPE_NAME,
+    StringDataTypeName,
+    StringFillValue,
+)
+from zarr_metadata.v3.data_type.struct import (
+    STRUCT_DATA_TYPE_NAME,
+    StructDataTypeName,
+    StructFillValue,
+)
+from zarr_metadata.v3.data_type.uint8 import (
+    UINT8_DATA_TYPE_NAME,
+    Uint8DataTypeName,
+    Uint8FillValue,
+)
+from zarr_metadata.v3.data_type.uint16 import (
+    UINT16_DATA_TYPE_NAME,
+    Uint16DataTypeName,
+    Uint16FillValue,
+)
+from zarr_metadata.v3.data_type.uint32 import (
+    UINT32_DATA_TYPE_NAME,
+    Uint32DataTypeName,
+    Uint32FillValue,
+)
+from zarr_metadata.v3.data_type.uint64 import (
+    UINT64_DATA_TYPE_NAME,
+    Uint64DataTypeName,
+    Uint64FillValue,
+)
 from zarr_metadata.v3.group import GroupMetadataV3, GroupMetadataV3Partial
 
 __version__ = version("zarr-metadata")
 
 
 __all__ = [
+    "ARRAY_DIMENSION_SEPARATOR_V2",
+    "ARRAY_ORDER_V2",
+    "BLOSC_CNAME",
+    "BLOSC_CODEC_NAME",
+    "BLOSC_SHUFFLE",
+    "BOOL_DATA_TYPE_NAME",
+    "BYTES_CODEC_NAME",
+    "BYTES_DATA_TYPE_NAME",
+    "CAST_OUT_OF_RANGE_MODE",
+    "CAST_ROUNDING_MODE",
+    "CAST_VALUE_CODEC_NAME",
+    "COMPLEX64_DATA_TYPE_NAME",
+    "COMPLEX128_DATA_TYPE_NAME",
+    "CRC32C_CODEC_NAME",
+    "DEFAULT_CHUNK_KEY_ENCODING_NAME",
+    "DEFAULT_CHUNK_KEY_ENCODING_SEPARATOR",
+    "ENDIANNESS",
+    "FLOAT16_DATA_TYPE_NAME",
+    "FLOAT32_DATA_TYPE_NAME",
+    "FLOAT64_DATA_TYPE_NAME",
+    "GZIP_CODEC_NAME",
+    "INT8_DATA_TYPE_NAME",
+    "INT16_DATA_TYPE_NAME",
+    "INT32_DATA_TYPE_NAME",
+    "INT64_DATA_TYPE_NAME",
+    "NUMPY_DATETIME64_DATA_TYPE_NAME",
+    "NUMPY_TIMEDELTA64_DATA_TYPE_NAME",
+    "NUMPY_TIME_UNIT",
+    "RECTILINEAR_CHUNK_GRID_NAME",
+    "REGULAR_CHUNK_GRID_NAME",
+    "SCALE_OFFSET_CODEC_NAME",
+    "SHARDING_INDEXED_CODEC_NAME",
+    "SHARDING_INDEX_LOCATION",
+    "STRING_DATA_TYPE_NAME",
+    "STRUCT_DATA_TYPE_NAME",
+    "TRANSPOSE_CODEC_NAME",
+    "UINT8_DATA_TYPE_NAME",
+    "UINT16_DATA_TYPE_NAME",
+    "UINT32_DATA_TYPE_NAME",
+    "UINT64_DATA_TYPE_NAME",
+    "V2_CHUNK_KEY_ENCODING_NAME",
+    "V2_CHUNK_KEY_ENCODING_SEPARATOR",
+    "ZSTD_CODEC_NAME",
     "ArrayDimensionSeparatorV2",
     "ArrayMetadataV2",
     "ArrayMetadataV2Partial",
     "ArrayMetadataV3",
     "ArrayMetadataV3Partial",
     "ArrayOrderV2",
+    "BloscCName",
+    "BloscCodecMetadata",
+    "BloscCodecName",
+    "BloscShuffle",
+    "BoolDataTypeName",
+    "BoolFillValue",
+    "BytesCodecMetadata",
+    "BytesCodecName",
+    "BytesDataTypeName",
+    "BytesFillValue",
+    "CastOutOfRangeMode",
+    "CastRoundingMode",
+    "CastValueCodecMetadata",
+    "CastValueCodecName",
     "CodecMetadataV2",
+    "Complex64DataTypeName",
+    "Complex64FillValue",
+    "Complex128DataTypeName",
+    "Complex128FillValue",
     "ConsolidatedMetadataV2",
     "ConsolidatedMetadataV3",
+    "Crc32cCodecMetadata",
+    "Crc32cCodecName",
     "DataTypeMetadataV2",
+    "DefaultChunkKeyEncodingMetadata",
+    "DefaultChunkKeyEncodingName",
+    "DefaultChunkKeyEncodingSeparator",
+    "Endianness",
     "ExtensionFieldV3",
+    "Float16DataTypeName",
+    "Float16FillValue",
+    "Float32DataTypeName",
+    "Float32FillValue",
+    "Float64DataTypeName",
+    "Float64FillValue",
     "GroupMetadataV2",
     "GroupMetadataV2Partial",
     "GroupMetadataV3",
     "GroupMetadataV3Partial",
+    "GzipCodecMetadata",
+    "GzipCodecName",
+    "Int8DataTypeName",
+    "Int8FillValue",
+    "Int16DataTypeName",
+    "Int16FillValue",
+    "Int32DataTypeName",
+    "Int32FillValue",
+    "Int64DataTypeName",
+    "Int64FillValue",
     "JSONValue",
-    "MetadataFieldV3",
-    "NamedConfig",
+    "MetadataV3",
+    "NamedConfigV3",
+    "NumpyDatetime64DataTypeName",
+    "NumpyDatetime64FillValue",
+    "NumpyTimeUnit",
+    "NumpyTimedelta64DataTypeName",
+    "NumpyTimedelta64FillValue",
+    "RawBytesDataTypeName",
+    "RawBytesFillValue",
+    "RectilinearChunkGridMetadata",
+    "RectilinearChunkGridName",
+    "RegularChunkGridMetadata",
+    "RegularChunkGridName",
+    "ScaleOffsetCodecMetadata",
+    "ScaleOffsetCodecName",
+    "ShardingIndexLocation",
+    "ShardingIndexedCodecMetadata",
+    "ShardingIndexedCodecName",
+    "StringDataTypeName",
+    "StringFillValue",
+    "StructDataTypeName",
+    "StructFillValue",
+    "TransposeCodecMetadata",
+    "TransposeCodecName",
+    "Uint8DataTypeName",
+    "Uint8FillValue",
+    "Uint16DataTypeName",
+    "Uint16FillValue",
+    "Uint32DataTypeName",
+    "Uint32FillValue",
+    "Uint64DataTypeName",
+    "Uint64FillValue",
+    "V2ChunkKeyEncodingMetadata",
+    "V2ChunkKeyEncodingName",
+    "V2ChunkKeyEncodingSeparator",
     "ZArrayMetadata",
     "ZAttrsMetadata",
     "ZGroupMetadata",
+    "ZstdCodecMetadata",
+    "ZstdCodecName",
     "__version__",
 ]