diff --git a/changes/3417.bugfix.md b/changes/3417.bugfix.md new file mode 100644 index 0000000000..be5b44f3e9 --- /dev/null +++ b/changes/3417.bugfix.md @@ -0,0 +1,4 @@ +Fixed `BytesCodec.from_dict` so that `BytesCodec` instances roundtrip to / from +their dict representation. `BytesCodec.from_dict` now interprets a missing +`endian` configuration as `endian=None` (matching what `BytesCodec.to_dict` +emits), instead of falling back to the system's native byte order. diff --git a/changes/3987.feature.md b/changes/3987.feature.md new file mode 100644 index 0000000000..2492b4d7dd --- /dev/null +++ b/changes/3987.feature.md @@ -0,0 +1 @@ +Two new fields on `ArrayConfig` control how the sharding codec coalesces partial-shard reads: `sharding_coalesce_max_gap_bytes` (default 1 MiB) and `sharding_coalesce_max_bytes` (default 16 MiB). When reading multiple chunks from the same shard, nearby byte ranges are merged into a single request to the store if separated by no more than `sharding_coalesce_max_gap_bytes` and the merged read stays within `sharding_coalesce_max_bytes`. Defaults are seeded from the matching `array.sharding_coalesce_max_gap_bytes` / `array.sharding_coalesce_max_bytes` keys in [`zarr.config`][] at array-creation time, and can be overridden per array by passing `config={...}` to [`zarr.create_array`][]. diff --git a/changes/4074.bugfix.md b/changes/4074.bugfix.md new file mode 100644 index 0000000000..d55a52b887 --- /dev/null +++ b/changes/4074.bugfix.md @@ -0,0 +1,7 @@ +Fixed several storage and codec bugs: + +- Reading a value with a `SuffixByteRequest` larger than the value now correctly returns the whole value (matching HTTP `bytes=-N` suffix-range semantics), instead of silently returning incorrect data for `MemoryStore`. +- `LoggingStore.get_partial_values` and `FsspecStore.get_partial_values` no longer return empty results when `key_ranges` is passed as a one-shot iterable (e.g. a generator). +- `Store.getsize_prefix` no longer over-counts sibling keys that merely share a string prefix (e.g. `getsize_prefix("foo")` no longer includes keys under `foobar/`). +- `ZipStore.close()` no longer raises `AttributeError` when the store was created but never opened (including when used as a context manager without any I/O). +- `codecs_from_list` now raises a descriptive `TypeError` when a `BytesBytesCodec` immediately follows an `ArrayArrayCodec`, instead of a misleading "Required ArrayBytesCodec was not found" `ValueError`. diff --git a/docs/user-guide/config.md b/docs/user-guide/config.md index 8a8fa94c3d..71c021b070 100644 --- a/docs/user-guide/config.md +++ b/docs/user-guide/config.md @@ -35,6 +35,7 @@ Configuration options include the following: - Async and threading options, e.g. `async.concurrency` and `threading.max_workers` - Selections of implementations of codecs, codec pipelines and buffers - Enabling GPU support with `zarr.config.enable_gpu()`. See GPU support for more. +- Control request merging when reading multiple chunks from the same shard with `array.sharding_coalesce_max_gap_bytes` and `array.sharding_coalesce_max_bytes`. Reads of nearby chunks are coalesced into a single request to the store when separated by at most `sharding_coalesce_max_gap_bytes` and the resulting merged read is no larger than `sharding_coalesce_max_bytes`. For selecting custom implementations of codecs, pipelines, buffers and ndbuffers, first register the implementations in the registry and then select them in the config. diff --git a/packages/zarr-metadata/CHANGELOG.md b/packages/zarr-metadata/CHANGELOG.md index 03e744470b..a3ff1177a0 100644 --- a/packages/zarr-metadata/CHANGELOG.md +++ b/packages/zarr-metadata/CHANGELOG.md @@ -2,6 +2,34 @@ +## 0.3.0 (2026-06-19) + +### Deprecations and Removals + +- Introduces a new `JSONValue` type that models python objects that serialize directly to JSON. This type is used to annotate the contents of `attributes` and `fill_value` fields, replacing the use of the overly wide `object` type. This is technically a breaking change. ([#4037](https://github.com/zarr-developers/zarr-python/issues/4037)) +- Promoted a curated "front door" of names to the top-level `zarr_metadata` + namespace, so consumers can write e.g. `from zarr_metadata import + ArrayMetadataV3, ShardingIndexLocation, BLOSC_CNAME` instead of importing from + deep submodule paths. The front door covers every metadata-document TypedDict, + each codec/chunk-grid/chunk-key-encoding canonical type, the full data-type + trio for every dtype, and every constant + `Literal` pair. Deep submodule paths + continue to work unchanged. + + Several promoted names were given clearer, less ambiguous spellings than their + deep-module names, since they now appear bare at the top level: + `Endian`/`ENDIAN` → `Endianness`/`ENDIANNESS`, + `IndexLocation`/`INDEX_LOCATION` → `ShardingIndexLocation`/`SHARDING_INDEX_LOCATION`, + `RoundingMode`/`ROUNDING_MODE` → `CastRoundingMode`/`CAST_ROUNDING_MODE`, + `OutOfRangeMode`/`OUT_OF_RANGE_MODE` → `CastOutOfRangeMode`/`CAST_OUT_OF_RANGE_MODE`, + `DateTimeUnit` → `NumpyTimeUnit`, + `NamedConfig` → `NamedConfigV3`, and + `MetadataFieldV3` → `MetadataV3` (matching the name `zarrs` uses for this + `name`-or-`{name, configuration}` shape). + + Also added the `NUMPY_TIME_UNIT` runtime constant (a `Final` tuple paired with + the `NumpyTimeUnit` Literal) in `zarr_metadata.v3.data_type.numpy_timedelta64`. ([#4083](https://github.com/zarr-developers/zarr-python/issues/4083)) + + ## 0.2.0 (2026-05-19) ### Bugfixes diff --git a/packages/zarr-metadata/changes/4037.misc.md b/packages/zarr-metadata/changes/4037.misc.md deleted file mode 100644 index fee5c69cca..0000000000 --- a/packages/zarr-metadata/changes/4037.misc.md +++ /dev/null @@ -1 +0,0 @@ -Introduces a new `JSONValue` type that models python objects that serialize directly to JSON. This type is used to annotate the contents of `attributes` and `fill_value` fields, replacing the use of the overly wide `object` type. This is technically a breaking change. \ No newline at end of file diff --git a/packages/zarr-metadata/src/zarr_metadata/__init__.py b/packages/zarr-metadata/src/zarr_metadata/__init__.py index 7c6461500e..46949570a2 100644 --- a/packages/zarr-metadata/src/zarr_metadata/__init__.py +++ b/packages/zarr-metadata/src/zarr_metadata/__init__.py @@ -1,7 +1,9 @@ from importlib.metadata import version -from zarr_metadata._common import JSONValue, NamedConfig +from zarr_metadata._common import JSONValue, NamedConfigV3 from zarr_metadata.v2.array import ( + ARRAY_DIMENSION_SEPARATOR_V2, + ARRAY_ORDER_V2, ArrayDimensionSeparatorV2, ArrayMetadataV2, ArrayMetadataV2Partial, @@ -13,35 +15,320 @@ from zarr_metadata.v2.codec import CodecMetadataV2 from zarr_metadata.v2.consolidated import ConsolidatedMetadataV2 from zarr_metadata.v2.group import GroupMetadataV2, GroupMetadataV2Partial, ZGroupMetadata -from zarr_metadata.v3._common import MetadataFieldV3 +from zarr_metadata.v3._common import MetadataV3 from zarr_metadata.v3.array import ArrayMetadataV3, ArrayMetadataV3Partial, ExtensionFieldV3 +from zarr_metadata.v3.chunk_grid.rectilinear import ( + RECTILINEAR_CHUNK_GRID_NAME, + RectilinearChunkGridMetadata, + RectilinearChunkGridName, +) +from zarr_metadata.v3.chunk_grid.regular import ( + REGULAR_CHUNK_GRID_NAME, + RegularChunkGridMetadata, + RegularChunkGridName, +) +from zarr_metadata.v3.chunk_key_encoding.default import ( + DEFAULT_CHUNK_KEY_ENCODING_NAME, + DEFAULT_CHUNK_KEY_ENCODING_SEPARATOR, + DefaultChunkKeyEncodingMetadata, + DefaultChunkKeyEncodingName, + DefaultChunkKeyEncodingSeparator, +) +from zarr_metadata.v3.chunk_key_encoding.v2 import ( + V2_CHUNK_KEY_ENCODING_NAME, + V2_CHUNK_KEY_ENCODING_SEPARATOR, + V2ChunkKeyEncodingMetadata, + V2ChunkKeyEncodingName, + V2ChunkKeyEncodingSeparator, +) +from zarr_metadata.v3.codec.blosc import ( + BLOSC_CNAME, + BLOSC_CODEC_NAME, + BLOSC_SHUFFLE, + BloscCName, + BloscCodecMetadata, + BloscCodecName, + BloscShuffle, +) +from zarr_metadata.v3.codec.bytes import ( + BYTES_CODEC_NAME, + ENDIANNESS, + BytesCodecMetadata, + BytesCodecName, + Endianness, +) +from zarr_metadata.v3.codec.cast_value import ( + CAST_OUT_OF_RANGE_MODE, + CAST_ROUNDING_MODE, + CAST_VALUE_CODEC_NAME, + CastOutOfRangeMode, + CastRoundingMode, + CastValueCodecMetadata, + CastValueCodecName, +) +from zarr_metadata.v3.codec.crc32c import CRC32C_CODEC_NAME, Crc32cCodecMetadata, Crc32cCodecName +from zarr_metadata.v3.codec.gzip import GZIP_CODEC_NAME, GzipCodecMetadata, GzipCodecName +from zarr_metadata.v3.codec.scale_offset import ( + SCALE_OFFSET_CODEC_NAME, + ScaleOffsetCodecMetadata, + ScaleOffsetCodecName, +) +from zarr_metadata.v3.codec.sharding_indexed import ( + SHARDING_INDEX_LOCATION, + SHARDING_INDEXED_CODEC_NAME, + ShardingIndexedCodecMetadata, + ShardingIndexedCodecName, + ShardingIndexLocation, +) +from zarr_metadata.v3.codec.transpose import ( + TRANSPOSE_CODEC_NAME, + TransposeCodecMetadata, + TransposeCodecName, +) +from zarr_metadata.v3.codec.zstd import ZSTD_CODEC_NAME, ZstdCodecMetadata, ZstdCodecName from zarr_metadata.v3.consolidated import ConsolidatedMetadataV3 +from zarr_metadata.v3.data_type.bool import ( + BOOL_DATA_TYPE_NAME, + BoolDataTypeName, + BoolFillValue, +) +from zarr_metadata.v3.data_type.bytes import ( + BYTES_DATA_TYPE_NAME, + BytesDataTypeName, + BytesFillValue, +) +from zarr_metadata.v3.data_type.complex64 import ( + COMPLEX64_DATA_TYPE_NAME, + Complex64DataTypeName, + Complex64FillValue, +) +from zarr_metadata.v3.data_type.complex128 import ( + COMPLEX128_DATA_TYPE_NAME, + Complex128DataTypeName, + Complex128FillValue, +) +from zarr_metadata.v3.data_type.float16 import ( + FLOAT16_DATA_TYPE_NAME, + Float16DataTypeName, + Float16FillValue, +) +from zarr_metadata.v3.data_type.float32 import ( + FLOAT32_DATA_TYPE_NAME, + Float32DataTypeName, + Float32FillValue, +) +from zarr_metadata.v3.data_type.float64 import ( + FLOAT64_DATA_TYPE_NAME, + Float64DataTypeName, + Float64FillValue, +) +from zarr_metadata.v3.data_type.int8 import ( + INT8_DATA_TYPE_NAME, + Int8DataTypeName, + Int8FillValue, +) +from zarr_metadata.v3.data_type.int16 import ( + INT16_DATA_TYPE_NAME, + Int16DataTypeName, + Int16FillValue, +) +from zarr_metadata.v3.data_type.int32 import ( + INT32_DATA_TYPE_NAME, + Int32DataTypeName, + Int32FillValue, +) +from zarr_metadata.v3.data_type.int64 import ( + INT64_DATA_TYPE_NAME, + Int64DataTypeName, + Int64FillValue, +) +from zarr_metadata.v3.data_type.numpy_datetime64 import ( + NUMPY_DATETIME64_DATA_TYPE_NAME, + NumpyDatetime64DataTypeName, + NumpyDatetime64FillValue, +) +from zarr_metadata.v3.data_type.numpy_timedelta64 import ( + NUMPY_TIME_UNIT, + NUMPY_TIMEDELTA64_DATA_TYPE_NAME, + NumpyTimedelta64DataTypeName, + NumpyTimedelta64FillValue, + NumpyTimeUnit, +) +from zarr_metadata.v3.data_type.raw import RawBytesDataTypeName, RawBytesFillValue +from zarr_metadata.v3.data_type.string import ( + STRING_DATA_TYPE_NAME, + StringDataTypeName, + StringFillValue, +) +from zarr_metadata.v3.data_type.struct import ( + STRUCT_DATA_TYPE_NAME, + StructDataTypeName, + StructFillValue, +) +from zarr_metadata.v3.data_type.uint8 import ( + UINT8_DATA_TYPE_NAME, + Uint8DataTypeName, + Uint8FillValue, +) +from zarr_metadata.v3.data_type.uint16 import ( + UINT16_DATA_TYPE_NAME, + Uint16DataTypeName, + Uint16FillValue, +) +from zarr_metadata.v3.data_type.uint32 import ( + UINT32_DATA_TYPE_NAME, + Uint32DataTypeName, + Uint32FillValue, +) +from zarr_metadata.v3.data_type.uint64 import ( + UINT64_DATA_TYPE_NAME, + Uint64DataTypeName, + Uint64FillValue, +) from zarr_metadata.v3.group import GroupMetadataV3, GroupMetadataV3Partial __version__ = version("zarr-metadata") __all__ = [ + "ARRAY_DIMENSION_SEPARATOR_V2", + "ARRAY_ORDER_V2", + "BLOSC_CNAME", + "BLOSC_CODEC_NAME", + "BLOSC_SHUFFLE", + "BOOL_DATA_TYPE_NAME", + "BYTES_CODEC_NAME", + "BYTES_DATA_TYPE_NAME", + "CAST_OUT_OF_RANGE_MODE", + "CAST_ROUNDING_MODE", + "CAST_VALUE_CODEC_NAME", + "COMPLEX64_DATA_TYPE_NAME", + "COMPLEX128_DATA_TYPE_NAME", + "CRC32C_CODEC_NAME", + "DEFAULT_CHUNK_KEY_ENCODING_NAME", + "DEFAULT_CHUNK_KEY_ENCODING_SEPARATOR", + "ENDIANNESS", + "FLOAT16_DATA_TYPE_NAME", + "FLOAT32_DATA_TYPE_NAME", + "FLOAT64_DATA_TYPE_NAME", + "GZIP_CODEC_NAME", + "INT8_DATA_TYPE_NAME", + "INT16_DATA_TYPE_NAME", + "INT32_DATA_TYPE_NAME", + "INT64_DATA_TYPE_NAME", + "NUMPY_DATETIME64_DATA_TYPE_NAME", + "NUMPY_TIMEDELTA64_DATA_TYPE_NAME", + "NUMPY_TIME_UNIT", + "RECTILINEAR_CHUNK_GRID_NAME", + "REGULAR_CHUNK_GRID_NAME", + "SCALE_OFFSET_CODEC_NAME", + "SHARDING_INDEXED_CODEC_NAME", + "SHARDING_INDEX_LOCATION", + "STRING_DATA_TYPE_NAME", + "STRUCT_DATA_TYPE_NAME", + "TRANSPOSE_CODEC_NAME", + "UINT8_DATA_TYPE_NAME", + "UINT16_DATA_TYPE_NAME", + "UINT32_DATA_TYPE_NAME", + "UINT64_DATA_TYPE_NAME", + "V2_CHUNK_KEY_ENCODING_NAME", + "V2_CHUNK_KEY_ENCODING_SEPARATOR", + "ZSTD_CODEC_NAME", "ArrayDimensionSeparatorV2", "ArrayMetadataV2", "ArrayMetadataV2Partial", "ArrayMetadataV3", "ArrayMetadataV3Partial", "ArrayOrderV2", + "BloscCName", + "BloscCodecMetadata", + "BloscCodecName", + "BloscShuffle", + "BoolDataTypeName", + "BoolFillValue", + "BytesCodecMetadata", + "BytesCodecName", + "BytesDataTypeName", + "BytesFillValue", + "CastOutOfRangeMode", + "CastRoundingMode", + "CastValueCodecMetadata", + "CastValueCodecName", "CodecMetadataV2", + "Complex64DataTypeName", + "Complex64FillValue", + "Complex128DataTypeName", + "Complex128FillValue", "ConsolidatedMetadataV2", "ConsolidatedMetadataV3", + "Crc32cCodecMetadata", + "Crc32cCodecName", "DataTypeMetadataV2", + "DefaultChunkKeyEncodingMetadata", + "DefaultChunkKeyEncodingName", + "DefaultChunkKeyEncodingSeparator", + "Endianness", "ExtensionFieldV3", + "Float16DataTypeName", + "Float16FillValue", + "Float32DataTypeName", + "Float32FillValue", + "Float64DataTypeName", + "Float64FillValue", "GroupMetadataV2", "GroupMetadataV2Partial", "GroupMetadataV3", "GroupMetadataV3Partial", + "GzipCodecMetadata", + "GzipCodecName", + "Int8DataTypeName", + "Int8FillValue", + "Int16DataTypeName", + "Int16FillValue", + "Int32DataTypeName", + "Int32FillValue", + "Int64DataTypeName", + "Int64FillValue", "JSONValue", - "MetadataFieldV3", - "NamedConfig", + "MetadataV3", + "NamedConfigV3", + "NumpyDatetime64DataTypeName", + "NumpyDatetime64FillValue", + "NumpyTimeUnit", + "NumpyTimedelta64DataTypeName", + "NumpyTimedelta64FillValue", + "RawBytesDataTypeName", + "RawBytesFillValue", + "RectilinearChunkGridMetadata", + "RectilinearChunkGridName", + "RegularChunkGridMetadata", + "RegularChunkGridName", + "ScaleOffsetCodecMetadata", + "ScaleOffsetCodecName", + "ShardingIndexLocation", + "ShardingIndexedCodecMetadata", + "ShardingIndexedCodecName", + "StringDataTypeName", + "StringFillValue", + "StructDataTypeName", + "StructFillValue", + "TransposeCodecMetadata", + "TransposeCodecName", + "Uint8DataTypeName", + "Uint8FillValue", + "Uint16DataTypeName", + "Uint16FillValue", + "Uint32DataTypeName", + "Uint32FillValue", + "Uint64DataTypeName", + "Uint64FillValue", + "V2ChunkKeyEncodingMetadata", + "V2ChunkKeyEncodingName", + "V2ChunkKeyEncodingSeparator", "ZArrayMetadata", "ZAttrsMetadata", "ZGroupMetadata", + "ZstdCodecMetadata", + "ZstdCodecName", "__version__", ] diff --git a/packages/zarr-metadata/src/zarr_metadata/_common.py b/packages/zarr-metadata/src/zarr_metadata/_common.py index f6064d863f..598a12e80c 100644 --- a/packages/zarr-metadata/src/zarr_metadata/_common.py +++ b/packages/zarr-metadata/src/zarr_metadata/_common.py @@ -24,7 +24,7 @@ """ -class NamedConfig(TypedDict): +class NamedConfigV3(TypedDict): """ Externally-tagged union member for a metadata field. diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/__init__.py b/packages/zarr-metadata/src/zarr_metadata/v3/__init__.py index 7699aa744d..c897f20d52 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/__init__.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/__init__.py @@ -1,6 +1,6 @@ """Zarr v3 metadata types.""" -from zarr_metadata.v3._common import MetadataFieldV3 +from zarr_metadata.v3._common import MetadataV3 from zarr_metadata.v3.array import ArrayMetadataV3, ExtensionFieldV3 from zarr_metadata.v3.consolidated import ConsolidatedMetadataV3 from zarr_metadata.v3.group import GroupMetadataV3 @@ -10,5 +10,5 @@ "ConsolidatedMetadataV3", "ExtensionFieldV3", "GroupMetadataV3", - "MetadataFieldV3", + "MetadataV3", ] diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/_common.py b/packages/zarr-metadata/src/zarr_metadata/v3/_common.py index 8d8e21616a..3424587a43 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/_common.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/_common.py @@ -2,12 +2,12 @@ This module is private (underscore-prefixed) and exists to avoid circular imports between leaf modules and sub-package `__init__.py` re-exports. -Public consumers should import `MetadataFieldV3` from `zarr_metadata.v3`. +Public consumers should import `MetadataV3` from `zarr_metadata.v3`. """ -from zarr_metadata._common import NamedConfig +from zarr_metadata._common import NamedConfigV3 -MetadataFieldV3 = str | NamedConfig +MetadataV3 = str | NamedConfigV3 """The JSON shape of any v3 metadata extension-point entry: either a bare short-hand name string or a `{name, configuration}` envelope. @@ -19,5 +19,5 @@ __all__ = [ - "MetadataFieldV3", + "MetadataV3", ] diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/array.py b/packages/zarr-metadata/src/zarr_metadata/v3/array.py index d9cea4aef4..a8b0fa3358 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/array.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/array.py @@ -6,7 +6,7 @@ from typing_extensions import TypedDict from zarr_metadata._common import JSONValue -from zarr_metadata.v3._common import MetadataFieldV3 +from zarr_metadata.v3._common import MetadataV3 class ExtensionFieldV3(TypedDict, extra_items=JSONValue): # type: ignore[call-arg] @@ -52,14 +52,14 @@ class ArrayMetadataV3(TypedDict, extra_items=ExtensionFieldV3): # type: ignore[ zarr_format: Literal[3] node_type: Literal["array"] - data_type: MetadataFieldV3 + data_type: MetadataV3 shape: tuple[int, ...] - chunk_grid: MetadataFieldV3 - chunk_key_encoding: MetadataFieldV3 + chunk_grid: MetadataV3 + chunk_key_encoding: MetadataV3 fill_value: JSONValue - codecs: tuple[MetadataFieldV3, ...] + codecs: tuple[MetadataV3, ...] attributes: NotRequired[Mapping[str, JSONValue]] - storage_transformers: NotRequired[tuple[MetadataFieldV3, ...]] + storage_transformers: NotRequired[tuple[MetadataV3, ...]] dimension_names: NotRequired[tuple[str | None, ...]] @@ -88,14 +88,14 @@ class ArrayMetadataV3Partial(TypedDict, total=False, extra_items=ExtensionFieldV zarr_format: Literal[3] node_type: Literal["array"] - data_type: MetadataFieldV3 + data_type: MetadataV3 shape: tuple[int, ...] - chunk_grid: MetadataFieldV3 - chunk_key_encoding: MetadataFieldV3 + chunk_grid: MetadataV3 + chunk_key_encoding: MetadataV3 fill_value: JSONValue - codecs: tuple[MetadataFieldV3, ...] + codecs: tuple[MetadataV3, ...] attributes: NotRequired[Mapping[str, JSONValue]] - storage_transformers: NotRequired[tuple[MetadataFieldV3, ...]] + storage_transformers: NotRequired[tuple[MetadataV3, ...]] dimension_names: NotRequired[tuple[str | None, ...]] diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/codec/__init__.py b/packages/zarr-metadata/src/zarr_metadata/v3/codec/__init__.py index 8cc819496d..b4f357117f 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/codec/__init__.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/codec/__init__.py @@ -11,7 +11,7 @@ `CodecConfiguration`, etc., import directly from the leaf submodule. For the field-level "any codec entry" alias (used in array metadata's -`codecs` list and in sharding's inner pipelines), import `MetadataFieldV3` +`codecs` list and in sharding's inner pipelines), import `MetadataV3` from `zarr_metadata.v3`. See https://zarr-specs.readthedocs.io/en/latest/v3/codecs/index.html diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/codec/bytes.py b/packages/zarr-metadata/src/zarr_metadata/v3/codec/bytes.py index 522cbe10f5..04e746f898 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/codec/bytes.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/codec/bytes.py @@ -14,10 +14,10 @@ BytesCodecName = Literal["bytes"] """Literal type of the `name` field of the `bytes` codec.""" -Endian = Literal["little", "big"] +Endianness = Literal["little", "big"] """Literal type of byte order of multi-byte numeric data.""" -ENDIAN: Final = ("little", "big") +ENDIANNESS: Final = ("little", "big") """Tuple of permitted values for the `endian` field of the `bytes` codec.""" @@ -28,7 +28,7 @@ class BytesCodecConfiguration(TypedDict): The `endian` field is required for multi-byte data types. """ - endian: NotRequired[Endian] + endian: NotRequired[Endianness] class BytesCodecObject(TypedDict): @@ -55,10 +55,10 @@ class BytesCodecObject(TypedDict): __all__ = [ "BYTES_CODEC_NAME", - "ENDIAN", + "ENDIANNESS", "BytesCodecConfiguration", "BytesCodecMetadata", "BytesCodecName", "BytesCodecObject", - "Endian", + "Endianness", ] diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/codec/cast_value.py b/packages/zarr-metadata/src/zarr_metadata/v3/codec/cast_value.py index fd6fb2ee4a..7e9b071669 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/codec/cast_value.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/codec/cast_value.py @@ -9,7 +9,7 @@ from typing_extensions import TypedDict from zarr_metadata._common import JSONValue -from zarr_metadata.v3._common import MetadataFieldV3 +from zarr_metadata.v3._common import MetadataV3 CAST_VALUE_CODEC_NAME: Final = "cast_value" """The `name` field value of the `cast_value` codec.""" @@ -17,7 +17,7 @@ CastValueCodecName = Literal["cast_value"] """Literal type of the `name` field of the `cast_value` codec.""" -RoundingMode = Literal[ +CastRoundingMode = Literal[ "nearest-even", "towards-zero", "towards-positive", @@ -29,7 +29,7 @@ Defaults to `"nearest-even"` if absent. """ -ROUNDING_MODE: Final = ( +CAST_ROUNDING_MODE: Final = ( "nearest-even", "towards-zero", "towards-positive", @@ -38,13 +38,13 @@ ) """Tuple of permitted values for the `rounding` field of the `cast_value` codec.""" -OutOfRangeMode = Literal["clamp", "wrap"] +CastOutOfRangeMode = Literal["clamp", "wrap"] """Literal type of permitted values for the `out_of_range` configuration field. If absent, out-of-range values are an encoding/decoding error. """ -OUT_OF_RANGE_MODE: Final = ("clamp", "wrap") +CAST_OUT_OF_RANGE_MODE: Final = ("clamp", "wrap") """Tuple of permitted values for the `out_of_range` field of the `cast_value` codec.""" ScalarMapEntry = tuple[JSONValue, JSONValue] @@ -71,9 +71,9 @@ class CastValueCodecConfiguration(TypedDict): bare-string primitive name or a `{name, configuration}` envelope. """ - data_type: MetadataFieldV3 - rounding: NotRequired[RoundingMode] - out_of_range: NotRequired[OutOfRangeMode] + data_type: MetadataV3 + rounding: NotRequired[CastRoundingMode] + out_of_range: NotRequired[CastOutOfRangeMode] scalar_map: NotRequired[ScalarMap] @@ -93,15 +93,15 @@ class CastValueCodecObject(TypedDict): __all__ = [ + "CAST_OUT_OF_RANGE_MODE", + "CAST_ROUNDING_MODE", "CAST_VALUE_CODEC_NAME", - "OUT_OF_RANGE_MODE", - "ROUNDING_MODE", + "CastOutOfRangeMode", + "CastRoundingMode", "CastValueCodecConfiguration", "CastValueCodecMetadata", "CastValueCodecName", "CastValueCodecObject", - "OutOfRangeMode", - "RoundingMode", "ScalarMap", "ScalarMapEntry", ] diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/codec/sharding_indexed.py b/packages/zarr-metadata/src/zarr_metadata/v3/codec/sharding_indexed.py index 93a0774e4e..a1488f7c30 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/codec/sharding_indexed.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/codec/sharding_indexed.py @@ -8,7 +8,7 @@ from typing_extensions import TypedDict -from zarr_metadata.v3._common import MetadataFieldV3 +from zarr_metadata.v3._common import MetadataV3 SHARDING_INDEXED_CODEC_NAME: Final = "sharding_indexed" """The `name` field value of the `sharding_indexed` codec.""" @@ -16,10 +16,10 @@ ShardingIndexedCodecName = Literal["sharding_indexed"] """Literal type of the `name` field of the `sharding_indexed` codec.""" -IndexLocation = Literal["start", "end"] +ShardingIndexLocation = Literal["start", "end"] """Literal type of the position of the shard index within the encoded shard.""" -INDEX_LOCATION: Final = ("start", "end") +SHARDING_INDEX_LOCATION: Final = ("start", "end") """Tuple of permitted values for the `index_location` field of the `sharding_indexed` codec.""" @@ -40,9 +40,9 @@ class ShardingIndexedCodecConfiguration(TypedDict): """ chunk_shape: tuple[int, ...] - codecs: tuple[MetadataFieldV3, ...] - index_codecs: tuple[MetadataFieldV3, ...] - index_location: NotRequired[IndexLocation] + codecs: tuple[MetadataV3, ...] + index_codecs: tuple[MetadataV3, ...] + index_location: NotRequired[ShardingIndexLocation] class ShardingIndexedCodecObject(TypedDict): @@ -61,9 +61,9 @@ class ShardingIndexedCodecObject(TypedDict): """ __all__ = [ - "INDEX_LOCATION", "SHARDING_INDEXED_CODEC_NAME", - "IndexLocation", + "SHARDING_INDEX_LOCATION", + "ShardingIndexLocation", "ShardingIndexedCodecConfiguration", "ShardingIndexedCodecMetadata", "ShardingIndexedCodecName", diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/data_type/numpy_datetime64.py b/packages/zarr-metadata/src/zarr_metadata/v3/data_type/numpy_datetime64.py index 243d5fb6f6..8784160f71 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/data_type/numpy_datetime64.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/data_type/numpy_datetime64.py @@ -14,7 +14,7 @@ NumpyDatetime64DataTypeName = Literal["numpy.datetime64"] """Literal type of the `name` field of the `numpy.datetime64` data type.""" -DateTimeUnit = Literal[ +NumpyTimeUnit = Literal[ "Y", "M", "W", "D", "h", "m", "s", "ms", "us", "μs", "ns", "ps", "fs", "as", "generic" ] """Time unit codes used by numpy.datetime64.""" @@ -32,7 +32,7 @@ class NumpyDatetime64Configuration(TypedDict): The multiplier relative to the unit. """ - unit: ReadOnly[DateTimeUnit] + unit: ReadOnly[NumpyTimeUnit] scale_factor: ReadOnly[int] @@ -52,9 +52,9 @@ class NumpyDatetime64(TypedDict): __all__ = [ "NUMPY_DATETIME64_DATA_TYPE_NAME", - "DateTimeUnit", "NumpyDatetime64", "NumpyDatetime64Configuration", "NumpyDatetime64DataTypeName", "NumpyDatetime64FillValue", + "NumpyTimeUnit", ] diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/data_type/numpy_timedelta64.py b/packages/zarr-metadata/src/zarr_metadata/v3/data_type/numpy_timedelta64.py index 41e35e7aae..f5c8c77bf8 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/data_type/numpy_timedelta64.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/data_type/numpy_timedelta64.py @@ -14,11 +14,30 @@ NumpyTimedelta64DataTypeName = Literal["numpy.timedelta64"] """Literal type of the `name` field of the `numpy.timedelta64` data type.""" -DateTimeUnit = Literal[ +NumpyTimeUnit = Literal[ "Y", "M", "W", "D", "h", "m", "s", "ms", "us", "μs", "ns", "ps", "fs", "as", "generic" ] """Time unit codes used by numpy.timedelta64.""" +NUMPY_TIME_UNIT: Final = ( + "Y", + "M", + "W", + "D", + "h", + "m", + "s", + "ms", + "us", + "μs", + "ns", + "ps", + "fs", + "as", + "generic", +) +"""Runtime tuple of the permitted `numpy.timedelta64`/`numpy.datetime64` unit strings.""" + class NumpyTimedelta64Configuration(TypedDict): """ @@ -32,7 +51,7 @@ class NumpyTimedelta64Configuration(TypedDict): The multiplier relative to the unit. """ - unit: ReadOnly[DateTimeUnit] + unit: ReadOnly[NumpyTimeUnit] scale_factor: ReadOnly[int] @@ -52,7 +71,8 @@ class NumpyTimedelta64(TypedDict): __all__ = [ "NUMPY_TIMEDELTA64_DATA_TYPE_NAME", - "DateTimeUnit", + "NUMPY_TIME_UNIT", + "NumpyTimeUnit", "NumpyTimedelta64", "NumpyTimedelta64Configuration", "NumpyTimedelta64DataTypeName", diff --git a/packages/zarr-metadata/src/zarr_metadata/v3/data_type/struct.py b/packages/zarr-metadata/src/zarr_metadata/v3/data_type/struct.py index 282bcc83d6..5291e5c309 100644 --- a/packages/zarr-metadata/src/zarr_metadata/v3/data_type/struct.py +++ b/packages/zarr-metadata/src/zarr_metadata/v3/data_type/struct.py @@ -10,7 +10,7 @@ from typing_extensions import ReadOnly, TypedDict from zarr_metadata._common import JSONValue -from zarr_metadata.v3._common import MetadataFieldV3 +from zarr_metadata.v3._common import MetadataV3 STRUCT_DATA_TYPE_NAME: Final = "struct" """The `name` field value of the `struct` data type.""" @@ -33,7 +33,7 @@ class StructField(TypedDict): """ name: ReadOnly[str] - data_type: ReadOnly[MetadataFieldV3] + data_type: ReadOnly[MetadataV3] class StructConfiguration(TypedDict): diff --git a/packages/zarr-metadata/tests/test_public_api.py b/packages/zarr-metadata/tests/test_public_api.py new file mode 100644 index 0000000000..d3270579c3 --- /dev/null +++ b/packages/zarr-metadata/tests/test_public_api.py @@ -0,0 +1,215 @@ +"""Test that the curated front-door names are accessible from the top-level zarr_metadata package.""" + +import re +from typing import get_args + +import zarr_metadata as zm + + +def _group_rank(s: str) -> int: + """RUF022 groups `__all__` as: SCREAMING_SNAKE (0), then TitleCase (1), then dunders (2). + + The exact intra-group ordering is ruff's own natural sort and is enforced by + ruff itself (pre-commit + CI); this test only asserts the grouping, not the + fragile tie-breaking, so it can't drift out of sync with ruff's implementation. + """ + if s.startswith("__") and s.endswith("__"): + return 2 + stripped = re.sub(r"[\d_]", "", s) + return 0 if stripped.isupper() else 1 + + +EXPECTED = [ + # Category A — metadata-document types + "ArrayMetadataV2", + "ArrayMetadataV2Partial", + "ZArrayMetadata", + "GroupMetadataV2", + "GroupMetadataV2Partial", + "ZGroupMetadata", + "ConsolidatedMetadataV2", + "ZAttrsMetadata", + "CodecMetadataV2", + "ArrayMetadataV3", + "ArrayMetadataV3Partial", + "ExtensionFieldV3", + "GroupMetadataV3", + "GroupMetadataV3Partial", + "ConsolidatedMetadataV3", + "NamedConfigV3", + "MetadataV3", + "JSONValue", + # v2 data-type encoding union + "DataTypeMetadataV2", + # Category B — codec canonical unions + "BloscCodecMetadata", + "BytesCodecMetadata", + "CastValueCodecMetadata", + "Crc32cCodecMetadata", + "GzipCodecMetadata", + "ScaleOffsetCodecMetadata", + "ShardingIndexedCodecMetadata", + "TransposeCodecMetadata", + "ZstdCodecMetadata", + # Category C — grid/key canonical unions + "RegularChunkGridMetadata", + "RectilinearChunkGridMetadata", + "DefaultChunkKeyEncodingMetadata", + "V2ChunkKeyEncodingMetadata", + # Category D — dtype trios + # bool + "BoolDataTypeName", + "BOOL_DATA_TYPE_NAME", + "BoolFillValue", + # int8/16/32/64 + "Int8DataTypeName", + "INT8_DATA_TYPE_NAME", + "Int8FillValue", + "Int16DataTypeName", + "INT16_DATA_TYPE_NAME", + "Int16FillValue", + "Int32DataTypeName", + "INT32_DATA_TYPE_NAME", + "Int32FillValue", + "Int64DataTypeName", + "INT64_DATA_TYPE_NAME", + "Int64FillValue", + # uint8/16/32/64 (actual casing is Uint, not UInt) + "Uint8DataTypeName", + "UINT8_DATA_TYPE_NAME", + "Uint8FillValue", + "Uint16DataTypeName", + "UINT16_DATA_TYPE_NAME", + "Uint16FillValue", + "Uint32DataTypeName", + "UINT32_DATA_TYPE_NAME", + "Uint32FillValue", + "Uint64DataTypeName", + "UINT64_DATA_TYPE_NAME", + "Uint64FillValue", + # float16/32/64 + "Float16DataTypeName", + "FLOAT16_DATA_TYPE_NAME", + "Float16FillValue", + "Float32DataTypeName", + "FLOAT32_DATA_TYPE_NAME", + "Float32FillValue", + "Float64DataTypeName", + "FLOAT64_DATA_TYPE_NAME", + "Float64FillValue", + # complex64/128 + "Complex64DataTypeName", + "COMPLEX64_DATA_TYPE_NAME", + "Complex64FillValue", + "Complex128DataTypeName", + "COMPLEX128_DATA_TYPE_NAME", + "Complex128FillValue", + # bytes + "BytesDataTypeName", + "BYTES_DATA_TYPE_NAME", + "BytesFillValue", + # string + "StringDataTypeName", + "STRING_DATA_TYPE_NAME", + "StringFillValue", + # numpy_datetime64 + "NumpyDatetime64DataTypeName", + "NUMPY_DATETIME64_DATA_TYPE_NAME", + "NumpyDatetime64FillValue", + # numpy_timedelta64 + "NumpyTimedelta64DataTypeName", + "NUMPY_TIMEDELTA64_DATA_TYPE_NAME", + "NumpyTimedelta64FillValue", + # struct + "StructDataTypeName", + "STRUCT_DATA_TYPE_NAME", + "StructFillValue", + # raw (no _DATA_TYPE_NAME constant) + "RawBytesDataTypeName", + "RawBytesFillValue", + # Category E — constant+Literal pairs + "ARRAY_ORDER_V2", + "ArrayOrderV2", + "ARRAY_DIMENSION_SEPARATOR_V2", + "ArrayDimensionSeparatorV2", + "ENDIANNESS", + "Endianness", + "BYTES_CODEC_NAME", + "BytesCodecName", + "BLOSC_CODEC_NAME", + "BloscCodecName", + "BLOSC_CNAME", + "BloscCName", + "BLOSC_SHUFFLE", + "BloscShuffle", + "CAST_ROUNDING_MODE", + "CastRoundingMode", + "CAST_OUT_OF_RANGE_MODE", + "CastOutOfRangeMode", + "CAST_VALUE_CODEC_NAME", + "CastValueCodecName", + "CRC32C_CODEC_NAME", + "Crc32cCodecName", + "GZIP_CODEC_NAME", + "GzipCodecName", + "SCALE_OFFSET_CODEC_NAME", + "ScaleOffsetCodecName", + "SHARDING_INDEX_LOCATION", + "ShardingIndexLocation", + "SHARDING_INDEXED_CODEC_NAME", + "ShardingIndexedCodecName", + "TRANSPOSE_CODEC_NAME", + "TransposeCodecName", + "ZSTD_CODEC_NAME", + "ZstdCodecName", + "REGULAR_CHUNK_GRID_NAME", + "RegularChunkGridName", + "RECTILINEAR_CHUNK_GRID_NAME", + "RectilinearChunkGridName", + "DEFAULT_CHUNK_KEY_ENCODING_NAME", + "DefaultChunkKeyEncodingName", + "DEFAULT_CHUNK_KEY_ENCODING_SEPARATOR", + "DefaultChunkKeyEncodingSeparator", + "V2_CHUNK_KEY_ENCODING_NAME", + "V2ChunkKeyEncodingName", + "V2_CHUNK_KEY_ENCODING_SEPARATOR", + "V2ChunkKeyEncodingSeparator", + "NUMPY_TIME_UNIT", + "NumpyTimeUnit", +] + + +def test_front_door_names_public() -> None: + missing = [n for n in EXPECTED if n not in zm.__all__ or not hasattr(zm, n)] + assert not missing, f"missing from top-level API: {missing}" + + +def test_front_door_is_exactly_expected() -> None: + """`__all__` must contain exactly the curated names (plus `__version__`). + + Guards against a name being promoted to the front door without a + corresponding, deliberate entry in `EXPECTED` — i.e. an accidental + addition to the public API surface. + """ + assert set(zm.__all__) - {"__version__"} == set(EXPECTED) + + +def test_all_is_grouped_and_unique() -> None: + ranks = [_group_rank(n) for n in zm.__all__] + assert ranks == sorted(ranks), "`__all__` groups out of order (SCREAMING, TitleCase, dunder)" + assert len(zm.__all__) == len(set(zm.__all__)) + + +def test_promoted_pairs_drift() -> None: + pairs = [ + (zm.ENDIANNESS, zm.Endianness), + (zm.BLOSC_CNAME, zm.BloscCName), + (zm.BLOSC_SHUFFLE, zm.BloscShuffle), + (zm.SHARDING_INDEX_LOCATION, zm.ShardingIndexLocation), + (zm.NUMPY_TIME_UNIT, zm.NumpyTimeUnit), + (zm.CAST_ROUNDING_MODE, zm.CastRoundingMode), + (zm.CAST_OUT_OF_RANGE_MODE, zm.CastOutOfRangeMode), + (zm.ARRAY_ORDER_V2, zm.ArrayOrderV2), + ] + for const, lit in pairs: + assert set(const) == set(get_args(lit)) diff --git a/packages/zarr-metadata/tests/v3/data_type/numpy_timedelta64/test_fixtures.py b/packages/zarr-metadata/tests/v3/data_type/numpy_timedelta64/test_fixtures.py index 1d4bd86a2d..2a6c651582 100644 --- a/packages/zarr-metadata/tests/v3/data_type/numpy_timedelta64/test_fixtures.py +++ b/packages/zarr-metadata/tests/v3/data_type/numpy_timedelta64/test_fixtures.py @@ -4,13 +4,16 @@ import json from pathlib import Path +from typing import get_args import pytest from pydantic import TypeAdapter from zarr_metadata.v3.data_type.numpy_timedelta64 import ( + NUMPY_TIME_UNIT, NumpyTimedelta64, NumpyTimedelta64FillValue, + NumpyTimeUnit, ) DIR = Path(__file__).parent @@ -24,3 +27,7 @@ def test_data_type() -> None: @pytest.mark.parametrize("case", FILL_VALUES.values(), ids=list(FILL_VALUES)) def test_fill_value(case: object) -> None: TypeAdapter(NumpyTimedelta64FillValue).validate_python(case) + + +def test_time_unit_constant_matches_literal() -> None: + assert set(NUMPY_TIME_UNIT) == set(get_args(NumpyTimeUnit)) diff --git a/pyproject.toml b/pyproject.toml index 9b372192e9..493b18822a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -105,7 +105,7 @@ test = [ "pytest-benchmark==5.2.3", "pytest-codspeed==5.0.3", "tomlkit==0.15.0", - "uv==0.11.19", + "uv==0.11.20", ] remote-tests = [ {include-group = "test"}, @@ -393,7 +393,7 @@ show_error_code_links = true show_error_context = true strict = true warn_unreachable = true -enable_error_code = ["ignore-without-code", "redundant-expr", "truthy-bool"] +enable_error_code = ["ignore-without-code", "redundant-expr", "truthy-bool", "truthy-iterable"] [[tool.mypy.overrides]] module = [ diff --git a/src/zarr/abc/store.py b/src/zarr/abc/store.py index 304d0cddb5..7c187594df 100644 --- a/src/zarr/abc/store.py +++ b/src/zarr/abc/store.py @@ -536,6 +536,8 @@ async def getsize_prefix(self, prefix: str) -> int: from zarr.core.common import concurrent_map from zarr.core.config import config + if prefix != "" and not prefix.endswith("/"): + prefix += "/" keys = [(x,) async for x in self.list_prefix(prefix)] limit = config.get("async.concurrency") sizes = await concurrent_map(keys, self.getsize, limit=limit) diff --git a/src/zarr/codecs/bytes.py b/src/zarr/codecs/bytes.py index 1c67c65e98..240c077627 100644 --- a/src/zarr/codecs/bytes.py +++ b/src/zarr/codecs/bytes.py @@ -62,6 +62,7 @@ def from_dict(cls, data: dict[str, JSON]) -> Self: data, "bytes", require_configuration=False ) configuration_parsed = configuration_parsed or {} + configuration_parsed.setdefault("endian", None) return cls(**configuration_parsed) # type: ignore[arg-type] def to_dict(self) -> dict[str, JSON]: diff --git a/src/zarr/codecs/crc32c_.py b/src/zarr/codecs/crc32c_.py index ebe2ac8f7a..7d41e11637 100644 --- a/src/zarr/codecs/crc32c_.py +++ b/src/zarr/codecs/crc32c_.py @@ -1,11 +1,11 @@ from __future__ import annotations +from collections.abc import Buffer as ABCBuffer from dataclasses import dataclass from typing import TYPE_CHECKING, cast import google_crc32c import numpy as np -import typing_extensions from zarr.abc.codec import BytesBytesCodec from zarr.core.common import JSON, parse_named_configuration @@ -41,9 +41,7 @@ def _decode_sync( inner_bytes = data[:-4] # Need to do a manual cast until https://github.com/numpy/numpy/issues/26783 is resolved - computed_checksum = np.uint32( - google_crc32c.value(cast("typing_extensions.Buffer", inner_bytes)) - ).tobytes() + computed_checksum = np.uint32(google_crc32c.value(cast(ABCBuffer, inner_bytes))).tobytes() stored_checksum = bytes(crc32_bytes) if computed_checksum != stored_checksum: raise ValueError( @@ -65,9 +63,7 @@ def _encode_sync( ) -> Buffer | None: data = chunk_bytes.as_numpy_array() # Calculate the checksum and "cast" it to a numpy array - checksum = np.array( - [google_crc32c.value(cast("typing_extensions.Buffer", data))], dtype=np.uint32 - ) + checksum = np.array([google_crc32c.value(cast(ABCBuffer, data))], dtype=np.uint32) # Append the checksum (as bytes) to the data return chunk_spec.prototype.buffer.from_array_like(np.append(data, checksum.view("B"))) diff --git a/src/zarr/codecs/sharding.py b/src/zarr/codecs/sharding.py index 1fe24719c9..332aab3351 100644 --- a/src/zarr/codecs/sharding.py +++ b/src/zarr/codecs/sharding.py @@ -527,6 +527,8 @@ async def _decode_partial_single( chunk_spec.prototype, chunks_per_shard, all_chunk_coords, + max_gap_bytes=shard_spec.config.sharding_coalesce_max_gap_bytes, + max_coalesced_bytes=shard_spec.config.sharding_coalesce_max_bytes, ) if shard_dict_maybe is None: @@ -846,10 +848,16 @@ async def _load_partial_shard_maybe( prototype: BufferPrototype, chunks_per_shard: tuple[int, ...], all_chunk_coords: set[tuple[int, ...]], + max_gap_bytes: int, + max_coalesced_bytes: int, ) -> ShardMapping | None: """ Read chunks from `byte_getter` for the case where the read is less than a full shard. Returns a mapping of chunk coordinates to bytes or None. + + `max_gap_bytes` and `max_coalesced_bytes` are forwarded to + `Store.get_ranges` to control byte-range coalescing across the requested + chunks. """ shard_index = await self._load_shard_index_maybe(byte_getter, chunks_per_shard) if shard_index is None: @@ -873,7 +881,11 @@ async def _load_partial_shard_maybe( byte_ranges = [byte_range for _, byte_range in chunk_coord_byte_ranges] try: async for group in byte_getter.store.get_ranges( - byte_getter.path, byte_ranges, prototype=prototype + byte_getter.path, + byte_ranges, + prototype=prototype, + max_gap_bytes=max_gap_bytes, + max_coalesced_bytes=max_coalesced_bytes, ): for idx, buf in group: if buf is not None: diff --git a/src/zarr/core/array.py b/src/zarr/core/array.py index 2c2a4622e3..977520b12e 100644 --- a/src/zarr/core/array.py +++ b/src/zarr/core/array.py @@ -1,5 +1,6 @@ from __future__ import annotations +import math import warnings from asyncio import gather from collections.abc import Iterable, Mapping, Sequence @@ -905,7 +906,7 @@ def size(self) -> int: int Total number of elements in the array """ - return np.prod(self.metadata.shape).item() + return math.prod(self.metadata.shape) @property def filters(self) -> tuple[Numcodec, ...] | tuple[ArrayArrayCodec, ...]: diff --git a/src/zarr/core/array_spec.py b/src/zarr/core/array_spec.py index 2b5eb0191c..89163f7d83 100644 --- a/src/zarr/core/array_spec.py +++ b/src/zarr/core/array_spec.py @@ -7,6 +7,7 @@ MemoryOrder, parse_bool, parse_fill_value, + parse_int, parse_order, parse_shapelike, ) @@ -29,6 +30,8 @@ class ArrayConfigParams(TypedDict): order: NotRequired[MemoryOrder] write_empty_chunks: NotRequired[bool] read_missing_chunks: NotRequired[bool] + sharding_coalesce_max_gap_bytes: NotRequired[int] + sharding_coalesce_max_bytes: NotRequired[int] @dataclass(frozen=True) @@ -45,22 +48,42 @@ class ArrayConfig: read_missing_chunks : bool If True, missing chunks will be filled with the array's fill value on read. If False, reading missing chunks will raise a ``ChunkNotFoundError``. + sharding_coalesce_max_gap_bytes : int + When reading multiple chunks from the same shard, nearby byte ranges + separated by no more than this many bytes are coalesced into a single + request to the store. + sharding_coalesce_max_bytes : int + Requests will not be coalesced if doing so would exceed this byte size. """ order: MemoryOrder write_empty_chunks: bool read_missing_chunks: bool + sharding_coalesce_max_gap_bytes: int + sharding_coalesce_max_bytes: int def __init__( - self, order: MemoryOrder, write_empty_chunks: bool, *, read_missing_chunks: bool = True + self, + order: MemoryOrder, + write_empty_chunks: bool, + *, + read_missing_chunks: bool = True, + sharding_coalesce_max_gap_bytes: int = 1 << 20, # 1 MiB + sharding_coalesce_max_bytes: int = 16 << 20, # 16 MiB ) -> None: order_parsed = parse_order(order) write_empty_chunks_parsed = parse_bool(write_empty_chunks) read_missing_chunks_parsed = parse_bool(read_missing_chunks) + sharding_coalesce_max_gap_bytes_parsed = parse_int(sharding_coalesce_max_gap_bytes) + sharding_coalesce_max_bytes_parsed = parse_int(sharding_coalesce_max_bytes) object.__setattr__(self, "order", order_parsed) object.__setattr__(self, "write_empty_chunks", write_empty_chunks_parsed) object.__setattr__(self, "read_missing_chunks", read_missing_chunks_parsed) + object.__setattr__( + self, "sharding_coalesce_max_gap_bytes", sharding_coalesce_max_gap_bytes_parsed + ) + object.__setattr__(self, "sharding_coalesce_max_bytes", sharding_coalesce_max_bytes_parsed) @classmethod def from_dict(cls, data: ArrayConfigParams) -> Self: @@ -72,7 +95,8 @@ def from_dict(cls, data: ArrayConfigParams) -> Self: kwargs_out: ArrayConfigParams = {} for f in fields(ArrayConfig): field_name = cast( - "Literal['order', 'write_empty_chunks', 'read_missing_chunks']", f.name + "Literal['order', 'write_empty_chunks', 'read_missing_chunks', 'sharding_coalesce_max_gap_bytes', 'sharding_coalesce_max_bytes']", + f.name, ) if field_name not in data: kwargs_out[field_name] = zarr_config.get(f"array.{field_name}") @@ -88,6 +112,8 @@ def to_dict(self) -> ArrayConfigParams: "order": self.order, "write_empty_chunks": self.write_empty_chunks, "read_missing_chunks": self.read_missing_chunks, + "sharding_coalesce_max_gap_bytes": self.sharding_coalesce_max_gap_bytes, + "sharding_coalesce_max_bytes": self.sharding_coalesce_max_bytes, } diff --git a/src/zarr/core/chunk_grids.py b/src/zarr/core/chunk_grids.py index 7459908e0a..2cb9762775 100644 --- a/src/zarr/core/chunk_grids.py +++ b/src/zarr/core/chunk_grids.py @@ -852,7 +852,7 @@ def _guess_num_chunks_per_axis_shard( ------- The number of chunks per axis. """ - bytes_per_chunk = np.prod(chunk_shape) * item_size + bytes_per_chunk = math.prod(chunk_shape) * item_size if max_bytes < bytes_per_chunk: return 1 num_axes = len(chunk_shape) diff --git a/src/zarr/core/codec_pipeline.py b/src/zarr/core/codec_pipeline.py index 5c26681d6b..23ecb0e255 100644 --- a/src/zarr/core/codec_pipeline.py +++ b/src/zarr/core/codec_pipeline.py @@ -1,7 +1,7 @@ from __future__ import annotations from dataclasses import dataclass, field -from itertools import islice, pairwise +from itertools import batched, pairwise from typing import TYPE_CHECKING, Any from warnings import warn @@ -42,14 +42,6 @@ def _unzip2[T, U](iterable: Iterable[tuple[T, U]]) -> tuple[list[T], list[U]]: return (out0, out1) -def batched[T](iterable: Iterable[T], n: int) -> Iterable[tuple[T, ...]]: - if n < 1: - raise ValueError("n must be at least one") - it = iter(iterable) - while batch := tuple(islice(it, n)): - yield batch - - def resolve_batched(codec: Codec, chunk_specs: Iterable[ArraySpec]) -> Iterable[ArraySpec]: return [codec.resolve_metadata(chunk_spec) for chunk_spec in chunk_specs] @@ -679,6 +671,7 @@ def codecs_from_list( "must be preceded by either another BytesBytesCodec, or an ArrayBytesCodec. " f"Got {type(prev_codec)} instead." ) + raise TypeError(msg) bytes_bytes += (cur_codec,) else: raise TypeError diff --git a/src/zarr/core/common.py b/src/zarr/core/common.py index eafffa1818..20664e553e 100644 --- a/src/zarr/core/common.py +++ b/src/zarr/core/common.py @@ -1,9 +1,7 @@ from __future__ import annotations import asyncio -import functools import math -import operator import warnings from collections.abc import Iterable, Mapping, Sequence from enum import Enum @@ -83,7 +81,7 @@ class NamedRequiredConfig[TName: str, TConfig: Mapping[str, object]](TypedDict): def product(tup: tuple[int, ...]) -> int: - return functools.reduce(operator.mul, tup, 1) + return math.prod(tup) def ceildiv(a: float, b: float) -> int: @@ -217,6 +215,12 @@ def parse_bool(data: Any) -> bool: raise ValueError(f"Expected bool, got {data} instead.") +def parse_int(data: Any) -> int: + if isinstance(data, int) and not isinstance(data, bool): + return data + raise ValueError(f"Expected int, got {data} instead.") + + def _warn_write_empty_chunks_kwarg() -> None: # TODO: link to docs page on array configuration in this message msg = ( diff --git a/src/zarr/core/config.py b/src/zarr/core/config.py index 7dcbc78e31..08d2a50ace 100644 --- a/src/zarr/core/config.py +++ b/src/zarr/core/config.py @@ -99,6 +99,8 @@ def enable_gpu(self) -> ConfigSet: "read_missing_chunks": True, "target_shard_size_bytes": None, "rectilinear_chunks": False, + "sharding_coalesce_max_gap_bytes": 1 << 20, # 1 MiB + "sharding_coalesce_max_bytes": 16 << 20, # 16 MiB }, "async": {"concurrency": 10, "timeout": None}, "threading": {"max_workers": None}, diff --git a/src/zarr/core/indexing.py b/src/zarr/core/indexing.py index d205d49a11..f6eb495cd9 100644 --- a/src/zarr/core/indexing.py +++ b/src/zarr/core/indexing.py @@ -1,12 +1,12 @@ from __future__ import annotations import itertools +import math import numbers -import operator from collections.abc import Iterator, Sequence from dataclasses import dataclass from enum import Enum -from functools import lru_cache, reduce +from functools import lru_cache from types import EllipsisType from typing import ( TYPE_CHECKING, @@ -1187,7 +1187,7 @@ def __init__( cdata_shape = (1,) else: cdata_shape = tuple(g.nchunks for g in dim_grids) - nchunks = reduce(operator.mul, cdata_shape, 1) + nchunks = math.prod(cdata_shape) # some initial normalization selection_normalized = cast("CoordinateSelectionNormalized", ensure_tuple(selection)) diff --git a/src/zarr/core/sync.py b/src/zarr/core/sync.py index 260d4ad841..160950ba64 100644 --- a/src/zarr/core/sync.py +++ b/src/zarr/core/sync.py @@ -8,8 +8,6 @@ from concurrent.futures import ThreadPoolExecutor, wait from typing import TYPE_CHECKING -from typing_extensions import ParamSpec - from zarr.core.config import config if TYPE_CHECKING: @@ -19,8 +17,6 @@ logger = logging.getLogger(__name__) -P = ParamSpec("P") - # From https://github.com/fsspec/filesystem_spec/blob/master/fsspec/asyn.py iothread: list[threading.Thread | None] = [None] # dedicated IO thread diff --git a/src/zarr/storage/_fsspec.py b/src/zarr/storage/_fsspec.py index 29201a6fee..89d788af1a 100644 --- a/src/zarr/storage/_fsspec.py +++ b/src/zarr/storage/_fsspec.py @@ -424,30 +424,31 @@ async def get_partial_values( key_ranges: Iterable[tuple[str, ByteRequest | None]], ) -> list[Buffer | None]: # docstring inherited - if key_ranges: - # _cat_ranges expects a list of paths, start, and end ranges, so we need to reformat each ByteRequest. - key_ranges = list(key_ranges) - paths: list[str] = [] - starts: list[int | None] = [] - stops: list[int | None] = [] - for key, byte_range in key_ranges: - paths.append(_dereference_path(self.path, key)) - if byte_range is None: - starts.append(None) - stops.append(None) - elif isinstance(byte_range, RangeByteRequest): - starts.append(byte_range.start) - stops.append(byte_range.end) - elif isinstance(byte_range, OffsetByteRequest): - starts.append(byte_range.offset) - stops.append(None) - elif isinstance(byte_range, SuffixByteRequest): - starts.append(-byte_range.suffix) - stops.append(None) - else: - raise ValueError(f"Unexpected byte_range, got {byte_range}.") - else: + # Materialise first: key_ranges may be a one-shot iterable, so a bare + # truthiness check (e.g. `if key_ranges`) would be unreliable for an + # empty generator. _cat_ranges also expects lists of paths/starts/stops. + key_ranges = list(key_ranges) + if not key_ranges: return [] + paths: list[str] = [] + starts: list[int | None] = [] + stops: list[int | None] = [] + for key, byte_range in key_ranges: + paths.append(_dereference_path(self.path, key)) + if byte_range is None: + starts.append(None) + stops.append(None) + elif isinstance(byte_range, RangeByteRequest): + starts.append(byte_range.start) + stops.append(byte_range.end) + elif isinstance(byte_range, OffsetByteRequest): + starts.append(byte_range.offset) + stops.append(None) + elif isinstance(byte_range, SuffixByteRequest): + starts.append(-byte_range.suffix) + stops.append(None) + else: + raise ValueError(f"Unexpected byte_range, got {byte_range}.") # TODO: expectations for exceptions or missing keys? res = await self.fs._cat_ranges(paths, starts, stops, on_error="return") # the following is an s3-specific condition we probably don't want to leak diff --git a/src/zarr/storage/_local.py b/src/zarr/storage/_local.py index 3d9882d3db..1627c1a6b5 100644 --- a/src/zarr/storage/_local.py +++ b/src/zarr/storage/_local.py @@ -363,10 +363,10 @@ async def move(self, dest_root: Path | str) -> None: if isinstance(dest_root, str): dest_root = Path(dest_root) os.makedirs(dest_root.parent, exist_ok=True) - if os.path.exists(dest_root): + if dest_root.exists(): raise FileExistsError(f"Destination root {dest_root} already exists.") shutil.move(self.root, dest_root) self.root = dest_root async def getsize(self, key: str) -> int: - return os.path.getsize(self.root / key) + return (self.root / key).stat().st_size diff --git a/src/zarr/storage/_logging.py b/src/zarr/storage/_logging.py index 5de300c144..c6f58ccd61 100644 --- a/src/zarr/storage/_logging.py +++ b/src/zarr/storage/_logging.py @@ -179,6 +179,7 @@ async def get_partial_values( key_ranges: Iterable[tuple[str, ByteRequest | None]], ) -> list[Buffer | None]: # docstring inherited + key_ranges = list(key_ranges) keys = ",".join([k[0] for k in key_ranges]) with self.log(keys): return await self._store.get_partial_values(prototype=prototype, key_ranges=key_ranges) diff --git a/src/zarr/storage/_utils.py b/src/zarr/storage/_utils.py index 1f8e9b0a29..b100f862cf 100644 --- a/src/zarr/storage/_utils.py +++ b/src/zarr/storage/_utils.py @@ -153,7 +153,7 @@ def _normalize_byte_range_index(data: Buffer, byte_range: ByteRequest | None) -> start = byte_range.offset stop = len(data) + 1 elif isinstance(byte_range, SuffixByteRequest): - start = len(data) - byte_range.suffix + start = max(0, len(data) - byte_range.suffix) stop = len(data) + 1 else: raise ValueError(f"Unexpected byte_range, got {byte_range}.") diff --git a/src/zarr/storage/_zip.py b/src/zarr/storage/_zip.py index 897797e999..430b0c3e2a 100644 --- a/src/zarr/storage/_zip.py +++ b/src/zarr/storage/_zip.py @@ -120,6 +120,8 @@ def __setstate__(self, state: dict[str, Any]) -> None: def close(self) -> None: # docstring inherited + if not self._is_open: + return super().close() with self._lock: self._zf.close() diff --git a/src/zarr/testing/stateful.py b/src/zarr/testing/stateful.py index d6c43f4ecc..9817ebd618 100644 --- a/src/zarr/testing/stateful.py +++ b/src/zarr/testing/stateful.py @@ -1,6 +1,6 @@ import builtins import functools -from collections.abc import Callable +from collections.abc import Callable, Iterable from typing import Any, cast import hypothesis.extra.numpy as npst @@ -18,7 +18,12 @@ import zarr from zarr import Array -from zarr.abc.store import Store +from zarr.abc.store import ( + OffsetByteRequest, + RangeByteRequest, + Store, + SuffixByteRequest, +) from zarr.codecs.bytes import BytesCodec from zarr.core.buffer import Buffer, BufferPrototype, cpu, default_buffer_prototype from zarr.core.sync import SyncMixin @@ -460,7 +465,7 @@ def get(self, key: str, prototype: BufferPrototype) -> Buffer | None: return self._sync(self.store.get(key, prototype=prototype)) def get_partial_values( - self, key_ranges: builtins.list[Any], prototype: BufferPrototype + self, key_ranges: Iterable[Any], prototype: BufferPrototype ) -> builtins.list[Buffer | None]: return self._sync(self.store.get_partial_values(prototype=prototype, key_ranges=key_ranges)) @@ -476,6 +481,9 @@ def clear(self) -> None: def exists(self, key: str) -> bool: return self._sync(self.store.exists(key)) + def getsize_prefix(self, prefix: str) -> int: + return self._sync(self.store.getsize_prefix(prefix)) + def list_dir(self, prefix: str) -> None: raise NotImplementedError @@ -555,7 +563,9 @@ def get_partial_values(self, data: DataObject) -> None: key_ranges(keys=st.sampled_from(sorted(self.model.keys())), max_size=MAX_BINARY_SIZE) ) note(f"(get partial) {key_range=}") - obs_maybe = self.store.get_partial_values(key_range, self.prototype) + # Pass a one-shot generator rather than a list: stores (and wrappers such + # as LoggingStore) must not exhaust the iterable before using it. + obs_maybe = self.store.get_partial_values((kr for kr in key_range), self.prototype) observed = [] for obs in obs_maybe: @@ -565,9 +575,23 @@ def get_partial_values(self, data: DataObject) -> None: model_vals_ls = [] for key, byte_range in key_range: - start = byte_range.start - stop = byte_range.end - model_vals_ls.append(self.model[key][start:stop]) + # Independently model each ByteRequest variant (do NOT reuse the + # store's _normalize_byte_range_index helper, so this stays an + # independent oracle). Bounds may exceed the value length. + value = self.model[key] + n = len(value) + if byte_range is None: + expected = value[:] + elif isinstance(byte_range, RangeByteRequest): + expected = value[byte_range.start : byte_range.end] + elif isinstance(byte_range, OffsetByteRequest): + expected = value[byte_range.offset :] + elif isinstance(byte_range, SuffixByteRequest): + # "last suffix bytes"; suffix > n means the whole value. + expected = value[max(0, n - byte_range.suffix) :] + else: + raise AssertionError(f"unexpected byte_range {byte_range!r}") + model_vals_ls.append(expected) assert all( obs == exp.to_bytes() for obs, exp in zip(observed, model_vals_ls, strict=True) @@ -612,6 +636,21 @@ def exists(self, key: str) -> None: assert self.store.exists(key) == (key in self.model) + @precondition(lambda self: len(self.model.keys()) > 0) + @rule(data=st.data()) + def getsize_prefix(self, data: DataObject) -> None: + # Measure the size under the first path segment of some existing key. + # getsize_prefix(node) must count only keys under the directory "node/", + # not sibling keys that merely share the string prefix (e.g. measuring + # "a" must not include a sibling key "ab/..."). + key = data.draw(st.sampled_from(sorted(self.model.keys()))) + node = key.split("/")[0] + note(f"(getsize_prefix) {node=}") + + observed = self.store.getsize_prefix(node) + expected = sum(len(value) for k, value in self.model.items() if k.startswith(node + "/")) + assert observed == expected, (observed, expected, node) + @invariant() def check_paths_equal(self) -> None: note("Checking that paths are equal") diff --git a/src/zarr/testing/store.py b/src/zarr/testing/store.py index 81024c85c8..11ceeee83a 100644 --- a/src/zarr/testing/store.py +++ b/src/zarr/testing/store.py @@ -299,10 +299,16 @@ async def test_getsize(self, store: S, key: str, data: bytes) -> None: async def test_getsize_prefix(self, store: S) -> None: """ Test the result of store.getsize_prefix(). + + Includes a sibling key ("cc/0") that shares the string prefix "c" but + belongs to a different directory: getsize_prefix("c") must not count it, + i.e. the prefix is matched as a directory ("c/...") not a raw substring. """ data_buf = self.buffer_cls.from_bytes(b"\x01\x02\x03\x04") keys = ["c/0/0", "c/0/1", "c/1/0", "c/1/1"] - keys_values = [(k, data_buf) for k in keys] + # Sibling directory sharing the "c" string prefix; must be excluded. + sibling_keys = ["cc/0"] + keys_values = [(k, data_buf) for k in keys + sibling_keys] await store._set_many(keys_values) expected = len(data_buf) * len(keys) observed = await store.getsize_prefix("c") @@ -370,11 +376,19 @@ async def test_get_partial_values( for key, _ in key_ranges: await self.set(store, key, self.buffer_cls.from_bytes(bytes(key, encoding="utf-8"))) - # read back just part of it + # read back just part of it. Pass key_ranges as a one-shot generator + # (a valid Iterable per the method signature) to ensure stores and + # wrappers do not exhaust the iterable before handing it to the backend. observed_maybe = await store.get_partial_values( - prototype=default_buffer_prototype(), key_ranges=key_ranges + prototype=default_buffer_prototype(), + key_ranges=(kr for kr in key_ranges), ) + # One result must be returned per requested key range. Checking this + # explicitly guards against a store/wrapper exhausting the key_ranges + # iterable early and silently returning fewer (or no) results. + assert len(observed_maybe) == len(key_ranges) + observed: list[Buffer] = [] expected: list[Buffer] = [] @@ -382,8 +396,7 @@ async def test_get_partial_values( assert obs is not None observed.append(obs) - for idx in range(len(observed)): - key, byte_range = key_ranges[idx] + for key, byte_range in key_ranges: result = await store.get( key, prototype=default_buffer_prototype(), byte_range=byte_range ) diff --git a/src/zarr/testing/strategies.py b/src/zarr/testing/strategies.py index 7d6556a359..0ef1ba99bb 100644 --- a/src/zarr/testing/strategies.py +++ b/src/zarr/testing/strategies.py @@ -11,7 +11,13 @@ from hypothesis.strategies import SearchStrategy import zarr -from zarr.abc.store import RangeByteRequest, Store +from zarr.abc.store import ( + ByteRequest, + OffsetByteRequest, + RangeByteRequest, + Store, + SuffixByteRequest, +) from zarr.codecs.bytes import BytesCodec from zarr.codecs.crc32c_ import Crc32cCodec from zarr.codecs.sharding import SUBCHUNK_WRITE_ORDER, ShardingCodec, SubchunkWriteOrder @@ -654,22 +660,29 @@ def predicate(value: tuple[Any, ...]) -> bool: def key_ranges( keys: SearchStrategy[str] = node_names, max_size: int = sys.maxsize -) -> SearchStrategy[list[tuple[str, RangeByteRequest]]]: +) -> SearchStrategy[list[tuple[str, ByteRequest | None]]]: """ Function to generate key_ranges strategy for get_partial_values() returns list strategy w/ form:: - [(key, (range_start, range_end)), - (key, (range_start, range_end)),...] + [(key, byte_request), + (key, byte_request),...] + + where ``byte_request`` is ``None`` or any of the concrete ``ByteRequest`` + subtypes. The bounds are drawn independently of each value's length, so the + offsets/suffixes routinely exceed the data and exercise the clamping logic + in ``_normalize_byte_range_index``. """ - def make_request(start: int, length: int) -> RangeByteRequest: + def make_range(start: int, length: int) -> RangeByteRequest: return RangeByteRequest(start, end=min(start + length, max_size)) - byte_ranges = st.builds( - make_request, - start=st.integers(min_value=0, max_value=max_size), - length=st.integers(min_value=0, max_value=max_size), + bound = st.integers(min_value=0, max_value=max_size) + byte_ranges: SearchStrategy[ByteRequest | None] = st.one_of( + st.none(), + st.builds(make_range, start=bound, length=bound), + st.builds(OffsetByteRequest, offset=bound), + st.builds(SuffixByteRequest, suffix=bound), ) key_tuple = st.tuples(keys, byte_ranges) return st.lists(key_tuple, min_size=1, max_size=10) diff --git a/tests/test_codec_pipeline.py b/tests/test_codec_pipeline.py index fa41c2867b..4d596164db 100644 --- a/tests/test_codec_pipeline.py +++ b/tests/test_codec_pipeline.py @@ -1,15 +1,28 @@ from __future__ import annotations +from typing import TYPE_CHECKING + import numpy as np import pytest +pytest.importorskip("hypothesis") + +import hypothesis.strategies as st +from hypothesis import given + import zarr -from zarr.codecs import BytesCodec, CastValue +from zarr.codecs import BytesCodec, CastValue, GzipCodec, TransposeCodec from zarr.core.array import _get_chunk_spec from zarr.core.buffer.core import default_buffer_prototype +from zarr.core.codec_pipeline import codecs_from_list from zarr.core.indexing import BasicIndexer from zarr.storage import MemoryStore +if TYPE_CHECKING: + from collections.abc import Callable + + from zarr.abc.codec import Codec + @pytest.mark.parametrize( ("write_slice", "read_slice", "expected_statuses"), @@ -120,3 +133,65 @@ def test_codec_pipeline_threads_dtype_through_evolve(source_dtype: str, target_d ) arr[:] = np.asarray([0, 1, 2, 3], dtype=source_dtype) np.testing.assert_array_equal(arr[:], np.asarray([0, 1, 2, 3], dtype=source_dtype)) + + +# Property-based check of codecs_from_list ordering validation. +# +# Valid codec orderings are exactly: (ArrayArrayCodec)* (ArrayBytesCodec) +# (BytesBytesCodec)*. codecs_from_list walks adjacent pairs and must raise +# TypeError the moment a codec appears in a structurally invalid position -- +# notably, a BytesBytesCodec immediately following an ArrayArrayCodec with no +# ArrayBytesCodec in between (which previously built an error message but never +# raised it, falling through to an unrelated ValueError instead). +_AA = "AA" # ArrayArrayCodec -> TransposeCodec +_AB = "AB" # ArrayBytesCodec -> BytesCodec +_BB = "BB" # BytesBytesCodec -> GzipCodec + +_CODEC_FACTORY: dict[str, Callable[[], Codec]] = { + _AA: lambda: TransposeCodec(order=(0, 1)), + _AB: BytesCodec, + _BB: GzipCodec, +} + + +def _expected_codec_order_outcome(labels: list[str]) -> str: + """Independently predict codecs_from_list's outcome: 'TypeError', + 'ValueError' or 'ok', mirroring its left-to-right scan and the order in + which it checks ordering violations (TypeError) vs. the ArrayBytes-count + constraints (ValueError).""" + prev = None + seen_array_bytes = False + for cur in labels: + if cur == _AA: + if prev in (_AB, _BB): + return "TypeError" + elif cur == _AB: + if prev == _BB: + return "TypeError" + if seen_array_bytes: + return "ValueError" # two ArrayBytesCodecs + seen_array_bytes = True + else: # _BB + if prev == _AA: + return "TypeError" + prev = cur + if not seen_array_bytes: + return "ValueError" # Required ArrayBytesCodec was not found + return "ok" + + +@given(labels=st.lists(st.sampled_from([_AA, _AB, _BB]), min_size=1, max_size=5)) +def test_codecs_from_list_outcome_matches_order_rules(labels: list[str]) -> None: + codecs = [_CODEC_FACTORY[label]() for label in labels] + expected = _expected_codec_order_outcome(labels) + if expected == "TypeError": + with pytest.raises(TypeError): + codecs_from_list(codecs) + elif expected == "ValueError": + with pytest.raises(ValueError): + codecs_from_list(codecs) + else: + # Valid ordering: must classify without raising. + aa, _ab, bb = codecs_from_list(codecs) + assert labels.count(_AA) == len(aa) + assert labels.count(_BB) == len(bb) diff --git a/tests/test_codecs/test_bytes.py b/tests/test_codecs/test_bytes.py index 25c786a405..03dd0b40c6 100644 --- a/tests/test_codecs/test_bytes.py +++ b/tests/test_codecs/test_bytes.py @@ -5,10 +5,14 @@ import enum import sys import warnings -from typing import Any, cast +from typing import TYPE_CHECKING, Any, Literal, cast +import numpy as np import pytest +import zarr +from tests.conftest import Expect, ExpectFail +from zarr.abc.codec import SupportsSyncCodec from zarr.codecs.bytes import ( ENDIAN, BytesCodec, @@ -16,9 +20,83 @@ EndianLiteral, ) from zarr.core.array_spec import ArrayConfig, ArraySpec -from zarr.core.buffer import default_buffer_prototype +from zarr.core.buffer import NDBuffer, default_buffer_prototype +from zarr.core.dtype import get_data_type_from_native_dtype from zarr.core.dtype.npy.int import Int8, Int32 from zarr.core.dtype.npy.structured import Struct +from zarr.storage import StorePath + +from .test_codecs import _AsyncArrayProxy + +if TYPE_CHECKING: + from zarr.abc.store import Store + + +@pytest.mark.parametrize("store", ["local", "memory"], indirect=["store"]) +@pytest.mark.parametrize("input_dtype", [">u2", "u2", " None: + """ + The `bytes` codec stores multi-byte data in the byte order configured on the + codec, regardless of the input array's byte order, and reads it back to the + original values. The input-dtype/store-endian cross-product exercises the + encode-side byteswap (input byte order != store byte order) and the no-op + case alike. Compression is disabled so the stored chunk is the codec's raw + output and its byte layout can be asserted directly. + """ + data = np.arange(0, 256, dtype=input_dtype).reshape((16, 16)) + path = "endian" + spath = StorePath(store, path) + a = await zarr.api.asynchronous.create_array( + spath, + shape=data.shape, + chunks=(16, 16), + dtype="uint16", + fill_value=0, + compressors=None, + serializer=BytesCodec(endian=store_endian), + ) + + await _AsyncArrayProxy(a)[:, :].set(data) + + # The stored chunk is laid out in the byte order configured on the codec. + stored = await store.get(f"{path}/c/0/0", prototype=default_buffer_prototype()) + assert stored is not None + expected_dtype = ">u2" if store_endian == "big" else " None: + assert isinstance(BytesCodec(), SupportsSyncCodec) + + +def test_bytes_codec_sync_roundtrip() -> None: + codec = BytesCodec() + arr = np.arange(100, dtype="float64") + zdtype = get_data_type_from_native_dtype(arr.dtype) + spec = ArraySpec( + shape=arr.shape, + dtype=zdtype, + fill_value=zdtype.cast_scalar(0), + config=ArrayConfig(order="C", write_empty_chunks=True), + prototype=default_buffer_prototype(), + ) + nd_buf: NDBuffer = default_buffer_prototype().nd_buffer.from_numpy_array(arr) + + codec = codec.evolve_from_array_spec(spec) + + encoded = codec._encode_sync(nd_buf, spec) + assert encoded is not None + decoded = codec._decode_sync(encoded, spec) + np.testing.assert_array_equal(arr, decoded.as_numpy_array()) @pytest.mark.parametrize("endian", ENDIAN) @@ -46,6 +124,43 @@ def test_bytes_codec_json_roundtrip(endian: EndianLiteral) -> None: assert restored == codec +# to_dict and from_dict are inverses over this (endian setting, wire dict) mapping: +# to_dict turns the endian setting into the dict; from_dict recovers it. +_ENDIAN_DICT_CASES: list[Expect[EndianLiteral | None, dict[str, Any]]] = [ + Expect( + input="little", + output={"name": "bytes", "configuration": {"endian": "little"}}, + id="little", + ), + Expect( + input="big", + output={"name": "bytes", "configuration": {"endian": "big"}}, + id="big", + ), + Expect(input=None, output={"name": "bytes"}, id="missing"), +] + + +@pytest.mark.parametrize("case", _ENDIAN_DICT_CASES, ids=lambda c: c.id) +def test_to_dict(case: Expect[EndianLiteral | None, dict[str, Any]]) -> None: + assert BytesCodec(endian=case.input).to_dict() == case.output + + +@pytest.mark.parametrize("case", _ENDIAN_DICT_CASES, ids=lambda c: c.id) +def test_from_dict(case: Expect[EndianLiteral | None, dict[str, Any]]) -> None: + assert BytesCodec.from_dict(case.output).endian == case.input + + +@pytest.mark.parametrize("endian", ["little", "big", pytest.param(None, id="missing")]) +def test_roundtrip(endian: EndianLiteral | None) -> None: + codec = BytesCodec(endian=endian) + + encoded = codec.to_dict() + roundtripped = BytesCodec.from_dict(encoded) + + assert codec == roundtripped + + @pytest.mark.parametrize( ("member", "expected"), [("little", "little"), ("big", "big")], @@ -105,14 +220,25 @@ def test_bytes_codec_init_with_deprecated_class_member() -> None: assert codec.endian == "little" -def test_bytes_codec_rejects_unknown_endian() -> None: +@pytest.mark.parametrize( + "case", + [ + ExpectFail( + input="north", + exception=ValueError, + id="unknown-string", + msg="endian must be one of", + ), + ], + ids=lambda c: c.id, +) +def test_bytes_codec_rejects_unknown_endian(case: ExpectFail[Any]) -> None: """ - `BytesCodec.__init__` raises `ValueError` when given a string outside + `BytesCodec.__init__` raises `ValueError` when given a value outside `ENDIAN`, and the error message names the offending parameter. """ - kwargs: dict[str, Any] = {"endian": "north"} - with pytest.raises(ValueError, match="endian must be one of"): - BytesCodec(**kwargs) + with case.raises(): + BytesCodec(endian=case.input) def test_endian_attribute_error_for_unknown_member() -> None: diff --git a/tests/test_codecs/test_endian.py b/tests/test_codecs/test_endian.py deleted file mode 100644 index c505cee828..0000000000 --- a/tests/test_codecs/test_endian.py +++ /dev/null @@ -1,89 +0,0 @@ -from typing import Literal - -import numpy as np -import pytest - -import zarr -from zarr.abc.codec import SupportsSyncCodec -from zarr.abc.store import Store -from zarr.codecs import BytesCodec -from zarr.core.array_spec import ArrayConfig, ArraySpec -from zarr.core.buffer import NDBuffer, default_buffer_prototype -from zarr.core.dtype import get_data_type_from_native_dtype -from zarr.storage import StorePath - -from .test_codecs import _AsyncArrayProxy - - -@pytest.mark.filterwarnings("ignore:The endianness of the requested serializer") -@pytest.mark.parametrize("store", ["local", "memory"], indirect=["store"]) -@pytest.mark.parametrize("endian", ["big", "little"]) -async def test_endian(store: Store, endian: Literal["big", "little"]) -> None: - data = np.arange(0, 256, dtype="uint16").reshape((16, 16)) - path = "endian" - spath = StorePath(store, path) - a = await zarr.api.asynchronous.create_array( - spath, - shape=data.shape, - chunks=(16, 16), - dtype=data.dtype, - fill_value=0, - chunk_key_encoding={"name": "v2", "separator": "."}, - serializer=BytesCodec(endian=endian), - ) - - await _AsyncArrayProxy(a)[:, :].set(data) - readback_data = await _AsyncArrayProxy(a)[:, :].get() - assert np.array_equal(data, readback_data) - - -def test_bytes_codec_supports_sync() -> None: - assert isinstance(BytesCodec(), SupportsSyncCodec) - - -def test_bytes_codec_sync_roundtrip() -> None: - codec = BytesCodec() - arr = np.arange(100, dtype="float64") - zdtype = get_data_type_from_native_dtype(arr.dtype) - spec = ArraySpec( - shape=arr.shape, - dtype=zdtype, - fill_value=zdtype.cast_scalar(0), - config=ArrayConfig(order="C", write_empty_chunks=True), - prototype=default_buffer_prototype(), - ) - nd_buf: NDBuffer = default_buffer_prototype().nd_buffer.from_numpy_array(arr) - - codec = codec.evolve_from_array_spec(spec) - - encoded = codec._encode_sync(nd_buf, spec) - assert encoded is not None - decoded = codec._decode_sync(encoded, spec) - np.testing.assert_array_equal(arr, decoded.as_numpy_array()) - - -@pytest.mark.filterwarnings("ignore:The endianness of the requested serializer") -@pytest.mark.parametrize("store", ["local", "memory"], indirect=["store"]) -@pytest.mark.parametrize("dtype_input_endian", [">u2", "u2", " None: - data = np.arange(0, 256, dtype=dtype_input_endian).reshape((16, 16)) - path = "endian" - spath = StorePath(store, path) - a = await zarr.api.asynchronous.create_array( - spath, - shape=data.shape, - chunks=(16, 16), - dtype="uint16", - fill_value=0, - chunk_key_encoding={"name": "v2", "separator": "."}, - serializer=BytesCodec(endian=dtype_store_endian), - ) - - await _AsyncArrayProxy(a)[:, :].set(data) - readback_data = await _AsyncArrayProxy(a)[:, :].get() - assert np.array_equal(data, readback_data) diff --git a/tests/test_codecs/test_sharding_unit.py b/tests/test_codecs/test_sharding_unit.py index 6e022ed9fa..2e3872e7a6 100644 --- a/tests/test_codecs/test_sharding_unit.py +++ b/tests/test_codecs/test_sharding_unit.py @@ -1,3 +1,8 @@ +from __future__ import annotations + +from typing import TYPE_CHECKING, cast +from unittest.mock import AsyncMock + import numpy as np import pytest @@ -10,9 +15,14 @@ ) from zarr.core.buffer import default_buffer_prototype from zarr.core.buffer.cpu import Buffer +from zarr.core.config import config from zarr.storage._common import StorePath from zarr.storage._memory import MemoryStore +if TYPE_CHECKING: + from zarr.core.array import ShardsConfigParam + from zarr.core.array_spec import ArrayConfigParams + # ============================================================================ # _ShardIndex tests # ============================================================================ @@ -155,6 +165,8 @@ async def test_load_partial_shard_maybe_index_load_fails() -> None: prototype=default_buffer_prototype(), chunks_per_shard=(2,), all_chunk_coords={(0,)}, + max_gap_bytes=1 << 20, + max_coalesced_bytes=16 << 20, ) assert result is None @@ -187,6 +199,8 @@ async def mock_load_index( prototype=default_buffer_prototype(), chunks_per_shard=chunks_per_shard, all_chunk_coords={(0,), (1,), (2,)}, + max_gap_bytes=1 << 20, + max_coalesced_bytes=16 << 20, ) assert result is not None @@ -220,6 +234,8 @@ async def mock_load_index( prototype=default_buffer_prototype(), chunks_per_shard=chunks_per_shard, all_chunk_coords={(0,), (1,), (2,)}, + max_gap_bytes=1 << 20, + max_coalesced_bytes=16 << 20, ) assert result == {} @@ -251,6 +267,8 @@ async def mock_load_index( prototype=default_buffer_prototype(), chunks_per_shard=chunks_per_shard, all_chunk_coords={(0,), (1,)}, + max_gap_bytes=1 << 20, + max_coalesced_bytes=16 << 20, ) assert result is not None @@ -292,6 +310,8 @@ async def mock_load_index( prototype=default_buffer_prototype(), chunks_per_shard=chunks_per_shard, all_chunk_coords={(0,)}, + max_gap_bytes=1 << 20, + max_coalesced_bytes=16 << 20, ) assert result is None @@ -336,6 +356,8 @@ async def boom(*args: object, **kwargs: object) -> Buffer | None: prototype=default_buffer_prototype(), chunks_per_shard=chunks_per_shard, all_chunk_coords={(0,)}, + max_gap_bytes=1 << 20, + max_coalesced_bytes=16 << 20, ) @@ -368,6 +390,8 @@ async def mock_load_index( prototype=default_buffer_prototype(), chunks_per_shard=chunks_per_shard, all_chunk_coords={(0,), (1,)}, + max_gap_bytes=1 << 20, + max_coalesced_bytes=16 << 20, ) assert result is not None @@ -405,6 +429,8 @@ async def mock_load_index( prototype=default_buffer_prototype(), chunks_per_shard=chunks_per_shard, all_chunk_coords={(0,)}, + max_gap_bytes=1 << 20, + max_coalesced_bytes=16 << 20, ) assert result == {} @@ -486,3 +512,140 @@ def test_is_total_shard_1d() -> None: # Partial partial_coords: set[tuple[int, ...]] = {(0,), (2,)} assert codec._is_total_shard(partial_coords, chunks_per_shard) is False + + +# ============================================================================ +# Coalescing config option tests +# +# Assert that the `array.sharding_coalesce_max_gap_bytes` and +# `array.sharding_coalesce_max_bytes` global config keys flow through +# `ArrayConfig` to `Store.get_ranges` as `max_gap_bytes` / +# `max_coalesced_bytes` kwargs, and that per-array `config={...}` overrides +# the global default. +# ============================================================================ + + +def _trigger_partial_shard_read(array_config: ArrayConfigParams | None = None) -> AsyncMock: + """Build a sharded array on a mocked `MemoryStore`, trigger a partial-shard + read via the public read path, and return the `get_ranges` mock. + """ + import zarr + + chunk_shape = (2,) + shard_shape = (8,) + data = np.arange(8, dtype="int32") + + store = MemoryStore() + store_mock = AsyncMock(wraps=store, spec=store.__class__) + + shards: ShardsConfigParam = { + "shape": shard_shape, + "index_location": "end", + } + a = zarr.create_array( + StorePath(store_mock), + shape=(8,), + chunks=chunk_shape, + shards=shards, + dtype=data.dtype, + fill_value=-1, + config=array_config, + ) + a[:] = data + + store_mock.reset_mock() + + # Read a strict subset of chunks to take the partial-shard read path. + _ = a[0:4] + + return cast(AsyncMock, store_mock.get_ranges) + + +def test_load_partial_shard_forwards_global_config_to_get_ranges() -> None: + """Global `array.sharding_coalesce_*` values flow into ArrayConfig at + array-creation time and are forwarded to `Store.get_ranges`.""" + with config.set( + { + "array.sharding_coalesce_max_gap_bytes": 4242, + "array.sharding_coalesce_max_bytes": 424242, + } + ): + get_ranges_mock = _trigger_partial_shard_read() + + assert get_ranges_mock.call_count >= 1 + for call in get_ranges_mock.call_args_list: + kwargs = call.kwargs + assert kwargs["max_gap_bytes"] == 4242 + assert kwargs["max_coalesced_bytes"] == 424242 + + +def test_load_partial_shard_per_array_config_overrides_global() -> None: + """Per-array `config={...}` passed to `create_array` takes precedence over + the global config and is forwarded to `Store.get_ranges`.""" + with config.set( + { + "array.sharding_coalesce_max_gap_bytes": 4242, + "array.sharding_coalesce_max_bytes": 424242, + } + ): + get_ranges_mock = _trigger_partial_shard_read( + array_config={ + "sharding_coalesce_max_gap_bytes": 99, + "sharding_coalesce_max_bytes": 9999, + }, + ) + + assert get_ranges_mock.call_count >= 1 + for call in get_ranges_mock.call_args_list: + kwargs = call.kwargs + assert kwargs["max_gap_bytes"] == 99 + assert kwargs["max_coalesced_bytes"] == 9999 + + +def test_load_partial_shard_uses_config_defaults() -> None: + """Without explicit config, defaults from `zarr.config` are forwarded.""" + get_ranges_mock = _trigger_partial_shard_read() + + assert get_ranges_mock.call_count >= 1 + for call in get_ranges_mock.call_args_list: + kwargs = call.kwargs + assert kwargs["max_gap_bytes"] == config.get("array.sharding_coalesce_max_gap_bytes") + assert kwargs["max_coalesced_bytes"] == config.get("array.sharding_coalesce_max_bytes") + + +async def test_load_partial_shard_explicit_kwargs_passthrough( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """`_load_partial_shard_maybe` forwards its explicit kwargs to `get_ranges`.""" + codec = ShardingCodec(chunk_shape=(2,)) + chunks_per_shard = (4,) + + index = _ShardIndex.create_empty(chunks_per_shard) + index.set_chunk_slice((0,), slice(0, 100)) + index.set_chunk_slice((2,), slice(200, 300)) + + store = MemoryStore() + await store.set("shard", Buffer.from_bytes(b"x" * 300)) + store_mock = AsyncMock(wraps=store, spec=store.__class__) + byte_getter = StorePath(store_mock, "shard") + + async def mock_load_index( + self: ShardingCodec, byte_getter: StorePath, cps: tuple[int, ...] + ) -> _ShardIndex: + return index + + monkeypatch.setattr(ShardingCodec, "_load_shard_index_maybe", mock_load_index) + + await codec._load_partial_shard_maybe( + byte_getter=byte_getter, + prototype=default_buffer_prototype(), + chunks_per_shard=chunks_per_shard, + all_chunk_coords={(0,), (2,)}, + max_gap_bytes=12345, + max_coalesced_bytes=67890, + ) + + store_mock.get_ranges.assert_called_once() + kwargs = store_mock.get_ranges.call_args.kwargs + assert kwargs["max_gap_bytes"] == 12345 + assert kwargs["max_coalesced_bytes"] == 67890 diff --git a/tests/test_common.py b/tests/test_common.py index 0dedde1d6b..2fe0743e14 100644 --- a/tests/test_common.py +++ b/tests/test_common.py @@ -9,6 +9,7 @@ from zarr.core.common import ( ANY_ACCESS_MODE, AccessModeLiteral, + parse_int, parse_name, parse_shapelike, product, @@ -72,6 +73,18 @@ def test_parse_indexing_order_invalid(data: Any) -> None: parse_indexing_order(data) +@pytest.mark.parametrize("data", ["1", 1.0, True, False, None, [1], (1,)]) +def test_parse_int_invalid(data: Any) -> None: + """Non-int values (including bools, which are int subclasses) are rejected.""" + with pytest.raises(ValueError, match="Expected int"): + parse_int(data) + + +@pytest.mark.parametrize("data", [0, 1, -1, 2**63]) +def test_parse_int_valid(data: int) -> None: + assert parse_int(data) == data + + @pytest.mark.parametrize("data", ["C", "F"]) def parse_indexing_order_valid(data: Literal["C", "F"]) -> None: assert parse_indexing_order(data) == data diff --git a/tests/test_config.py b/tests/test_config.py index 4e293e968f..a758378dc7 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -1,3 +1,4 @@ +import inspect import os from collections.abc import Iterable from typing import Any @@ -17,7 +18,7 @@ Crc32cCodec, ShardingCodec, ) -from zarr.core.array_spec import ArraySpec +from zarr.core.array_spec import ArrayConfig, ArraySpec from zarr.core.buffer import NDBuffer from zarr.core.buffer.core import Buffer from zarr.core.codec_pipeline import BatchedCodecPipeline @@ -56,6 +57,8 @@ def test_config_defaults_set() -> None: "read_missing_chunks": True, "target_shard_size_bytes": None, "rectilinear_chunks": False, + "sharding_coalesce_max_gap_bytes": 1 << 20, + "sharding_coalesce_max_bytes": 16 << 20, }, "async": {"concurrency": 10, "timeout": None}, "threading": {"max_workers": None}, @@ -109,6 +112,25 @@ def test_config_defaults_set() -> None: assert config.get("json_indent") == 2 +def test_array_config_init_defaults_match_global_config() -> None: + """Each `ArrayConfig.__init__` parameter that has a default must match the + value of `array.` in the global config. Catches drift between + the two sources of truth.""" + params = inspect.signature(ArrayConfig.__init__).parameters + has_defaults = { + name: p.default + for name, p in params.items() + if name != "self" and p.default is not inspect.Parameter.empty + } + assert has_defaults, "expected at least one default to check" + for name, default in has_defaults.items(): + assert default == config.get(f"array.{name}"), ( + f"ArrayConfig.__init__ default for {name!r} ({default!r}) does not " + f"match global config value for 'array.{name}' " + f"({config.get(f'array.{name}')!r})" + ) + + @pytest.mark.parametrize( ("key", "old_val", "new_val"), [("array.order", "C", "F"), ("async.concurrency", 10, 128), ("json_indent", 2, 0)], diff --git a/tests/test_store/test_utils.py b/tests/test_store/test_utils.py index b1934e7eae..291526fab8 100644 --- a/tests/test_store/test_utils.py +++ b/tests/test_store/test_utils.py @@ -5,7 +5,9 @@ import pytest -from zarr.storage._utils import ParsedStoreUrl, parse_store_url +from zarr.abc.store import SuffixByteRequest +from zarr.core.buffer.core import default_buffer_prototype +from zarr.storage._utils import ParsedStoreUrl, _normalize_byte_range_index, parse_store_url class TestParseStoreUrl: @@ -95,3 +97,32 @@ def test_drive_letter_not_special_on_non_windows(self, url: str) -> None: result = parse_store_url(url) # urlparse interprets the drive letter as a scheme assert result.scheme == "c" + + +class TestNormalizeByteRangeIndex: + """Tests for _normalize_byte_range_index.""" + + def test_suffix_larger_than_data_returns_all_bytes(self) -> None: + """Regression: SuffixByteRequest with suffix > len(data) must not produce a + negative start index that causes numpy to return fewer bytes than available.""" + prototype = default_buffer_prototype() + data = prototype.buffer.from_bytes(b"hello") # 5 bytes + byte_range = SuffixByteRequest(suffix=7) + start, stop = _normalize_byte_range_index(data, byte_range) + assert start == 0, f"start should be 0 (clamped), got {start}" + result = data[start:stop] + assert len(result) == 5, f"expected all 5 bytes, got {len(result)}" + + def test_suffix_exact_length(self) -> None: + """SuffixByteRequest with suffix == len(data) returns all bytes.""" + prototype = default_buffer_prototype() + data = prototype.buffer.from_bytes(b"hello") + start, _stop = _normalize_byte_range_index(data, SuffixByteRequest(suffix=5)) + assert start == 0 + + def test_suffix_shorter_than_data(self) -> None: + """SuffixByteRequest with suffix < len(data) returns the last n bytes.""" + prototype = default_buffer_prototype() + data = prototype.buffer.from_bytes(b"hello") + start, _stop = _normalize_byte_range_index(data, SuffixByteRequest(suffix=3)) + assert start == 2 diff --git a/tests/test_store/test_zip.py b/tests/test_store/test_zip.py index be51bcedcb..ed69114b51 100644 --- a/tests/test_store/test_zip.py +++ b/tests/test_store/test_zip.py @@ -8,11 +8,20 @@ import numpy as np import pytest +from hypothesis import settings +from hypothesis.stateful import ( + RuleBasedStateMachine, + initialize, + precondition, + rule, + run_state_machine_as_test, +) import zarr from zarr import create_array from zarr.core.buffer import Buffer, cpu, default_buffer_prototype from zarr.core.group import Group +from zarr.core.sync import sync from zarr.storage import ZipStore from zarr.testing.store import StoreTests @@ -177,3 +186,66 @@ async def test_move(self, tmp_path: Path) -> None: assert destination.exists() assert not origin.exists() assert np.array_equal(array[...], np.arange(10)) + + +class ZipStoreLifecycleMachine(RuleBasedStateMachine): + """Drive a ZipStore through construct / open / write / close transitions. + + Invariant under test: a constructed ZipStore can always be closed without + raising, regardless of whether it was ever opened or did any I/O. This is a + property-based generalization of the former example-based regression tests + for ZipStore.close() being called on a never-opened store (which raised + AttributeError because ``_lock`` is created lazily in ``_sync_open``). + """ + + def __init__(self, tmp_path: Path) -> None: + super().__init__() + self._tmp_path = tmp_path + self._counter = 0 + self.store: ZipStore | None = None + self._opened = False + + @initialize() + def start(self) -> None: + self.store = None + self._opened = False + + @precondition(lambda self: self.store is None) + @rule() + def construct(self) -> None: + # Fresh path each time so mode="w" never clobbers a closed archive. + self._counter += 1 + self.store = ZipStore(self._tmp_path / f"s{self._counter}.zip", mode="w") + self._opened = False + + @precondition(lambda self: self.store is not None and not self._opened) + @rule() + def open(self) -> None: + assert self.store is not None + self.store._sync_open() + self._opened = True + + @precondition(lambda self: self.store is not None and not self._opened) + @rule() + def write(self) -> None: + assert self.store is not None + # store.set auto-opens the store. + sync(self.store.set("a", cpu.Buffer.from_bytes(b"hi"))) + self._opened = True + + @precondition(lambda self: self.store is not None) + @rule() + def close(self) -> None: + assert self.store is not None + # The property under test: close() must never raise, even with no + # prior open or I/O. + self.store.close() + self.store = None + self._opened = False + + +def test_zipstore_close_lifecycle(tmp_path: Path) -> None: + run_state_machine_as_test( # type: ignore[no-untyped-call] + lambda: ZipStoreLifecycleMachine(tmp_path), + settings=settings(max_examples=50, deadline=None), + ) diff --git a/uv.lock b/uv.lock index dedaf964fa..adc71bae62 100644 --- a/uv.lock +++ b/uv.lock @@ -339,14 +339,14 @@ wheels = [ [[package]] name = "bleach" -version = "6.3.0" +version = "6.4.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "webencodings" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/07/18/3c8523962314be6bf4c8989c79ad9531c825210dd13a8669f6b84336e8bd/bleach-6.3.0.tar.gz", hash = "sha256:6f3b91b1c0a02bb9a78b5a454c92506aa0fdf197e1d5e114d2e00c6f64306d22", size = 203533, upload-time = "2025-10-27T17:57:39.211Z" } +sdist = { url = "https://files.pythonhosted.org/packages/48/3c/e12ac860709702bd5ebeb9b56a4fe334f1001246ee1b8f2b7ee28912df7d/bleach-6.4.0.tar.gz", hash = "sha256:4202482733d85cedd04e59fcb2f89f4e4c7c385a78d3c3c23c30446843a37452", size = 204857, upload-time = "2026-06-05T13:01:13.734Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/cd/3a/577b549de0cc09d95f11087ee63c739bba856cd3952697eec4c4bb91350a/bleach-6.3.0-py3-none-any.whl", hash = "sha256:fe10ec77c93ddf3d13a73b035abaac7a9f5e436513864ccdad516693213c65d6", size = 164437, upload-time = "2025-10-27T17:57:37.538Z" }, + { url = "https://files.pythonhosted.org/packages/58/9d/40b6267367182187139a4000b82a3b287d84d745bccd808e75d916920e9d/bleach-6.4.0-py3-none-any.whl", hash = "sha256:4b6b6a54fff2e69a3dde9d21cc6301220bee3c3cb792187d11403fd795031081", size = 165109, upload-time = "2026-06-05T13:01:12.504Z" }, ] [package.optional-dependencies] @@ -2259,52 +2259,52 @@ wheels = [ [[package]] name = "obstore" -version = "0.10.0" +version = "0.10.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "typing-extensions", marker = "python_full_version < '3.13'" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/78/e3/34852e48d5acedc1dbfa8bfb1ddcee448c7bb1dee7cefca659dce5ee10b4/obstore-0.10.0.tar.gz", hash = "sha256:b581d2f78b521c7c72862721384c109b6764c57222b7bf06dc83626a7c4a6945", size = 126379, upload-time = "2026-06-01T20:56:59.204Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/36/47/46357284f39465aeaec6b49f7741a26aa6c0258be6d0f968a3c5c1a93401/obstore-0.10.0-cp311-abi3-macosx_10_12_x86_64.whl", hash = "sha256:03b4e07d97e5271da136942dae71febd5a42c455aeee42a40892a6977171e9fa", size = 4090761, upload-time = "2026-06-01T20:55:33.434Z" }, - { url = "https://files.pythonhosted.org/packages/48/40/dc3d2acc664af08c4b1062cd677c36a1f3a7face6fa4e966562ba58d92cc/obstore-0.10.0-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:e45deb76f3f1a54cf730c43e917af72d3f1a46e4c502190a9111d2dabec3d71e", size = 3871052, upload-time = "2026-06-01T20:55:35.001Z" }, - { url = "https://files.pythonhosted.org/packages/f9/22/2a4eec925df64b46839b37222fb46fa32faa6790173688b6bbc1a5fd303c/obstore-0.10.0-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:d90894aafb8baf02d4d7d4e6debbbd43b26890bfbe0bcad7c6d75b41b019ed6b", size = 4024833, upload-time = "2026-06-01T20:55:36.527Z" }, - { url = "https://files.pythonhosted.org/packages/27/a4/638775f75c7df43596c44684f722db4f130d9a47bc95047c81dcffd8461d/obstore-0.10.0-cp311-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e8a443c729dfb6324591664aa4cca5394d89924014d585ea4ac7e2d448ad3a8e", size = 4122218, upload-time = "2026-06-01T20:55:38.02Z" }, - { url = "https://files.pythonhosted.org/packages/0a/31/e7ad24144996cb1c84414f5dfca83ce149ccfca621399a4fafcabdf18635/obstore-0.10.0-cp311-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:52ed02949c1ea3dd445c963b246e95801ccd54b5d0d24050ad7ae9570e2c1195", size = 4410778, upload-time = "2026-06-01T20:55:39.468Z" }, - { url = "https://files.pythonhosted.org/packages/8d/37/d5710ed7aa32082933c89d3864197e9b413dbabd954ebd5504b976b998a6/obstore-0.10.0-cp311-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e360ce37f4b5f7b6e3eaf6517b8bbb481379eb0773a16bb2971c02b4363ca787", size = 4291676, upload-time = "2026-06-01T20:55:41.184Z" }, - { url = "https://files.pythonhosted.org/packages/b2/5a/11b2947870663c4d804a5467462dcfc66a13505cd360e97b2d6ccbc93686/obstore-0.10.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:70accdf1fc624559c810bed68041f599a218f3dc4fb3afdeed50f7151e240ff8", size = 4210395, upload-time = "2026-06-01T20:55:42.923Z" }, - { url = "https://files.pythonhosted.org/packages/63/2d/363d7d7f89378753e5491e5d1189b0c728d68ea39d2a02baa6644fb8c4ca/obstore-0.10.0-cp311-abi3-manylinux_2_24_aarch64.whl", hash = "sha256:4fc646ad2c2ceaa3dbe07536eb207e1e868e4c701c89f71dd35678e1ee957283", size = 4101733, upload-time = "2026-06-01T20:55:44.561Z" }, - { url = "https://files.pythonhosted.org/packages/51/fd/7ceffe6b89feb6169f81d2b1b5d06eb53998422346c79365424dd06978b1/obstore-0.10.0-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:72610b6e7b3da608762b6a8a37989e0559c813ac115b9376a176810eb0173bd8", size = 4285774, upload-time = "2026-06-01T20:55:46.661Z" }, - { url = "https://files.pythonhosted.org/packages/ad/d9/4c534074516645236157135127e3379184f6f8e2622ab93485946ce2df42/obstore-0.10.0-cp311-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:38a10746b540aab3f898422d87fa7112dab35a824dfa912b64480eb37b55c755", size = 4258354, upload-time = "2026-06-01T20:55:48.435Z" }, - { url = "https://files.pythonhosted.org/packages/ff/dc/3f40b59c19054d8dc0c2699cdc72348bfd56edec78008c5e6bebe8aed55e/obstore-0.10.0-cp311-abi3-musllinux_1_2_i686.whl", hash = "sha256:e62afd200b2e5bc93cff75e6f930877f4203a6a2f0e13252c6ce7dba78d77c9c", size = 4247831, upload-time = "2026-06-01T20:55:50.005Z" }, - { url = "https://files.pythonhosted.org/packages/a0/08/28d6917d454ae587ef10b10b5d591c22ceb42bcb42c777463fe04ff3efb7/obstore-0.10.0-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:25e1cad14a7723e48e358c5635dc199ec1d0ddb0f6724ff58d92057ed93ad81b", size = 4429790, upload-time = "2026-06-01T20:55:51.765Z" }, - { url = "https://files.pythonhosted.org/packages/f1/de/5c683e75a7504ee73d51e34ac4b98c1eb2ab7679d53d292009028e05338b/obstore-0.10.0-cp311-abi3-win_amd64.whl", hash = "sha256:7e14e1ef6bb63730d6aec78499b0aa48dde160d1ad8fdd1e5553c930e7664107", size = 4166690, upload-time = "2026-06-01T20:55:53.314Z" }, - { url = "https://files.pythonhosted.org/packages/de/c1/782617a10203916d193ab2bc6efe363f8f42b5bc6fbde0e0cc3bb51a58dc/obstore-0.10.0-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:f343227c37c6f217aaf56e82f58335fa5f34b70a727f4e5e718becc9c613060a", size = 4073158, upload-time = "2026-06-01T20:55:54.886Z" }, - { url = "https://files.pythonhosted.org/packages/15/6b/31d0a6801a05fb032060bb38c94acb5348ac4ceb0a3a55d0b100439005b4/obstore-0.10.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:5b071781d2ef95653f78aef588bd054480381d679ae57c4f5d2553b366de1a05", size = 3862303, upload-time = "2026-06-01T20:55:56.509Z" }, - { url = "https://files.pythonhosted.org/packages/b6/0c/70c1ba253ab9a7dbdb73996381b73ff9a121b032e9d04125709378f44b67/obstore-0.10.0-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:71657a256f938715f5a4d3fe84c5141b79572dbe2436b90d0020ecb9cfe6548b", size = 4021507, upload-time = "2026-06-01T20:55:58.331Z" }, - { url = "https://files.pythonhosted.org/packages/81/d5/5495cb6056fac0f9394518c8eb85340ba625245a14de4384385966efeaba/obstore-0.10.0-cp313-cp313t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1a96beb5628f3303d8fdfd003b1957f4c2a38f0f224a47b6e488ab0b4eb23edb", size = 4113513, upload-time = "2026-06-01T20:55:59.991Z" }, - { url = "https://files.pythonhosted.org/packages/eb/80/90b292e9989ea387b60f66077cafcb6cc637f3a3ec109ba2c3fd7849b19d/obstore-0.10.0-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:2509b29b89e5a00480f165d5351132da2efeeca6a57d29f018c4e0534a5a5dc0", size = 4400976, upload-time = "2026-06-01T20:56:01.512Z" }, - { url = "https://files.pythonhosted.org/packages/3d/33/7ae84485bccb672f4abab5a4362e2e1554d3df83f07f76e2d68bee2a0657/obstore-0.10.0-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6b53a9c3e3afe24ea53e554c1d5626fdd456b2e7960091fce4a77af5b7a0ffb7", size = 4298268, upload-time = "2026-06-01T20:56:03.197Z" }, - { url = "https://files.pythonhosted.org/packages/cd/f1/9934e3ae0869085449d87c54e41cef137c8ea8527a87d60f9934f822bc08/obstore-0.10.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2d07b04a16c6a586444de1997f2121a332db05aac7953fbcb3e8fb28140306a8", size = 4208768, upload-time = "2026-06-01T20:56:05.314Z" }, - { url = "https://files.pythonhosted.org/packages/69/76/50cd42e12d8fecef6637a00a857b39c4aade6fa8a100634fe8a000fe56ca/obstore-0.10.0-cp313-cp313t-manylinux_2_24_aarch64.whl", hash = "sha256:012ebb24ba0c47e54840fa02bef780ce4f4d73a3c9685d9addd53222f1b375b2", size = 4100278, upload-time = "2026-06-01T20:56:07.187Z" }, - { url = "https://files.pythonhosted.org/packages/e5/be/378d4ea417771bfffd49222711f70b426f475475881aa6adddf67e62be18/obstore-0.10.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:defe889c3baf0781eb83e0335e89fb1331723bebd6a015dc4eecb99794c70210", size = 4285696, upload-time = "2026-06-01T20:56:08.898Z" }, - { url = "https://files.pythonhosted.org/packages/1f/26/40a9a419de6ccc9e828336fab4a89a5d90c1e01523cce781600dfa970b6b/obstore-0.10.0-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:b4b1fcf083474384565c5fd06523894419280fca4aeb23a7f3f88add8db2d824", size = 4255725, upload-time = "2026-06-01T20:56:10.609Z" }, - { url = "https://files.pythonhosted.org/packages/6c/db/038ae746b5f8ca2677fa95418234edfa22ebdeff77a55fa349abcb6a6823/obstore-0.10.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:c2c4974845b1192ec50959ad8579e01699b6ae8c64adb6433addcfbf35263901", size = 4242564, upload-time = "2026-06-01T20:56:12.127Z" }, - { url = "https://files.pythonhosted.org/packages/95/a3/dad7562624b89ed8b8c06241eea9340812decb4bbf1e4f744e9fbf652461/obstore-0.10.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:315fe386cc631247175c2cc64dfa77f7ec4746248d0a03a7126d9b6a063a1a1d", size = 4429578, upload-time = "2026-06-01T20:56:13.887Z" }, - { url = "https://files.pythonhosted.org/packages/96/d2/b2a232c428b7848862da2b31428694237bd4fe71166ace65d48bbfea6eb4/obstore-0.10.0-cp313-cp313t-win_amd64.whl", hash = "sha256:1fc3a232026f9b33affa4c6aa9d4a1765c08aa4e2d0aa8a6034801cf3f8a3cf6", size = 4160793, upload-time = "2026-06-01T20:56:15.629Z" }, - { url = "https://files.pythonhosted.org/packages/13/54/f3ed3d770279f59d87f476d36715f96ee097a8ab42f08c660c1e29d2a56e/obstore-0.10.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:fedac1464b8835063a5661da8dd33519d0d24988d2b256c92b5841c26b18ce28", size = 4073290, upload-time = "2026-06-01T20:56:17.221Z" }, - { url = "https://files.pythonhosted.org/packages/e4/3d/8a20aa21c615b25fc37b1396f4d643a3f0a951b5d73284b1822b443912b0/obstore-0.10.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2cccb3adef4e6e113fcd54a12c342c2a01fb8e470abfa1a5f9ce4ad2830514f1", size = 3862508, upload-time = "2026-06-01T20:56:19.011Z" }, - { url = "https://files.pythonhosted.org/packages/63/73/e1b00867e462c168de19a518688463f26a3560149373ab9a5b00a62770b7/obstore-0.10.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:d0396e857e403848a366ba38ee0289a5ea206cff1e184d2ce7f63eeb2a24da20", size = 4021738, upload-time = "2026-06-01T20:56:20.485Z" }, - { url = "https://files.pythonhosted.org/packages/f6/d9/23438e668d7eeeb7f30de4bece9467bdf9c2df25a849c10aec03edca4fc6/obstore-0.10.0-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:3911b95eac66328606cfa0c5545592ef18c04c242f5c5631c18ef4c3ce456009", size = 4113768, upload-time = "2026-06-01T20:56:22.116Z" }, - { url = "https://files.pythonhosted.org/packages/00/9c/e501563733f212f83ea3f2661f67c25dfd5cfd46bc9a979237662910b424/obstore-0.10.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5f86aa14f02e35db95b5331dda4387761366fefd52ec0856c9eed07bf92c681d", size = 4401455, upload-time = "2026-06-01T20:56:23.742Z" }, - { url = "https://files.pythonhosted.org/packages/35/12/2d1b356fc58e0868590f402067bcfe58d8de7501109f50924a2d691ae6e3/obstore-0.10.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b29f2477551114f1d3e01d9729fc54f88b6d6cd83fdc6e40d36eb4781ad03cd4", size = 4297930, upload-time = "2026-06-01T20:56:25.365Z" }, - { url = "https://files.pythonhosted.org/packages/94/5b/4deaf8795054977204925234ad9504b1fb942125bfeea11f6d11c7c04889/obstore-0.10.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8d2290638739df5c4c9e766e716fb536eb0e591350302ba492ff2dac7b03e913", size = 4209433, upload-time = "2026-06-01T20:56:27.151Z" }, - { url = "https://files.pythonhosted.org/packages/5b/fc/b80e7aa1119b46d7f232499ffda8c6e7db676405b421d566f856bd8034a6/obstore-0.10.0-cp314-cp314t-manylinux_2_24_aarch64.whl", hash = "sha256:0d6b1cf7af217dd6091e097e240347a64709a11eb22a89adcc85dda7462d2b4b", size = 4101258, upload-time = "2026-06-01T20:56:28.616Z" }, - { url = "https://files.pythonhosted.org/packages/56/7b/1518ad6d7ad39904378395c47fee2726d9d4518053089e7bd23406c827c3/obstore-0.10.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:c338a9fcbb18aebf5aab654fbb0731feaa27b80a0c1153aba0aa93188e89a6de", size = 4286477, upload-time = "2026-06-01T20:56:30.247Z" }, - { url = "https://files.pythonhosted.org/packages/25/27/035ac9f52e84ca7cf17554e11d522d6c213f56e656aff934ac6f1d4e7122/obstore-0.10.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:089d64f61649ded2b4f997ec94a4b659f594e6632f65c1c88622ad21c73f6089", size = 4255950, upload-time = "2026-06-01T20:56:31.883Z" }, - { url = "https://files.pythonhosted.org/packages/11/2c/5d9a5b2d1674db851bb8c9b886cf6a4f0a1d65adef113bd469bd7bae955d/obstore-0.10.0-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:d12bc7c84cb923c65055d83705870e4fd7a233aba9db91a26aebcf2327614429", size = 4243073, upload-time = "2026-06-01T20:56:33.428Z" }, - { url = "https://files.pythonhosted.org/packages/27/38/2899c989997d8576c0a37160721fcc094155b01eca4eff0ef5faf3fdf434/obstore-0.10.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:f4e69ce8e579227f6416c4e547c485a9ff84425ec92fd95107f2ad0cfb02f1cc", size = 4430865, upload-time = "2026-06-01T20:56:35.242Z" }, - { url = "https://files.pythonhosted.org/packages/ab/ed/ddf7aa9f6fbaef49764af51c0aead36770afcb1cb43e05e5658c5bba404a/obstore-0.10.0-cp314-cp314t-win_amd64.whl", hash = "sha256:9a64ededa74677649358bb35a1f07d912c77c51f732b77c531383ef964e29cdd", size = 4161321, upload-time = "2026-06-01T20:56:36.801Z" }, +sdist = { url = "https://files.pythonhosted.org/packages/f1/b7/516498f128eeac220dd54df61fd8c4db88adb7675129ab5352f2706899a3/obstore-0.10.1.tar.gz", hash = "sha256:b193a53101bda703f887f1c0733cde7324ba6f9c80f0a81bdae5df8cb25c26f4", size = 126551, upload-time = "2026-06-09T20:29:33.848Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/42/23/532c9094def8ed33495d555749a21b6eac4c31c34a95e7154d4866e25666/obstore-0.10.1-cp311-abi3-macosx_10_12_x86_64.whl", hash = "sha256:e5b009a5c257e9811b8d22bad2f090f8cdf24dca6afa1bafab88cb0ff5140317", size = 4092339, upload-time = "2026-06-09T19:51:43.938Z" }, + { url = "https://files.pythonhosted.org/packages/6d/3c/947a40ef9d64575a261fb3c0fd0c7e8ad4f160b4c6d4ee5c671705d92d5e/obstore-0.10.1-cp311-abi3-macosx_11_0_arm64.whl", hash = "sha256:c949aa4d69c5a796f7daefa9bce2efcf5bc29a21399915e47efcbb6d18787f80", size = 3873610, upload-time = "2026-06-09T19:51:45.793Z" }, + { url = "https://files.pythonhosted.org/packages/53/15/1e8a507ae86c356e923f17ce0cfc3b7e2fdd3417b439c343f9ad09a3f452/obstore-0.10.1-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:beb2e6f5c2c633add1a80182c223c862bc523d9c7c55b793423851831ef8a9ac", size = 4028148, upload-time = "2026-06-09T19:51:47.265Z" }, + { url = "https://files.pythonhosted.org/packages/9e/7f/6d46085a65be661dbf10243de257d7d2705c5629af9279a3e7d404e8890d/obstore-0.10.1-cp311-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f195d3c1258406976848246cfb0490790ec0a22bd0560e548364afb846b4bce2", size = 4125215, upload-time = "2026-06-09T19:51:48.778Z" }, + { url = "https://files.pythonhosted.org/packages/b3/15/a681b578a104a28dc1098ac4f0b7c77b11f5f7a60a5d6a964b33889ade52/obstore-0.10.1-cp311-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5cbad4afb93c26e39e6c51c390b9b6fec9602e9af50c4c0552f10f883524d197", size = 4412793, upload-time = "2026-06-09T19:51:50.258Z" }, + { url = "https://files.pythonhosted.org/packages/be/89/a610cf57ad94698952aad88dccbb0b6f6256f1e563d317b9e1393c30c338/obstore-0.10.1-cp311-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6bbd313dd82bd66b054cc1567927d69b17caed2bdf110da558e46e0ee4ba4e41", size = 4293828, upload-time = "2026-06-09T19:51:51.62Z" }, + { url = "https://files.pythonhosted.org/packages/2d/7a/82568065a1c21f45ae35069268247948d659b6a4d1c0a6186aa97538a102/obstore-0.10.1-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ed4e795b9997f91041d2ad43b633099e0eb891337228d34e4c706ef8f0c5ae92", size = 4212724, upload-time = "2026-06-09T19:51:53.202Z" }, + { url = "https://files.pythonhosted.org/packages/03/7a/7581034bdf1c3e88df947eba1ba8512aaad71a96e95b4b355a40ff9febb6/obstore-0.10.1-cp311-abi3-manylinux_2_24_aarch64.whl", hash = "sha256:a3c35027a90ee1c97b82933907e5846c48d72bce714c7571f9fceac7a3c86551", size = 4103114, upload-time = "2026-06-09T19:51:54.776Z" }, + { url = "https://files.pythonhosted.org/packages/b4/ec/0448b41a9f111d2dfebfc4c7af8ad81f68c7ae3ccf0f61a181e652a73f1a/obstore-0.10.1-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:b2902e2c9e1ca193bff39530bb907fd38204e68037ffb694a50042118eeb7a0b", size = 4291239, upload-time = "2026-06-09T19:51:56.253Z" }, + { url = "https://files.pythonhosted.org/packages/43/d3/63dfe45c22b43d579d6ef75a7dc81122d55b1af8ee020a7a7c241d982c66/obstore-0.10.1-cp311-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:c16e29b975430690c72ec71be9e6a4fe63854ff25985e6a3a1682419b55898ac", size = 4263387, upload-time = "2026-06-09T19:51:57.755Z" }, + { url = "https://files.pythonhosted.org/packages/e1/bd/66433876ca18172144cbcd6ff2e011cb512a4696d426a1946585d3855887/obstore-0.10.1-cp311-abi3-musllinux_1_2_i686.whl", hash = "sha256:c51488d41646bfd75fbb67507bbc55d6f5623d5b12ce0506a260a7a1f3e792da", size = 4253238, upload-time = "2026-06-09T19:51:59.428Z" }, + { url = "https://files.pythonhosted.org/packages/65/a1/46d61c7b871d0824973c3616277a68dc8a97269898d50a4b023de66c6507/obstore-0.10.1-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:84deb458af8601eb1dd948d58b9760bed2f0e7f36c6e9bcd5a61425cb2683b2a", size = 4434050, upload-time = "2026-06-09T19:52:00.889Z" }, + { url = "https://files.pythonhosted.org/packages/3f/b6/287d34041e73f1c5620462ba2ad0beecd9ef40ed7c3dd6e3924933bfa5fd/obstore-0.10.1-cp311-abi3-win_amd64.whl", hash = "sha256:f1b6e994b719e294a2b2aeb74f2ae8e5a294453a47d8a9d6f3104a28ef7d8aa5", size = 4174095, upload-time = "2026-06-09T19:52:02.335Z" }, + { url = "https://files.pythonhosted.org/packages/34/cd/86a2acdd1d37db34bef79d45d9aaeab740df58ff69e03c58b2ba5f328340/obstore-0.10.1-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:04e5f13af678993997f03fbc210e5da3dd36dfd9898235e977dacafe0e3bebfc", size = 4073194, upload-time = "2026-06-09T19:52:03.814Z" }, + { url = "https://files.pythonhosted.org/packages/8f/b3/ee84dab5325dcb579e6687438286acbd6ac25b257434e185b90f615a8849/obstore-0.10.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:b9fc35c5642e3580497d9399e072ffb050b8e2fe8abf7d63b6dfdb62410071c8", size = 3864659, upload-time = "2026-06-09T19:52:05.523Z" }, + { url = "https://files.pythonhosted.org/packages/51/cb/db764c672e977c9f6fe9b16a16a93d24a22665bca28819a1b4795e0397ab/obstore-0.10.1-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5267922416b0e5c1092676ce386e5a0762011c3752eca92f58760ae5d01b5fba", size = 4023673, upload-time = "2026-06-09T19:52:06.995Z" }, + { url = "https://files.pythonhosted.org/packages/d6/68/249282efba38b21c070ebd4ac9ed5c958255c70c15d541935789b619f917/obstore-0.10.1-cp313-cp313t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:31e0c82595ef3ff89c2ee9713d5cd0edbb7f86b0f2e73916683c535ed568293c", size = 4116817, upload-time = "2026-06-09T19:52:08.523Z" }, + { url = "https://files.pythonhosted.org/packages/bb/49/3f9b88caf396d8ba6eda797bc04906cb498a9f24c382d74e598c4a46a4ab/obstore-0.10.1-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d7c7bda05975df4ef37c516a617c041955d0bb700864015dbfcd6be89ab87c71", size = 4405345, upload-time = "2026-06-09T19:52:10.165Z" }, + { url = "https://files.pythonhosted.org/packages/43/fe/ec6e09dfa16b48c5a5e6a268abfbe63cdb339f213ade7f210ac638bb2548/obstore-0.10.1-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0409cbc5ff7e6bc33b78562cac5ca78b528c856bce50c03285aeb1abcd879805", size = 4297996, upload-time = "2026-06-09T19:52:12.081Z" }, + { url = "https://files.pythonhosted.org/packages/8b/24/2982f1efedd71f4cb417e0f532e0372ee5504dcbdae79b6d80fa5e63caef/obstore-0.10.1-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:63bb830361b6d1c33aba41fb2466b9eb92c7ab84dcb061bfb96269c2b709e8c6", size = 4211926, upload-time = "2026-06-09T19:52:13.709Z" }, + { url = "https://files.pythonhosted.org/packages/e3/b5/169cf89cf67bb3750c9bea5d6d35424c964ae583ebc7f67614d7655acfb0/obstore-0.10.1-cp313-cp313t-manylinux_2_24_aarch64.whl", hash = "sha256:6dbbc0b3e672f4f822878361a07b9d3200871a460ef725e78bb68b063febb7a5", size = 4102832, upload-time = "2026-06-09T19:52:15.205Z" }, + { url = "https://files.pythonhosted.org/packages/ee/d1/d689516435a1e5e67ceea786325abfb43da10357f9ed114d8aa508f9066a/obstore-0.10.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:3afca514671fa3f989242ef2b1694b53748881e7b29055e34007b530e116a5b8", size = 4290991, upload-time = "2026-06-09T19:52:16.867Z" }, + { url = "https://files.pythonhosted.org/packages/90/e9/83cf0dd637d2754557767cf438460ddcbbab5892987dadb5c42bdb2ec0d2/obstore-0.10.1-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:cf240b93e0f7856e396df9f4fa417df961db9a16d9e98619ed3c275676bffea7", size = 4258992, upload-time = "2026-06-09T19:52:18.714Z" }, + { url = "https://files.pythonhosted.org/packages/67/ed/06ebe9875c80b05111f597bc954074031e4f207784ba5951248ddd97723d/obstore-0.10.1-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:2957cf29a1f6974e4d7d07e02ddc6b88994d010ae1f007237c945a3beb951728", size = 4244918, upload-time = "2026-06-09T19:52:20.321Z" }, + { url = "https://files.pythonhosted.org/packages/1d/c2/122c48a04f1a836f643378549fdc4d1bc3e905973d7d51d1aeb2f21c2017/obstore-0.10.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:730c7f0443aba5d0285245db65d6cf59bf87f3bf6fee8f99741a2e9254fb66a1", size = 4431686, upload-time = "2026-06-09T19:52:21.976Z" }, + { url = "https://files.pythonhosted.org/packages/72/09/25a8adf373b2b8824672b7a68211c6fdca8e950d815f3bc6df69a41abbb2/obstore-0.10.1-cp313-cp313t-win_amd64.whl", hash = "sha256:0440037e51f7e20224d84eb79bb49a47356916c5fc7e603dd5607d75997b73f8", size = 4165763, upload-time = "2026-06-09T19:52:23.531Z" }, + { url = "https://files.pythonhosted.org/packages/4c/d8/7c78f14d12472328c2fbf287405150bd98ff6111c465a0b9a0b7f24cb4be/obstore-0.10.1-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:34c7d76aa33bcac0e4d65d5760527f8c03be83b7584204dac142417a4c9703bf", size = 4073261, upload-time = "2026-06-09T19:52:25.203Z" }, + { url = "https://files.pythonhosted.org/packages/ef/15/84a1b3c4494ad7f7605a884e792a17fa4545f4e186e428ca84b58794d481/obstore-0.10.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:8e9cfbcaf4afe00aaceef277bf9ea0604eda4cacd3511f459aaf48ac2e118392", size = 3864599, upload-time = "2026-06-09T19:52:26.643Z" }, + { url = "https://files.pythonhosted.org/packages/5e/a8/8332ab8076abaa086d8c8d17ff6f8e571af1725ee17a5b2b80888f297c68/obstore-0.10.1-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:282c9f85c56084dc3377818b77b7738301a1152bd6a49b0a36e06dc13d3cf7a9", size = 4023493, upload-time = "2026-06-09T19:52:28.327Z" }, + { url = "https://files.pythonhosted.org/packages/e4/fe/a87aa674f6cde2f7c0924d225ae9092bc0ccc8148810eaa0d2807204c367/obstore-0.10.1-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ce13fd4693e5a5a52d5160a29863d8fa2aff26613e371c7041c78d3f1c1f14dc", size = 4116999, upload-time = "2026-06-09T20:28:47.361Z" }, + { url = "https://files.pythonhosted.org/packages/44/c5/2979d915c409d5dc24a01d27b0cfa81db06a7da2ea38bb54b19c0888b922/obstore-0.10.1-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:4565c293dec7f234ed1bfee8c42d10fac9747f4de4d8d4401a94f442d3f2ff82", size = 4405326, upload-time = "2026-06-09T20:28:49.478Z" }, + { url = "https://files.pythonhosted.org/packages/a8/b5/b994342548f835bb63218369055621ac3d29eb8576dcf98ed123ea5ece92/obstore-0.10.1-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3bcf96ff1c52637602e705fbf616edbbb3109fe2c32ed08af718895d7800dc33", size = 4297910, upload-time = "2026-06-09T20:28:52.346Z" }, + { url = "https://files.pythonhosted.org/packages/6b/1c/34dacaf6bbda9df81ec57ae477da1b6273968f6a37c395068e531b4696e5/obstore-0.10.1-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:912d4b94f2949c722e4c6b9da7e99438aef91e30b01fd671abaf339e7b8b8c8d", size = 4212175, upload-time = "2026-06-09T20:28:54.777Z" }, + { url = "https://files.pythonhosted.org/packages/cb/b5/c25986eea6d043d199f972d5315a990984be50f8118c83fe64ceec443bb6/obstore-0.10.1-cp314-cp314t-manylinux_2_24_aarch64.whl", hash = "sha256:8df647f6821ae55c5aacd4c449a0e38ad08d0341bd693deca6294d30140885f7", size = 4103176, upload-time = "2026-06-09T20:28:56.794Z" }, + { url = "https://files.pythonhosted.org/packages/5c/4f/d94cdb5d66914ed7825a6185cbb5f288894a4c746e0b7a0e11c319a1e00f/obstore-0.10.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:4f9bcf84db56d53e9cc720368f850f90989334047179eb8b44f39645290efca7", size = 4291332, upload-time = "2026-06-09T20:28:58.708Z" }, + { url = "https://files.pythonhosted.org/packages/11/42/94de2fc1ebdfef9b587961403e3b1bcde1fa7c6fe6f86bdd19519136c03b/obstore-0.10.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:59a3c5f98317c4a83cbed54772945314932d68f210f3f51a87f1955d73e31133", size = 4258949, upload-time = "2026-06-09T20:29:01.246Z" }, + { url = "https://files.pythonhosted.org/packages/b8/18/ce4fecee53b7ba8fd4c91180f3da2068e751b13620eabb03fea78a9a90e6/obstore-0.10.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:f3c43d431593276c620a6c2870eb607401c679748c57ad57268b44e921c460fb", size = 4244717, upload-time = "2026-06-09T20:29:03.105Z" }, + { url = "https://files.pythonhosted.org/packages/73/d4/d432e10a7a080224c37455714717e5be6cb2cc85a673363f839b6403eac9/obstore-0.10.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:344239c68ffd21723cd306b4535ccaa5a9986b3aa003e3fe29b6822b2cefa671", size = 4432256, upload-time = "2026-06-09T20:29:04.993Z" }, + { url = "https://files.pythonhosted.org/packages/1f/d0/0ebae9b02583e6e37c50ce198fd0829b5850aa55247a6b7f21225ac186d1/obstore-0.10.1-cp314-cp314t-win_amd64.whl", hash = "sha256:04c4c751ed360ae1faf4dbb2dd2f0ea98595735d4b6b3b36b5e009ceb4ea0e68", size = 4165922, upload-time = "2026-06-09T20:29:06.878Z" }, ] [[package]] @@ -3678,28 +3678,28 @@ wheels = [ [[package]] name = "uv" -version = "0.11.19" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/67/f0/6254502aebfdc0a9df6069269a126dd58252ac29d2d6cdf4777cea3e90b5/uv-0.11.19.tar.gz", hash = "sha256:f56f5bf853626a30423052d7ee00bf5cc940a08347d6ee7ede96862d084054a5", size = 4213580, upload-time = "2026-06-03T22:37:15.976Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/1a/73/be32c2f6ba30fa9d8b3baceb478107cc23722d4aaab87145a332e4985185/uv-0.11.19-py3-none-linux_armv6l.whl", hash = "sha256:c729f56ffef9b945053412c839695e8a0b13758aa15b7763e95a7dd539a6f522", size = 23620003, upload-time = "2026-06-03T22:37:53.017Z" }, - { url = "https://files.pythonhosted.org/packages/fd/ed/3aefe4a4ca4ac9204c6745670dbe12f4add69194d40f5abd1c7bd45ba9af/uv-0.11.19-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:a98495b9dd67287d8c1a0786f98cb037a50f0ee6c3d648572edaa7137aabc277", size = 23183211, upload-time = "2026-06-03T22:37:20.699Z" }, - { url = "https://files.pythonhosted.org/packages/5b/eb/5d1469f9e709d56066f292978711fbf1f805b7fb46f901d3c1f260fd9908/uv-0.11.19-py3-none-macosx_11_0_arm64.whl", hash = "sha256:7fdd881cd6d80782afcf8c1d446dd15a42985167fd812b763d38ba1e4a8d944d", size = 21754003, upload-time = "2026-06-03T22:37:05.027Z" }, - { url = "https://files.pythonhosted.org/packages/7b/93/109b5ee6678f54492f94fdef74149643eaa1f2f4716906a2a10816b31247/uv-0.11.19-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl", hash = "sha256:7222f45b5541551057bfc2e3021f113800704f665c119fdf3ea700c6c4859b21", size = 23518832, upload-time = "2026-06-03T22:37:28.794Z" }, - { url = "https://files.pythonhosted.org/packages/08/0c/8c59bbcf78e94ca9994256920efa99d1c4dc9d0b966eb62ebba075585a16/uv-0.11.19-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.musllinux_1_1_armv7l.whl", hash = "sha256:2e0e0b8ad59ec56f1440d6e4313b64a1d8119275dcec73d19eef33c43f99428c", size = 23163128, upload-time = "2026-06-03T22:37:23.226Z" }, - { url = "https://files.pythonhosted.org/packages/89/d6/69caf9e6f11c84b5fb92df190b46fbecb7dc6645ae891c6ed66d7aaaa310/uv-0.11.19-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f4aa17ffd719daf37b7a6265efd3ee4922a8ddaabaf0406d2b28c7e5ce2f20ff", size = 23164395, upload-time = "2026-06-03T22:37:18.11Z" }, - { url = "https://files.pythonhosted.org/packages/d6/83/0c2242b77c51ac33a0ddd8b06790429a0b8b9623974c9594ab2b0070ec47/uv-0.11.19-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:32d7988c0dfb6f90941f201c871a4478e96e4f2a32bdb2256d62a78ee20593fc", size = 24541708, upload-time = "2026-06-03T22:37:08.093Z" }, - { url = "https://files.pythonhosted.org/packages/54/10/b1404fc52c0eddc3655f57a8b76e79dcf8dd02568382272f17e2fa68c4bb/uv-0.11.19-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:2d663bacb97e2e8412d1c26eace28c7ebbde9d6f5d7d78760fafd114d693817f", size = 25575501, upload-time = "2026-06-03T22:37:47.526Z" }, - { url = "https://files.pythonhosted.org/packages/7c/17/4cda5994195ba9ce1f6971d40d5f2ceec58e2a79030d9052b3bf322557b1/uv-0.11.19-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:574f5dd4f31666661ea6386d3b91c5f0e8b84a8cae98ebba447c4674f2e6a4c7", size = 24827200, upload-time = "2026-06-03T22:37:34.039Z" }, - { url = "https://files.pythonhosted.org/packages/5a/74/2bd8b51e1d76210fd424ae55ec3f34ded5a10eeff3dd38aeb03c816a0af2/uv-0.11.19-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:731d9fab8db5d41590af64236d03f8069c8da665fd0f9493b85985f19c86cd90", size = 24872664, upload-time = "2026-06-03T22:37:11.301Z" }, - { url = "https://files.pythonhosted.org/packages/06/b1/44b0764f656bbdd0728118610a63f2feddd9cbe450f974d80c5bb56aad34/uv-0.11.19-py3-none-manylinux_2_28_aarch64.whl", hash = "sha256:301fd78309fc545c2cec2bfcc61a6bbdde876856c6d2041502737cf44085c178", size = 23617890, upload-time = "2026-06-03T22:37:44.796Z" }, - { url = "https://files.pythonhosted.org/packages/d2/25/312fa33cd4c34e7618f86cad0c9fdb312d8fef2e7fc61944c1a2f1bf1256/uv-0.11.19-py3-none-manylinux_2_31_riscv64.musllinux_1_1_riscv64.whl", hash = "sha256:62b0b35a51d3034ff30ecd0f381e9bbc20d5b335754f54b098da29424d551ceb", size = 24267220, upload-time = "2026-06-03T22:37:39.425Z" }, - { url = "https://files.pythonhosted.org/packages/8d/25/13856aeff9e14c98ee3e1ceae4d209301cbdeabde93abcd758433601dc82/uv-0.11.19-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:65e932720daed1af1f720a0ff5f9b33ee5f7ad97488dcceceb85154fc1323b82", size = 24376177, upload-time = "2026-06-03T22:37:50.276Z" }, - { url = "https://files.pythonhosted.org/packages/45/7d/590b3ab420e03504cf658d2981e1fcb4af60f3858d42da1d4d8740141dd9/uv-0.11.19-py3-none-musllinux_1_1_i686.whl", hash = "sha256:8f90b6687a480d154595aa619fb836a9a20d00ce37293db8099aad924f2b18f9", size = 23808336, upload-time = "2026-06-03T22:37:26.086Z" }, - { url = "https://files.pythonhosted.org/packages/9e/8e/40acebd4ea419c870930580623e8367e23d810a0ecb8cc2f44d852a27293/uv-0.11.19-py3-none-musllinux_1_1_x86_64.whl", hash = "sha256:28b0d612a766eb25756dbaa315433b726e93affa467d29a2682cc317547952ba", size = 25080747, upload-time = "2026-06-03T22:37:13.886Z" }, - { url = "https://files.pythonhosted.org/packages/9c/d3/4037b2acb2bb73b1a3ee47a1d23864ecc503f5840387afd29f621d4fd2ec/uv-0.11.19-py3-none-win32.whl", hash = "sha256:aa6a7e8d07b33ad22f4732848ebb1d9486503973c248d6e632c06ce4339fe347", size = 22459533, upload-time = "2026-06-03T22:37:36.741Z" }, - { url = "https://files.pythonhosted.org/packages/d4/43/f374fad7ad94e4a8c47cf09f00d803c76c6cc7f225668c41f4e2fb5de000/uv-0.11.19-py3-none-win_amd64.whl", hash = "sha256:480fc34a8d0967af6a90b3f99a6e5687cd5c6e29528de96bec04d6e305a59363", size = 25143888, upload-time = "2026-06-03T22:37:42.169Z" }, - { url = "https://files.pythonhosted.org/packages/18/98/d2db53ae036528b0a9407529ef175ee200b01f626c9c160978784c8af870/uv-0.11.19-py3-none-win_arm64.whl", hash = "sha256:50e4d4796ca1a6da359a4f723a0fea86640c381d3ff4fa759a41badd7cb52dee", size = 23601290, upload-time = "2026-06-03T22:37:31.393Z" }, +version = "0.11.20" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/80/09/c29c0b90bc9308cfa6f5d77ce9b38ce97852210fda17d79019c7bcf9c3a1/uv-0.11.20.tar.gz", hash = "sha256:a246f30931cbc93d0a39d0cfc75be045fddd45773a734ddf8afa869aabc46c63", size = 4237464, upload-time = "2026-06-10T17:20:05.905Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/6b/8a/25fc4d94ad3896e636466ee1fb03c4077f7a151c19cb25c1e6a731fa9cbf/uv-0.11.20-py3-none-linux_armv6l.whl", hash = "sha256:f867fd0807e39653fd101e16f2292ff488de14b5ffbb86a5678a87a27aa58ea0", size = 23713010, upload-time = "2026-06-10T17:19:32.058Z" }, + { url = "https://files.pythonhosted.org/packages/02/7d/bbaad5f0c616f7824149a4ac0271db14107b859b36bd75d16db1486c495f/uv-0.11.20-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:f3bacc52778775cef671867ddab744b50c4183bc3cd6419a5fb4eb01a02f9526", size = 22918378, upload-time = "2026-06-10T17:19:20.828Z" }, + { url = "https://files.pythonhosted.org/packages/39/3e/e3d39361b95c262b43ccfe260f41184da71c536a08f39d4ad59ab962e459/uv-0.11.20-py3-none-macosx_11_0_arm64.whl", hash = "sha256:c9f2062096e146b351fd5da3cd43e15a5c43bfa37166c509d884dab3dbb72f03", size = 21716975, upload-time = "2026-06-10T17:19:51.552Z" }, + { url = "https://files.pythonhosted.org/packages/24/30/9031204d7b592d1595322d7506944793728a74795a8c4a3c38bc4a8985f0/uv-0.11.20-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.musllinux_1_1_aarch64.whl", hash = "sha256:8fe143572a1f02d536c4e9c6994f0c89d9fa58ff6e5d91de55f61660658c694b", size = 23571826, upload-time = "2026-06-10T17:19:46.206Z" }, + { url = "https://files.pythonhosted.org/packages/98/c3/7f9f00c7a152e67d59ae5f25635d19ac4252929dd6ac454cdb1dee3119fa/uv-0.11.20-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.musllinux_1_1_armv7l.whl", hash = "sha256:3d00754be09a381030829526f7ff47c06d0e28ca90760c4439be48e71c1bf13a", size = 23249218, upload-time = "2026-06-10T17:19:26.603Z" }, + { url = "https://files.pythonhosted.org/packages/35/a9/1ce58670a89c25d4a8b84532207183b5a97b3623d9b9151afe641499fbc8/uv-0.11.20-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:edb59d33e602fc462b6ed8fde66d404445294de41ad4de31039f7a2c41153601", size = 23302149, upload-time = "2026-06-10T17:20:01.111Z" }, + { url = "https://files.pythonhosted.org/packages/61/cf/3a498a315364f906bd655a94fa6b5f74f5240dc90875d6ab67ef28c081ec/uv-0.11.20-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:14026b64ccbe0174e4fcf107f585f5d23f0f5b9f5b3e8b28394fe63fff3e60ff", size = 24652804, upload-time = "2026-06-10T17:19:16.692Z" }, + { url = "https://files.pythonhosted.org/packages/ff/8c/68e4a805e3b49d834e95b3838e53a623d2e0ad956bb44e681ece54b3308b/uv-0.11.20-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:73dcaf5543d1b24e4c6fa4c19af033ed015304171c132670ebe9ef01ddec3d17", size = 25660209, upload-time = "2026-06-10T17:20:08.156Z" }, + { url = "https://files.pythonhosted.org/packages/a6/de/e8e205a79b9a39454c5da5c2695e4911a42860a4404739e32802e599af0c/uv-0.11.20-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cac8ca5d187dc5040b1c22c83f6ed789195c3fc06b6a5a2b32c747c603f07820", size = 24866467, upload-time = "2026-06-10T17:19:37.981Z" }, + { url = "https://files.pythonhosted.org/packages/7e/38/f844d125db277d8ce0c921f0219078d292d0ee72d314d0c12f4cf510aa40/uv-0.11.20-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:61588768f04a24b0b4d87b43f03b4521ba66d5eec3ab1aca37cb59b1d52337db", size = 24957448, upload-time = "2026-06-10T17:19:40.979Z" }, + { url = "https://files.pythonhosted.org/packages/ab/7d/b0e28abfa41c424d2a3df83be2b00b2fbe3ad5795baf7361262c6d800a62/uv-0.11.20-py3-none-manylinux_2_28_aarch64.whl", hash = "sha256:2d307d47b1a0cf8f76aa69bde850be407bca9482c13cff066142e586a5c57e77", size = 23671087, upload-time = "2026-06-10T17:19:54.595Z" }, + { url = "https://files.pythonhosted.org/packages/80/ff/92bce88101ce61d708e888db6ab7f5ebf4ccd61f54d3316e7e48a4be56a5/uv-0.11.20-py3-none-manylinux_2_31_riscv64.musllinux_1_1_riscv64.whl", hash = "sha256:bb5839cbb68d7469925fe1599dacd16cefdb7698a7adeaf9c8e4d8a7cd4122bf", size = 24324677, upload-time = "2026-06-10T17:19:43.534Z" }, + { url = "https://files.pythonhosted.org/packages/0f/90/d308bd88c7a53cae93f7fe13ce5c621895b8fb94e37911660a3de7283b67/uv-0.11.20-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:efefbd491ba443b326fdd344d7efc89c1de8a006cab5bc639a4d5ce9d1dd1ab9", size = 24429959, upload-time = "2026-06-10T17:19:29.352Z" }, + { url = "https://files.pythonhosted.org/packages/e5/e6/b87c941b93b61dceaa11f4f8d02760de7aa7d1c58ea24a2298e7c5aafb4f/uv-0.11.20-py3-none-musllinux_1_1_i686.whl", hash = "sha256:daa41b97386699212b2266a80c178140c5396e3c444ff5553f68f79812334e41", size = 23880515, upload-time = "2026-06-10T17:19:23.733Z" }, + { url = "https://files.pythonhosted.org/packages/53/ab/11b07641f8387177889f4341a36318561208c18703837bc47163244aa11b/uv-0.11.20-py3-none-musllinux_1_1_x86_64.whl", hash = "sha256:37ca61eddb940d1c698fce46d63789adfda3033344dabab0e18e2958e1a69771", size = 25171603, upload-time = "2026-06-10T17:19:34.848Z" }, + { url = "https://files.pythonhosted.org/packages/b4/7d/62af57509c7007600ce11cd5222b2cb84d5cd8f80f427a499d1b44ef3601/uv-0.11.20-py3-none-win32.whl", hash = "sha256:32893ee9f94657fbf89e22638ac88850db4529d454f102bfa7ceea5fc9fe8d77", size = 22570328, upload-time = "2026-06-10T17:19:48.808Z" }, + { url = "https://files.pythonhosted.org/packages/66/35/c9ee48cdce11f5cdac9e2be41a5446bae7382cde18d2ce1050bcd81dc05a/uv-0.11.20-py3-none-win_amd64.whl", hash = "sha256:4836044213bb23a3be1f5550db340d3a19babe1dfc3ca1313544e8b614085ce9", size = 25228859, upload-time = "2026-06-10T17:19:57.824Z" }, + { url = "https://files.pythonhosted.org/packages/8b/8d/00a382c2f8f44b328cf98f734a3fcd72957698c30bed2392bd241b573384/uv-0.11.20-py3-none-win_arm64.whl", hash = "sha256:442ae26f47bf6e58b072e99dbfd6d5296ab90308574b2c02adff1dace051008d", size = 23664943, upload-time = "2026-06-10T17:20:03.774Z" }, ] [[package]] @@ -4114,7 +4114,7 @@ dev = [ { name = "tomlkit", specifier = "==0.15.0" }, { name = "towncrier", specifier = "==25.8.0" }, { name = "universal-pathlib" }, - { name = "uv", specifier = "==0.11.19" }, + { name = "uv", specifier = "==0.11.20" }, ] docs = [ { name = "astroid", specifier = "==4.1.2" }, @@ -4152,7 +4152,7 @@ remote-tests = [ { name = "requests", specifier = "==2.34.2" }, { name = "s3fs", specifier = ">=2023.10.0" }, { name = "tomlkit", specifier = "==0.15.0" }, - { name = "uv", specifier = "==0.11.19" }, + { name = "uv", specifier = "==0.11.20" }, ] test = [ { name = "coverage", specifier = "==7.14.1" }, @@ -4166,5 +4166,5 @@ test = [ { name = "pytest-cov", specifier = "==7.1.0" }, { name = "pytest-xdist", specifier = "==3.8.0" }, { name = "tomlkit", specifier = "==0.15.0" }, - { name = "uv", specifier = "==0.11.19" }, + { name = "uv", specifier = "==0.11.20" }, ]