@@ -81,7 +81,8 @@ writer, validation, scan, and C interface engines.
8181- Structural, semantic, and optional full-content validation tiers.
8282- Resource limits for untrusted IPC metadata and buffers.
8383- Apache Arrow gold-corpus tests, external IPC oracle tests, C interface oracle
84- tests, and a JuliaC ` --trim=safe ` compile gate.
84+ tests, deterministic differential and bounded mutation fuzzing, and a JuliaC
85+ ` --trim=safe ` compile gate.
8586
8687### Changed
8788
@@ -91,16 +92,73 @@ writer, validation, scan, and C interface engines.
9192- A table read from Arrow retains compatible schema details when it is written
9293 again, including temporal units, dictionary encoding and category order,
9394 list widths, composite descriptors, nullability, and ordered duplicate
94- metadata. Fresh heterogeneous Julia Union columns can be synthesized, but a
95- retained Union still fails clearly after facade materialization discards its
96- original routing. Nested Dictionaries fail clearly after their pool data is
97- lost.
95+ metadata. Fresh heterogeneous Julia Union columns can be synthesized.
96+ Registered ArrowTypes.jl public-domain values can also reconstruct a retained Union
97+ from their writer-side type evidence, including dense or sparse mode and type
98+ IDs. An unregistered retained Union still fails clearly after facade
99+ materialization discards its original routing.
100+ Nested Dictionaries fail clearly after their pool data is lost.
98101- Writing is validated before bytes are published to the output sink.
102+ - Fresh ` Union{Missing, NamedTuple} ` columns use Struct parent validity while
103+ preserving each child's declared nullability and type.
99104- Custom values are lowered recursively through ` ArrowTypes.ArrowType ` and
100105 ` ArrowTypes.toarrow ` . Extension names and metadata are written, and reads use
101106 ` ArrowTypes.JuliaType ` , ` ArrowTypes.fromarrow ` , and
102107 ` ArrowTypes.fromarrowstruct ` to restore registered logical types. An unknown
103108 extension name warns and returns its storage value.
109+ - A fresh abstract column with no mapping of its own uses concrete subtype
110+ evidence across the complete column. Concrete subtype extensions are kept,
111+ and heterogeneous subtype evidence forms an explicit bounded Union instead
112+ of silently erasing subtype metadata.
113+ - Registered logical types whose storage is a Union preserve external child
114+ order, child labels, type IDs, dense or sparse mode, nested descriptor details,
115+ and outer-null routing on rewrite. Sparse children use canonical hidden
116+ placeholders outside their active rows. An abstract registered target accepts
117+ an extensionless concrete subtype or one with the retained parent identity; a
118+ different explicit identity fails closed. A logical storage Union that already
119+ uses ` Missing ` cannot also add an outer missing state because the two states
120+ have no distinguishable Arrow representation.
121+ - Hidden retained composite slots are built directly from their Field and
122+ logical length. Null-only fixed-size-list descendants do not allocate one
123+ Julia placeholder per hidden element, including partly missing fresh or
124+ retained columns and inactive sparse-Union rows. Sparse Union children also
125+ distinguish inactive physical slots from selected values, so inactive null
126+ storage does not weaken selected-value validation.
127+ - Recursive ArrowTypes.jl storage schemas, recursive value containers, and
128+ custom mapping nesting beyond 64 levels fail with ` ArgumentError ` . The
129+ removed ` maxdepth ` keyword does not make writer recursion unbounded.
130+ - Declared writer Unions support up to 32 branches. Runtime writer or storage
131+ inference accepts at most 8 distinct types across the complete column. This
132+ covers abstract ArrowTypes storage, abstract or ` Any ` dictionary values, and
133+ abstract retained ArrowTypes targets, and bounds per-type schema planning and
134+ compiler work.
135+ - ArrowTypes logical resolution uses exact fixed-size-list tuple signatures
136+ through arity 1024 and a compact tuple-family signature above that limit,
137+ preventing fixed-size-list sizes from causing proportional type allocation.
138+ Extension Struct signatures are exact through 1024 children only when child
139+ names are unique, contain no embedded NUL, already exist as Julia ` Symbol ` s,
140+ and pass the 4096-byte per-name and 64-KiB total limits. Otherwise labelled
141+ Structs remain unknown extensions and return ordered ` Pair ` storage. One
142+ bounded ArrowTypes.jl Tuple compatibility exception may intern only a
143+ complete canonical positional sequence ` "1" ` , ` "2" ` , …, ` string(N) ` through
144+ ` N = 1024 ` . Unknown extension labels return before that check; arbitrary or
145+ partly positional Struct names are not interned.
146+ Any writer-side ` ArrowType ` result that is a concrete tuple above that limit
147+ is rejected before the writer specializes on the oversized storage shape.
148+ This includes ArrowTypes.jl's default mapping for a tuple value.
149+ - Core keeps schema names as strings. The Tables.jl facade preflights per-name,
150+ novel-name-count, and novel-name-byte limits before it interns top-level
151+ column names. Unknown ArrowTypes extension labels do not create Julia
152+ symbols. Unsupported-label warnings are deduplicated by label and capped at
153+ 16 distinct labels plus one suppression notice per table materialization. A
154+ novel ` JuliaLang.Symbol ` IPC payload now raises ` ValidationError ` instead of
155+ being interned, so an input that an earlier Arrow.jl release read successfully
156+ can now fail.
157+ - Registered public-domain values can rebuild compatible retained binary, list,
158+ date-like, duration, wide-decimal, and interval descriptors. Retained widths,
159+ sizes, units, child fields, and sorted Map claims are checked before output.
160+ - Empty and typed all-missing registered columns use their concrete declared
161+ element types as schema evidence instead of bypassing retained-field checks.
104162- Scan filters over fields that contain registered ArrowTypes.jl logical types
105163 at any depth evaluate over the public materialized values. The mapping
106164 interface does not require storage lowering to preserve Julia comparison
0 commit comments