Skip to content

Updated LZ4 plugin - #394

Merged
t20100 merged 5 commits into
silx-kit:mainfrom
t20100:update-lz4-upstream
Jun 23, 2026
Merged

Updated LZ4 plugin#394
t20100 merged 5 commits into
silx-kit:mainfrom
t20100:update-lz4-upstream

Conversation

@t20100

@t20100 t20100 commented Jun 23, 2026

Copy link
Copy Markdown
Member

This PR changes upstream project for the LZ4 from the initial repository (https://github.com/nexusformat/HDF5-External-Filter-Plugins) to the version hosted by the hdfgroup organisation (https://github.com/HDFGroup/hdf5_plugins).

The reason is that the former project is stalled while the later is actively maintained.
As raised in #326 this has consequences on performances.

Running the sample code from #326 (comment) shows performance improvement:

hdf5plugin 6.0.0: lz4 compression time 0.270s, lz4 decompression_time: 1.577s
hdf5plugin 7.0.0: lz4 compression time 0.252s, lz4 decompression_time: 0.365s

closes #326

@t20100 t20100 added this to the 7.0.0 milestone Jun 23, 2026
@t20100
t20100 requested a review from payno June 23, 2026 07:20
@t20100
t20100 marked this pull request as draft June 23, 2026 07:23
@t20100
t20100 removed the request for review from payno June 23, 2026 07:24
@t20100
t20100 force-pushed the update-lz4-upstream branch from a3e1acf to 0cdfb8d Compare June 23, 2026 07:35
@t20100
t20100 marked this pull request as ready for review June 23, 2026 07:35
@t20100
t20100 force-pushed the update-lz4-upstream branch from 0cdfb8d to 765bf7f Compare June 23, 2026 07:37
@t20100

t20100 commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

Only copying a small part of hdf5_plugins repository rather than using git subtree since a small part of it is used for now. It was already done this way for the bzip2 filter (rather than integrating the whole pytables repo here).

In the future, we should consider relying on the hdf group's hdf5_plugins repository.

@t20100
t20100 merged commit a1acc8f into silx-kit:main Jun 23, 2026
4 checks passed
@t20100
t20100 deleted the update-lz4-upstream branch June 23, 2026 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance regression for lz4 decompression after version 4.0.1

2 participants