The Wayback Machine - https://web.archive.org/web/20210828150040/https://github.com/mcmilk/zstdmt
Skip to content

/ zstdmt

Multithreading Library for Brotli, Lizard, LZ4, LZ5, Snappy and Zstandard

View license
86 stars 19 forks
Star
Notifications
master
Switch branches/tags
1 branch 9 tags
Code

Latest commit

@mcmilk
mcmilk Fix decompression of LZ4 files with multiple frames
939dcd0 Dec 19, 2020
Fix decompression of LZ4 files with multiple frames
939dcd0

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
.github/workflows
Change Github workflow test, move it mainly to Makefile
Jun 8, 2020
lib
Fix decompression of LZ4 files with multiple frames
Dec 19, 2020
man
Add manpage for snappy and bump version to v0.7
Jun 8, 2020
programs
Use 4MiB chunks for lizard, lz4 and lz5
Dec 6, 2020
CHANGES.md
Update version of zstdmt to 0.8
Oct 16, 2020
LICENSE
Add manpage for snappy and bump version to v0.7
Jun 8, 2020
README.md
Update Brotli to version 1.0.9 and some fixes in README
Oct 16, 2020
Multithreading Library for Brotli, Lizard, LZ4, LZ5, Snappy-c, LZFSE and Zstandard Description Generic skippable frame definition Brotli frame definition Snappy-c frame definition LZFSE frame definition Usage of the Testutils Usage of the Library

README.md

Multithreading Library for Brotli, Lizard, LZ4, LZ5, Snappy-c, LZFSE and Zstandard

Description

  • works with skippables frame id 0x184D2A50 (12 bytes per compressed frame)
  • brotli is supported the same way, it will encapsulate the real brotli stream within an 16 byte frame header

Generic skippable frame definition

size value description
4 bytes 0x184D2A50U magic for skippable frame
4 bytes 4 size of skippable frame
4 bytes compressed size size of the following frame (compressed data)

Brotli frame definition

  • the frame header for brotli is defined a bit different:
size value description
4 bytes 0x184D2A50U magic for skippable frame (like zstd)
4 bytes 8 size of skippable frame
4 bytes compressed size size of the following frame (compressed data)
2 bytes 0x5242U magic for brotli "BR"
2 bytes uncompressed size allocation hint for decompressor (64KB * this size)

Snappy-c frame definition

  • the frame header for Snappy-c is defined a bit different:
size value description
4 bytes 0x184D2A50U magic for skippable frame (like zstd)
4 bytes 8 size of skippable frame
4 bytes compressed size size of the following frame (compressed data)
2 bytes 0x5053U magic for Snappy-c "SP"
2 bytes uncompressed size allocation hint for decompressor (64KB * this size)

LZFSE frame definition

  • the frame header for Lzfse is defined a bit different:
size value description
4 bytes 0x184D2A50U magic for skippable frame (like zstd)
4 bytes 8 size of skippable frame
4 bytes compressed size size of the following frame (compressed data)
2 bytes 0x464CU magic for Lzfse "LF"
2 bytes uncompressed size allocation hint for decompressor (64KB * this size)

Usage of the Testutils

Usage of the Library

/TR 2020-10-15

About

Multithreading Library for Brotli, Lizard, LZ4, LZ5, Snappy and Zstandard

Topics

brotli snappy zstd lz4 zstandard lizard multithreading-library lz5 skippables-frame

Resources

Readme

License

View license

Releases 9

Version 0.8 Latest
Oct 16, 2020
+ 8 releases

Packages

No packages published

Contributors 4

Languages