Compressor "firn" improves ZSTD by 13%

themerone · 2 hours ago

This works by text substitution, I imagine that you could get similar or better gains using zstd custom dictionaries.

seccode · 2 hours ago

This is a direct comparison to zstd with a dictionary actually

seccode · 2 hours ago

Methodology for comparison: train zstd dictionary on enwik9. Then build my dictionary as most common words in enwik9. Mine does 13% better because of the way I discovered how you can generate dictionary replacement symbols

reply