I have not experimented with bzip3 recently, but more than a year ago I have done many tests with it.
Initially I was extremely impressed with it, because in a lot of tests it succeeded to compress hard-to-compress files, like movies, and in many cases it demonstrated a much better compromise between speed and compression ratio than zstd, i.e. depending on the command parameters I could make it either compress better than zstd at similar compression/decompression speed, or compress/decompress faster at a similar compression ratio.
Alas, the initial extremely favorable conclusion was short-lived, because trying later bzip3 on other data files gave worse results than zstd.
So the final conclusion was that the performance of bzip3 was somewhat unpredictable, being highly data dependent. For some files it provided outstanding compression ratio or speed, but for others it was inferior.
The problem was that without doing a compression there was no way to guess whether a file would be among those preferred by bzip3 or by zstd or by xz.
So now I would use it only for a file for which I want maximum compression and which I would compress once and decompress many times, so I can afford a very long compression time, during which I would test multiple compression algorithms, including bzip3 and zstd, with multiple parameter choices, and I would eventually choose the one that offers the best compromise between compression ratio and decompression time, for that particular file.
It certainly is a competitive compression algorithm, but unless it has changed since I last tested it, you cannot guess for which files it would win the compression competition.
The benchmark is very rudimentary. It does not test different levels/settings apart from its own -b 256/512 (does it affect decompression?), it doesn't measure compression time and memory usage. It does not specify parallel vs single-threaded (it mentions parallel on the one decoding number but what about the others?).
The lrzip test is interesting but it omits for example zstd and doesn't even have (de-)compression timings.
A lot more numbers are needed to present a fair and informative comparison.
I don't want this to be a swipe against bzip3, I only want to point out the presented benchmarks could be a lot better.
There is a comparison with "zstd -19" on the Silesia corpus, showing better compression ratio for bzip3 (47.2 vs 53MB) while being ~5 times faster (and using only half the memory).
Even if the examples are highly cherry-picked, it is quite suprising to me that such pareto-dominance is possible at all.
having used zstd, it has terrible defaults, optimized for speed and low-memory. You need to change it's params (not just level and dict size) to get high performance.
probably somebody should use a coding agent to do auto-research to optimize params for each compression algo, while matching one fixed goal - time, memory or size
(2 years ago, 176 comments) https://news.ycombinator.com/item?id=42899713
(4 years ago, 104 comments) https://news.ycombinator.com/item?id=31324439
Initially I was extremely impressed with it, because in a lot of tests it succeeded to compress hard-to-compress files, like movies, and in many cases it demonstrated a much better compromise between speed and compression ratio than zstd, i.e. depending on the command parameters I could make it either compress better than zstd at similar compression/decompression speed, or compress/decompress faster at a similar compression ratio.
Alas, the initial extremely favorable conclusion was short-lived, because trying later bzip3 on other data files gave worse results than zstd.
So the final conclusion was that the performance of bzip3 was somewhat unpredictable, being highly data dependent. For some files it provided outstanding compression ratio or speed, but for others it was inferior.
The problem was that without doing a compression there was no way to guess whether a file would be among those preferred by bzip3 or by zstd or by xz.
So now I would use it only for a file for which I want maximum compression and which I would compress once and decompress many times, so I can afford a very long compression time, during which I would test multiple compression algorithms, including bzip3 and zstd, with multiple parameter choices, and I would eventually choose the one that offers the best compromise between compression ratio and decompression time, for that particular file.
It certainly is a competitive compression algorithm, but unless it has changed since I last tested it, you cannot guess for which files it would win the compression competition.
The lrzip test is interesting but it omits for example zstd and doesn't even have (de-)compression timings.
A lot more numbers are needed to present a fair and informative comparison.
I don't want this to be a swipe against bzip3, I only want to point out the presented benchmarks could be a lot better.
how is this an honest benchmark:
Even if the examples are highly cherry-picked, it is quite suprising to me that such pareto-dominance is possible at all.
probably somebody should use a coding agent to do auto-research to optimize params for each compression algo, while matching one fixed goal - time, memory or size
The claim “stronger than bzip2” is strange. What does it even mean?
Also, comparing parallel decompression benchmarks with bzip2 instead of pbzip2 seems unfair.