Decompression efficiency compared between tar.bz2 and tar.xz (LZMA)

Firefox Nightly
#1

This is an additional question to this announcement about Firefox switching from tar.bz2 to tar.xz (LZMA).

I congratulate on that decision in context of sustainable software development and climate change.

I would like to know if there is information or data available if the efficiency of the new decompression is better than with the old decompression. I know it doubles in speed. But what is about the used CPU cores? Does it doubles the speed on a single core level, too? Does the tar.xz (LZMA) decompression use 50% less of CPU time (and energy/CO2) compared to tar.bz2?

Thanks in advance
Christian

#2

Hello @buhtz, thanks for your interest in our project to switch Linux builds to xz compression.

When assessing the performance for decompression we used a 2-core system, since the large majority of our user base will have 2 or more cores available on their computer, and only about 1% of users have single core CPUs.
If you’d like to have a peek at what the user base looks like, the Firefox Public Data Report has some good bits of info on it.

As you might have seen, this comment lists the decompression assessment for 2 CPUs.

That being said, I ran a quick test on a single CPU VM and verified that it still does decompress 2x as fast with xz compared to bzip2.
If you’d like to run your own tests, here is a quick snippet to get you started:

# Note: You may need sudo or root user for apt
# Install bzip2, xz, hyperfine (for benchmarking), curl (for downloading)
apt install bzip2 xz-utils hyperfine curl -y
# Download firefox
curl -O https://archive.mozilla.org/pub/firefox/releases/130.0/linux-x86_64/en-US/firefox-130.0.tar.bz2
# Create xz compressed from bz2
bzip2 --decompress -k --stdout firefox-130.0.tar.bz2 | xz -9 -e --lzma2 --stdout > firefox-130.0.tar.xz

# Test with hyperfine
hyperfine --runs 10 \
"xz -d -k --stdout firefox-130.0.tar.xz > /dev/null" \
"bzip2 -d -k --stdout firefox-130.0.tar.bz2 > /dev/null"

And here is my results on the single core VM:

$ hyperfine --runs 10 \
"xz -d -k --stdout firefox-130.0.tar.xz > /dev/null" \
"bzip2 -d -k --stdout firefox-130.0.tar.bz2 > /dev/null"
Benchmark 1: xz -d -k --stdout firefox-130.0.tar.xz > /dev/null
  Time (mean ± σ):      5.530 s ±  0.014 s    [User: 5.443 s, System: 0.069 s]
  Range (min … max):    5.509 s …  5.558 s    10 runs
 
Benchmark 2: bzip2 -d -k --stdout firefox-130.0.tar.bz2 > /dev/null
  Time (mean ± σ):     11.487 s ±  0.135 s    [User: 11.309 s, System: 0.142 s]
  Range (min … max):   11.396 s … 11.855 s    10 runs
 
Summary
  'xz -d -k --stdout firefox-130.0.tar.xz > /dev/null' ran
    2.08 ± 0.02 times faster than 'bzip2 -d -k --stdout firefox-130.0.tar.bz2 > /dev/null'

I hope this helps cover your concerns.

Best wishes,
Heitor Neiva

1 Like