rss logo
Computing
Home
How To
Music
The Band
Tracks
Misc
Links
🇫🇷 Fr

Data compression benchmark under GNU/Linux

Intro

I was wondering about which was the best files compression and decompression software under GNU/Linux so I made a real-world test to see the differences in performance.

Test Environment

6 folders
  • I created six folders with the following characteristics:
    • AUDIO: 101MB, seven mp3 and seven ogg files
    • IMG: 102MB, 17 jpg files
    • PDF: 102MB, 68 pdf files
    • RANDOM TXT: 95MB, one big txt file, made with base64 /dev/urandom | head -c 100000000 > file.txt command.
    • MAN TXT: 7.2MB, 870 man files
    • VIDEOS: 290MB, two mp4, one mpeg and one webm file

Tools

7zip

7zip logo
  • Compression: yes
  • Archiving: yes
  • Initial release: 1999
  • Misc: doesn't save owner/group of files

Compress

  • Add file/directory to the archive :
user@SOURCE:~$ 7z a DEST.7z SOURCE

Decompress

  • Extract with full path :
user@SOURCE:~$ 7z x DEST.7z

bzip2

Bzip2 logo
  • Compression: yes
  • Archiving: no
  • Initial release: 1996
  • Misc:

Compress

  • Compress FILE to FILE.bz2 and remove FILE (-k to keep):
user@SOURCE:~$ bzip2 -z FILE
  • Same but will keep FILE:
user@SOURCE:~$ bzip2 -c FILE > FILE.bz2

Decompress

  • Decompress FILE.bz2 to FILE and remove FILE.bz2:
user@SOURCE:~$ bzip2 -d FILE

LZ4

  • Compression: yes
  • Archiving: no
  • Initial release: 2011
  • Misc:

Compress File

  • Compress FILE to FILE.lz4:
user@SOURCE:~$ lz4 -z FILE FILE.lz4

Decompress

  • Decompress FILE.lz4 to FILE:
user@SOURCE:~$ lz4 -d FILE.lz4

gzip

Gzip logo
  • Compression: yes
  • Archiving: no
  • Initial release: 1992
  • Misc:

Compress

  • Compress FILE to FILE.gz and remove FILE (-k to keep):
user@SOURCE:~$ gzip FILE
  • Same but will keep FILE:
user@SOURCE:~$ gzip -c FILE > FILE.gz

Decompress

  • Decompress FILE.gz to FILE:
user@SOURCE:~$ gzip -d FILE.gz

xz

  • Compression: yes
  • Archiving: no
  • Initial release:
  • Misc:

Compress

  • Compress FILE to FILE.xz and remove FILE (-k to keep):
user@SOURCE:~$ xz -z FILE
  • Same but will keep FILE:
user@SOURCE:~$ xz -c FILE > FILE.gz

Decompress

  • Decompress FILE.xz to FILE:
user@SOURCE:~$ xz -d FILE.xz

zip

  • Compression: yes
  • Archiving: yes
  • Initial release: 1989
  • Misc: doesn't save owner/group of files

Compress

  • Zip file to the archive:
user@SOURCE:~$ zip FILE.zip FILE
  • Zip directory to the archive:
user@SOURCE:~$ zip -r DIR.zip DIR

Decompress

  • Decompress FILE.zip:
user@SOURCE:~$ unzip FILE.zip

zstd

zstd logo
  • Compression: yes
  • Archiving: no
  • Initial release: 2015
  • Misc:

Compress

  • Compress FILE to FILE.zstd:
user@SOURCE:~$ zstd -z FILE

Decompress

  • Decompress FILE.zst to FILE:
user@SOURCE:~$ zstd -d FILE.zst

Benchmark Results

Command used

The following command allows to measure the size of the elements before and after compression, and also to know the execution time.

user@SOURCE:~$ echo "BEFORE : $(du -sh "AUDIO")"; time ( tar --gzip -cvf "AUDIO".tar.gz "AUDIO" ); echo "AFTER : $(du -sh "AUDIO".tar.gz)" BEFORE : 101M AUDIO AUDIO/ AUDIO/The_Dolphins_-_Demo_-_Easy_Way.mp3 AUDIO/The_Dolphins_-_Demo_-_Easy_Way.ogg AUDIO/The_Dolphins_-_Demo_-_Illusions_And_Witnesses.mp3 AUDIO/The_Dolphins_-_Demo_-_Illusions_And_Witnesses.ogg AUDIO/The_Dolphins_-_Demo_-_Impro.mp3 AUDIO/The_Dolphins_-_Demo_-_Impro.ogg AUDIO/The_Dolphins_-_Demo_-_Insubstantial_As_Me.mp3 AUDIO/The_Dolphins_-_Demo_-_Something.ogg AUDIO/The_Dolphins_-_Demo_-_The_Elements_Of_A_State.mp3 AUDIO/The_Dolphins_-_Demo_-_The_Elements_Of_A_State.ogg AUDIO/The_Dolphins_-_Demo_-_The_Wisp.mp3 AUDIO/The_Dolphins_-_Demo_-_The_Wisp.ogg AUDIO/The_Dolphins_-_Demo_-_Unnecessary_Evil.mp3 AUDIO/The_Dolphins_-_Demo_-_Unnecessary_Evil.ogg real 0m3,390s user 0m3,294s sys 0m0,167s AFTER : 101M AUDIO.tar.gz

Performance Chart

Tests performed with a Intel(R) Core(TM) i3-4360 CPU @ 3.70GHz
Tool Medias Init Size Compressed Size Compression Rate Compression Time Command Decompress Time Command
Compression Decompression
7zip AUDIO 101MB 101MB 0% 11s 7z a DEST.7z SOURCE 0.8s 7z x DEST.7z
IMG 102MB 102MB 0% 12s 0.8s
PDF 102MB 78MB 24% 10s 3s
MAN TXT 7.2MB 1.4MB 85% 1.4s 0.0s
RANDOM TXT 96MB 74MB 23% 15s 4.2s
VIDEOS 290MB 288MB 1% 33s 2s
bzip2 AUDIO 101MB 101MB 0% 13s tar --bzip2 -cvf DEST.tar.bz2 SOURCE 7.7s tar --bzip2 -xvf DEST.tar.bz2 -C ./
IMG 102MB 101MB 1% 12s 7.7s
PDF 102MB 80MB 22% 12.5s 6.2s
MAN TXT 7.2MB 1.2MB 83% 0.4s 0.2s
RANDOM TXT 96MB 73MB 24% 10s 5.8s
VIDEOS 290MB 289MB 0% 37s 25s
LZ4 AUDIO 101MB 101MB 0% 0.1s tar cvf - SOURCE | lz4 > DEST.tar.lz4 0.0s tar -I lz4 -xvf DEST.tar.lz4 -C ./
IMG 102MB 102MB 0% 0.1s 0.0s
PDF 102MB 84MB 18% 0.2s 0.1s
MAN TXT 7.2MB 2.3MB 68% 0s 0.0s
RANDOM TXT 96MB 96MB 0% 0s 0.0s
VIDEOS 290MB 290MB 0% 0.4s 0.2s
gzip AUDIO 101MB 101MB 0% 3.4s tar --gzip -cvf DEST.tar.gz SOURCE 0.7s tar --gzip -xvf DEST.tar.gz -C ./
IMG 102MB 102MB 0% 3.4s 0.68s
PDF 102MB 83MB 19% 3.4s 0.63s
MAN TXT 7.2MB 1.5MB 79% 0.2s 0.0s
RANDOM TXT 96MB 73MB 24% 4.3s 0.68s
VIDEOS 290MB 289MB 0% 9.3s 1.6s
xz AUDIO 101MB 101MB 0% 37s tar --xz -cvf DEST.tar.xz SOURCE 0.9s tar --xz -xvf DEST.tar.xz -C ./
IMG 102MB 102MB 0% 38s 0.39s
PDF 102MB 78MB 24% 12.5s 3.3s
MAN TXT 7.2MB 1.1MB 85% 2.3s 0.08s
RANDOM TXT 96MB 74MB 23% 40s 4.9s
VIDEOS 290MB 288MB 1% 1m46s 3s
zip AUDIO 101MB 101MB 0% 3s zip -q -r DEST.zip SOURCE 0.7s unzip DEST.zip
IMG 102MB 102MB 0% 3s 0.6s
PDF 102MB 83MB 19% 3s 0.6s
MAN TXT 7.2MB 2MB 72% 4s 0.06s
RANDOM TXT 96MB 73MB 24% 4s 0.6s
VIDEOS 290MB 289MB 0% 8s 1.6s
zstd AUDIO 101MB 101MB 0% 0.4s tar --zstd -cvf DEST.tar.zst SOURCE 0.4s tar --zstd -xvf DEST.tar.zst -C ./
IMG 102MB 102MB 0% 0.3s 0.1s
PDF 102MB 79MB 23% 0.35s 0.1s
MAN TXT 7.2MB 1.4MB 81% 0.05s 0.02s
RANDOM TXT 96MB 72MB 25% 0.24s 0.15s
VIDEOS 290MB 290MB 0% 0.8s 0.26s
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Contact :

contact mail address