How to compress (bzip / gzip) a very large text quickly ?

Compressing files is very important when you are using platforms like Hadoop or FTP. Here is how I do it easily and quickly.

Step 1 :

Install GNU Parallels in your machine. How check here ?

Step 2 : csv to bz2

$ cat largefile.csv | /usr/local/bin/parallel — pipe -k bzip2 — best > largefile.bz2

Step 2 : csv to gz

$ cat largefile.csv | /usr/local/bin/parallel — pipe -k gzip > largefile.gz