Shell parallel processing

This is the description of the tool from the gnu site - Parallel
GNU parallel is a shell tool for executing jobs in parallel using one
or more computers. A job it can be a single command or a small script
that has to be run for each of the lines in the input. The typical
input is a list of files, a list of hosts, a list of users, a list of
URLs, or a list of tables. A job can also be a command that reads from
a pipe. GNU parallel can then split the input into blocks and pipe a
lock into each command in parallel.

The tool can do many things and has some very useful tools that come with it
see sql and niceload.
Bellow you can see some example on how to use it.

#!/bin/sh

# tail log files on different computers
# create a hosts file with all the computers you want to connect

echo '10.100.218.79' >> host.file
echo '107.22.24.219' >> host.file
cat host.file | parallel ssh {} "tail /var/log/php-fpm/error.log | awk '{print \$1,\$2,\$3,\$4,\$5,\$6}'"

[20-Feb-2012 15:05:05.323178] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:06.324028] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:07.324877] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:08.325727] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:09.326568] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:10.327418] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:11.328265] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:12.329118] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:13.329960] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),
[20-Feb-2012 15:05:14.330806] DEBUG: pid 7812, fpm_pctl_perform_idle_server_maintenance(),

GNU's site has lots more example see Examples

Delete empty lines into a file

Problem - you have a file that has a huge number of records and in between you have
empty lines.
This is my example.txt file:

asd

asd
as


sd
as

asd

asd

sd
g

To take the spaces off you have at least two simple tools to do it.

• sed with this tool you can achieve your goal just by doing this - open a command line prompt and type the following
  linux$ cat example.txt | sed -e '/^$/d' you will see on your screen the following
  

  asd
  asd
  as
  sd
  as
  asd
  asd
  sd
  g
  

• grep with this we use a trick saying that we don't want certain patterns from the file - the empty lines - open a command line prompt and type the following
  linux$ grep -v '^$' example.txt
  

asd
asd
as
sd
as
asd
asd
sd
g