splitjob 2.0

Useful to split up jobs over multiple CPU cores or even multiple computers. Examples: tar -cf - /bigdirectory | splitjob -j 4 gzip > big.tar.gz splitjob "ssh h1 bzip2" "ssh h2 bzip2" < f > f.bz2

Tags c command-line distributed-computing
License GNU GPL
State stable

Recent Releases

2.015 Oct 2017 20:33 major feature: Now accepts signal SIGUSR2 to increase number of parallel running jobs and SIGUSR1 to decrease number of parallel running jobs.
1.209 Oct 2017 21:11 minor feature: Might now be able to recover if sub process fails even if some data has been read out from the sub process.
1.131 Jan 2015 18:51 minor feature: Saving RAM by freeing up unused RAM in child processes.
1.014 Dec 2014 14:57 documentation: No changes since previos beta version which have now been tested for some months without any problems found.
0.9.2beta25 Aug 2014 11:36 minor bugfix: Avoiding short reads from stdin which could cause non optimal compression performance when read block sizes got smaller than intended.