Use the -p option to mkdir and make all parent directories along with their children in a single command.

mkdir -p tmp/a/b/c

Use && to run a second command if and only if a first command succeeds:

cd tmp/a/b/c && tar xvf ~/archive.tar

Use || to run a second command if and only if a first command fails:

cd /tmp/a/b || mkdir -p /tmp/a/b

Add commas to all numeric strings in a file, changing "1234567" to "1,234,567"

sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta' filename

Add a leading angle bracket and space to each line (quote a message)

sed 's/^/> /' filename

Place a filename at the beginning of the line to make it easier to edit the search at the end of the command.

$ </var/log/messages grep foo

$ </var/log/messages grep bar

$ </var/log/messages grep user1

Grepping for a process will return the grep command, this can be avoided by adding '| grep -v grep' to a command or easier in some cases altering the regular expression by adding brackets around a character.

ps | grep 'ss[h]'

The regular expression 'ss[h]' matches the literal string 'ssh' when it appears in the process list, but does not accidentally match the string 'ss[h]' when it appears in the process list as 'grep ss[h]'.

It can be useful to not only know what has gone to stdout and stderr but also where they occurred with respect to each other.
Allow stderr to go to err.txt, stdout to out.txt and both to mix.txt

((./program 2>&1 1>&3 | tee ~/err.txt) 3>&1 1>&2 | tee ~/out.txt) > ~/mix.txt 2>&1

Don't search history by grepping ~/.bash_history, or repeatedly hitting the up arrow, instead use CTRL+r (or '/' in vi-mode) for search-as-you type. You can immediately run the command by pressing Enter.

Instead of adding a prefix to the previous command with 'Up-arrow Home prefix Space' try 'prefix !!' to repeat the last command with 'prefix ' before it.
You can also use '!-n' to use commands other than the most recent.

Running a second command with the same arguments as the previous command, use '!*' to repeat all arguments or '!:2' to use the second argument. '!$' uses the final argument.

$ cd /home/user/foo

cd: /home/user/foo: No such file or directory

$ mkdir !*

mkdir /home/user/foo

Use awk to change the file extension for a group of files. For example to change all .htm files to .php:

ls *htm | awk -F. '{print "mv "$0" "$1".php"}' | sh

This can be tested first by leaving the '| sh' off to give a list of the commands that will be executed.

mknod backpipe p; while nc -l -p 80 0<backpipe | tee -a inflow | \
nc localhost 81 | tee -a outflow 1>backpipe; do echo \"restarting\"; done

Listen on localhost:80, forward to localhost:81 and log both sides of the conversation to outflow, automatically restarting if the connection dies.

CREATE THE TARBALL

tar cf - sourceFiles | gzip -c > fileToCreate.tgz

EXTRACTION

zip -d -c fileToExtract.tgz | tar xf -

#!/bin/ksh

#KSH only.
# Author: simonvc
# Purpose: Count the total number of occurances of a pattern in all files in the cwd.
# Alt: you could also do "$(grep $pattern * | wc -l) but where's the fun in that ;-)


pattern="WHATEVER"

$ echo $(($(grep -hc $pattern *| tr "\n" "+")"0"))

Prefer Perl over Sed? No problem! To use Perl as a Sed-like program:

perl -pe 's/foo/bar/; etc'

Have a Perl script ready? No problem!

perl -p foo.pl

GNU tar comes with native support for gzip, bzip2, and compress (adaptive LZ, LZW). However, many other useful compression algorithms exist, but most implementations of them don't support all the file system metadata that tar does. There are two general methods to using tar with arbitrary compression programs: via an option in tar itself and via piping. The first:

tar -cf foo.tar.ext --use-compress-program generic_compression_program file1 [file2] [file3] [...]

Note that this method is not very configurable and only works with programs that compress by default and will decompress files when invoked with the "-d" option.

The second method:

tar -c file1 [file2] [file3] [...] |generic_compression_program >archive.tar.ext

This method is more versatile, but it requires a bit more shell voo-doo to work (and may not require use of stdout redirection). For instance, to make a .tar.7z archive:

tar -c list of files and directories to compress |7z a -si outfile.tar.7z

To uncompress said archive:

7z e -so infile.tar.7z |tar -x

multiple command output into a single program:

diff -u <(ls -c1 dir_1) <(ls -c1 dir_2)

Will show you a diff of files in the root of dir_1 and dir_2

Using expansion to move a file aside without having to type the file name twice

cp ReallyLongFileNameYouDontWantToTypeTwice{,.orig}

Rename replaces string X in a set of file names with string Y.

rename 's/.html$/.php/' *.html

This will change the extension of every .html file in your CWD to .php.

Mail somebody about space running low in some path (ksh, bash):

PATHS="/export/home /home"

AWK=/usr/bin/awk

DU="/usr/bin/du -ks"

GREP=/usr/bin/grep

DF="/usr/bin/df -k"

TR=/usr/bin/tr

SED=/usr/bin/sed

CAT=/usr/bin/cat

MAILFILE=/tmp/mailviews$$

MAILER=/bin/mailx

mailto="all@company.com" 

for path in $PATHS

do

	DISK_AVAIL=`$DF $path |  $GREP -v "Filesystem" | $AWK '{print $5}'|$SED 's/%//g'` 

	if [ $DISK_AVAIL -gt 90 ];then

		echo "Please clean up your stuff\n\n" > $MAILFILE

		$CAT $MAILFILE | $MAILER -s "Clean up stuff" $mailto	

	fi

done 

writevt --term=/dev/pts/1 --text=foo

ls -A -l -h --color=auto

Does ls but with :
-column list format (think dos)
-human readable sizes (megabytes, kb)
-color if enabled

Save to your default settings !

alias webshare='python -c "import SimpleHTTPServer;SimpleHTTPServer.test()"'

Want to show something on your machine to someone over the web? Don't copy it or upload it somewhere. Just run "webshare" and the current directory and everything beneath it will be served from a new web server listening on port 8000. When your pal is finished, hit control-c.

Throw this in your command pipeline:

perl -ne '$sum += $_;}{ print "Sum: $sum\n";'

The

}{

closes the

while ( <> ) {

loop that Perl wraps your code in (because of the

-n

) and opens a new one that will only execute after the while loop has finished. You can use this in lots of Perl one-liner tricks.

For a contrived example, to sum the sizes of files in the current directory (learn the joys of cut(1) first):

ls -l | cut -c 30-42 | perl -ne '$sum += $_;}{ print "Sum: $sum\n";'

cut grabs characters 30-42 from stdin, which happen to coincide with my file size column in ls.