4.11. Using Shell Redirection and Piping
The Unix/Linux philosophy revolves around the concept of programs as building blockseach one intended to do one job and do it well. Redirection lets you connect these commands to files, and piping enables you to plug commands together like a child's toy.
4.11.1. How Do I Do That?
By default, these file descriptors are connected to the terminal, if one is available, so standard input comes from the terminal keyboard, and standard output and standard error go to the terminal screen. Programs may open any other connections they need to read or write files, communicate with other local programs, or communicate with programs over the network.
$ cal 7 2006 July 2006 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 $ cal 7 2006 >month.txt $ cat month.txt July 2006 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
When you redirect output with >, the previous contents of the file are overwritten. To append (add) to the file, use >>:
$ cal 3 2009 >>month.txt $ cat month.txt July 2006 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 March 2009 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Error messages are not sent to standard output, so you can still see the messages even when standard output is redirected:
$ cal 17 2009 >month.txt cal: illegal month value: use 1-12
$ cal 17 2009 2>errors $ cat errors cal: illegal month value: use 1-12
$ cal 17 2009 >month.txt 2>errors
$ echo "2^8" >problem $ bc <problem 256
bc is a calculator program. The first command places a numeric expression in the file problem; the second line starts bc, using problem as the input. The output from bc is the solution of the expression: 256.
Of course, you can redirect both input and output:
$ bc <problem >result
$ mount /dev/mapper/main-root on / type ext3 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) /dev/hdc2 on /boot type ext3 (rw) tmpfs on /dev/shm type tmpfs (rw) /dev/mapper/main-home on /home type ext3 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/sdb on /media/disk type vfat (rw,noexec,nosuid,nodev,shortname=winnt,uid=503) $ mount | grep /dev/mapper /dev/mapper/main-root on / type ext3 (rw) /dev/mapper/main-home on /home type ext3 (rw)
In this example, the output of the mount command is used as the input to the grep command, which outputs only lines that match the specified pattern. A group of commands connected together with pipe symbols is known as a pipeline. You can extend a pipeline by connecting additional commands:
$ mount | grep /dev/mapper | sort /dev/mapper/main-home on /home type ext3 (rw) /dev/mapper/main-root on / type ext3 (rw)
The input to a pipeline and the output from a pipeline may be redirected:
$ cut -d: -f1 </etc/passwd|sort|head >output $ cat output adm apache avahi beaglidx bin chip chris daemon dbus distcache
However, it's essential that the input redirect take place at the start of the pipeline (at the command on the left) and that the output redirection take place at the end (at the command on the right). Consider this wrong example:
$ cut -d: -f1 </etc/passwd|sort >output|head
In this case, it's unclear whether the standard output of sort should be placed in the file output or used as the standard input to the head command. The result is undefined (which means don't do this!).
4.11.2. How Does It Work?
Redirection is set up by the bash shell before the command is executed. If there is a redirection error (such as an invalid filename or a file permission problem), it will be reported by the shell and the command will not be executed:
$ cal >foo/bar/baz bash: foo/bar/baz: No such file or directory
A command is not aware of file redirection unless it has specifically been programed to check the standard file descriptors or perform special operations on them (such as changing terminal characteristics). Redirected file descriptors are inherited by applications that were started by commands; in this example, the nice command starts the cal command, and cal inherits the redirection set up for nice:
$ nice "cal" >test.txt $ cat test.txt July 2006 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
4.11.3. What About...
184.108.40.206. ...redirecting standard output and standard error to the same destination?
$ cal 17 2009 >/tmp/calresult 2>&1
Notice that the order of the redirections matters. The preceding command will redirect all output to /tmp/calresult, but this command will not redirect standard error:
$ cal 17 2009 2>&1 >/tmp/calresult
The 2>&1 redirection is evaluated first, so standard error is directed to the same destination as standard output (which, at that point, is the terminal); > /tmp/calresult then redirects standard output by itself.
This construct can also be used with piping:
$ cal 17 2009 2>&1 | head -2
This will feed both the standard output and the standard error from cal into the standard input of head.
220.127.116.11. ...redirecting to a device?
$ head -50 /etc/services >/dev/lp0
18.104.22.168. ...splitting a pipe to send data to two destinations?
$ cal -y | tee /tmp/thisyear.txt | head -2
22.214.171.124. ...piping and redirecting data that is not text?
No assumptions are made about the type of data being piped or redirected; in fact, there are many programs that are designed to work with piped graphics, audio, or video data streams. For example, this pipeline will decode a color JPEG image, scale it to half-size, convert it to grayscale, normalize it, convert it back into a JPEG, save a copy as /tmp/final.jpg, and display the output in a window:
$ djpeg /usr/share/wallpapers/floating-leaves.jpg | pnmscale 0.5 | ppmtopgm | ppmnorm | cjpeg | tee /tmp/final.jpg | display -
4.11.4. Where Can I Learn More?