Shell scripts

To create a shell script, put shell commands into a file starting with

#!/bin/sh

and run chmod a+x file.

Many shell scripts use temporary files. To ensure that these get cleaned up on exit and if the shell script is interrupted, include a line like this at the beginning of the script:

trap '/bin/rm -fr $tmp; exit' 0 1 2 3 15

Names of temporary files should be of the form /tmp/prog$$ for uniqueness. $$ is a built-in shell variable whose value is the shell's process number.

NOTE In the trap command, 0 means ordinary exit, while the other numbers are signal numbers (e.g. 2 is interrupt, usually generated with ^C). You can find out about other signal numbers by reading /usr/include/sys/signal.h

Note you can use arguments such as myscript.sh arg1 arg2

To access this within the program usinf $1 $2 etc $0 is the name of the script itself

Why learn shell programming?

Many small tasks can be carried out entirely by shell programming.
Bigger tasks, or parts of them, can be prototyped in shell.
Many Unix tools build upon the shell (e.g. make, shar).
Writers of Unix programs must know what to expect from the shell, and what the shell expects from them.
It is a different kind of programming language, giving new perspectives.

Shell variables

Shell variables have strings as values. Dollar signs introduce references to the value of a variable.

$ var=value

$ echo var $var

var value

$ echo ${var}cont

valuecont

One can supply a default for use when the variable has not been set.

$ echo ${var-def}

value

$ echo ${x-def} $x

def

$ echo ${x=def} $x

def def

NOTE There must be no spaces around the equal sign in var=value. If there is one, the shell will think you are trying to invoke a command called var.

The environment

By default, shell variables are internal to the shell. However, one can make them available to commands invoked by the shell by executing

$ export var

The set of exported variables becomes the environment of invoked commands.

C programs can get the values of these environment variables by executing the getenv library function.

NOTE A process can modify its own environment, and pass whatever initial environment it likes to the processes it creates, but it cannot modify the environment of another running process.

Quoting

Single forward quotes protect against all processing. Double quotes allow variable substitutions, with the result being a single argument.

$ var='word1 word2'

$ arg $var

arg 0:

arg 1:

arg 2:

$ arg "$var"

arg 0:

arg 1:

$ arg '$var'

arg 0:

arg 1: <$var>

NOTE Shell scripts will often use e.g. "$1" instead of just $1 simply to protect against any spaces in the value of the variable, and to protect against the value having the empty string as its value (a variable that has never been set also has the empty string as its value).

Failing to put in the proper quotation marks will yield a script that works with all ordinary filenames but falls over on filenames containing special characters.

Exit Status

All commands should indicate success or failure via their exit status, the argument of the exit system call. By convention, zero indicates success, other values indicate failure.

The convention of the test command is the opposite of C's convention, in which a condition being true is denoted by a non-zero value.

The reason for the shell's convention is that a program can succeed only one way, but it may encounter many possible errors, and the exit code should be able to indicate which was in fact encountered.

NOTE When writing programs for Unix, always make sure that you either call exit or return a value from main.

If a command does not explicitly return an exit status, its exit status is effectively random (it is whatever happened to be in the right register at the time of their exit). This is bad; for instance, it means that an invocation of such a command in an action in a Makefile will typically cause make to stop executing the rest of the action (unless the action line starts with a minus sign, in which case make ignores the exit status of the action).

The test command

The command test exits with zero status if and only if a given condition is true. Conditions are boolean expressions involving

string comparison (=, !=)

numeric comparison (-eq, -ne, -lt, -le, -gt, -ge)

tests on files (given a file name, can test

    whether it is a regular file (-f),

    whether it is a directory (-d),

    whether it is writeable (-w), etc)

For example, ``test $x = $y'' returns success when the variables x and y hold the same string, and ``test -d $dir'' returns success if the variable dir holds the name of a directory.

If-then-else

The condition in a shell if-then-else statement is a command that is terminated by end of line or by a semicolon. The command's exit status decides which way the if goes.

if test ! -r "$file" ; then

    echo "$prog: $file does not exist \

 or is not readable."

    exit 1

fi

if grep "^$user:" /etc/passwd > /dev/null

then

    ...

else

    echo "$prog: $user is not a valid user"

fi

NOTE The command [ is another name for test; when invoked by this name, test ignores a final ]. Therefore the first example could have been written like this:

if [ ! -r "$file" ]

then

    echo "$prog: $file does not exist \

 or is not readable."

    exit 1

fi

The quotes around $file are to protect against file not being set or set to a value containing only blanks. In such cases, and without the quotes, the shell would not pass any arguments to the test command after the -r; with the quotes, it will pass an empty string as the final argument.

grep exits with status 0 if it finds the given pattern, and with status 1 if it doesn't. Status 2 indicates a syntax error on the command line or that it couldn't open a file. The message in the fragment above is correct only if you know that $user will always be acceptable to grep.

If /etc/passwd does not exist or is not readable, the system is in deep trouble. Since it stores people's encrypted passwords, which are checked against the encrypted form of the passwords supplied by users when they log in, noone will be able to log in if /etc/passwd is missing.

If you did not include the redirection to /dev/null, any lines in /etc/passwd matching the pattern will appear on standard output.

It is one of the cardinal principles of error message design that the error message should give the name of the program that generated it.

Case

case "$file" in

*.c)    echo C source ;;

*.h)    echo C header ;;

.*)     echo dot file ;;

*)      echo unknown

 exit 1

 ;;

esac

The shell will attempt to match the value of $file first against the pattern *.c, then against *.h etc. When it finds a match, it executes the corresponding commands until it finds a double semicolon. This acts like C's break statement, except that it is compulsory.

Looping constructs

While loops use conditions the same way if statements do:

count=0

while test "$count" -lt 6 ; do

    sleep 600

    uptime

    count=`expr "$count" + 1`

done

For loops set a variable successively to values from a given list:

for file in *.c

do

    gcc -c "$file"

done

NOTE The while example prints the system load every ten minutes for an hour. The for example compiles every C source file in the current directory.

The shell has no arithmetic capabilities built in, which is why you must use the expr command to do arithmetic. You must remember to give each component of the expression to expr as a separate argument, or it will not work as you expect.

The above for statement has a bug caused by a mistake in sh's design. If there are no files with names ending in .c in the directory, then sh will execute the loop body once, with the variable ``file'' set to the string ``*.c''. Unless you are sure that this situation will never arise, you have to test for this and handle it separately.

Composition constructs

Pipes can connect commands even if they contain control structures.

generate_filenames |

while read file ; do

    gcc -c "$file"

done

Commands within backquotes are replaced by their output. Commands in parentheses are executed in a subshell; changes to the subshell's environment do not affect the parent.

for dir in `cat dirtydirs` ; do

    (cd "$dir" ; make clean)

done

NOTE You should already have seen what happens when a string is enclosed in back quotes: sh interprets the string as a command, executes it, and substitutes the output for the command string.

The advantage of invoking a separate subshell for each dirty directory is that the shell does not have to try to go back to the original directory. This can be a difficult operation, since the names of the dirty directories can refer to symbolic links,

A symbolic link is a special type of file, which has a flag that tells the operating system ``this file or directory is not here, it is over there'', and a string that tells the operating system where it is. If you do ``ls -l /usr'' on the CASE machines, you will see that many of the subdirectories of /usr are actually symbolic links, as shown by the arrows.

File descriptors

When a Unix program opens a file, it gets back a file descriptor. When the program wants to read from a file or write to a file, it must specify the file descriptor of the file on which the operation should be performed.

By convention, file descriptor 0 refers to standard input, 1 refers to standard output, and 2 refers to standard error output.

On Unix systems, every process created by the shell has these three file descriptors when they start running. They are normally set to the keyboard, the screen and the screen respectively.

NOTE The Unix convention is that normal output should be printed to standard output (stdout), while output reporting errors should be printed to the error output (stderr). This is so that even if a user redirects output away from the screen, he or she is still able to see any error messages.