Lnxgeek.org Wiki - Freedom of choice, I choose Freedom

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Bash test and comparison functions ======


===== Demystify test, [, [[, ((, and if-then-else =====

Are you confused by the plethora of testing and comparison options in the Bash shell? This tip helps you demystify the various types of file, arithmetic, and string tests so you will always know when to use test, <nowiki>[ ], [[ ]], (( ))</nowiki>, or if-then-else constructs.

The Bash shell is available on many Linux® and UNIX® systems today, and is a common default shell on Linux. Bash includes powerful programming capabilities, including extensive functions for testing file types and attributes, as well as the arithmetic and string comparisons available in most programming languages. Understanding the various tests and knowing that the shell can also interpret some operators as shell metacharacters is an important step to becoming a power shell user. This article, excerpted from the developerWorks tutorial LPI exam 102 prep: Shells, scripting, programming, and compiling, shows you how to understand and use the test and comparison operations of the Bash shell.

This tip explains the shell test and comparison functions and shows you how to add programming capability to the shell. You may have already seen simple shell logic using the && and || operators, which allow you to execute a command based on whether the previous command exits normally or with an error. In this tip, you will see how to extend these basic techniques to more complex shell programming.


===== Tests =====

In any programming language, after you learn how to assign values to variables and pass parameters, you need to test those values and parameters. In shells, the tests set the return status, which is the same thing that other commands do. In fact, test is a builtin command!


==== test and [ ====


The test builtin command returns 0 (True) or 1 (False), depending on the evaluation of an expression, expr. You can also use square brackets: test expr and [ expr ] are equivalent. You can examine the return value by displaying $?; you can use the return value with && and ||; or you can test it using the various conditional constructs that are covered later in this tip.

**Listing 1. Some simple tests**
<code>
[ian@pinguino ~]$ test 3 -gt 4 && echo True || echo false
false
[ian@pinguino ~]$ [ "abc" != "def" ];echo $?
0
[ian@pinguino ~]$ test -d "$HOME" ;echo $?
0
</code>


In the first example in Listing 1, the -gt operator performs an arithmetic comparison between two literal values. In the second example, the alternate [ ] form compares two strings for inequality. In the final example, the value of the HOME variable is tested to see if it is a directory using the -d unary operator.

You can compare arithmetic values using one of -eq, -ne, -lt, -le, -gt, or -ge, meaning equal, not equal, less than, less than or equal, greater than, and greater than or equal, respectively.

You can compare strings for equality, inequality, or whether the first string sorts before or after the second one using the operators =, !=, <, and >, respectively. The unary operator -z tests for a null string, while -n or no operator at all returns True if a string is not empty.

Note: the < and > operators are also used by the shell for redirection, so you must escape them using \< or \>. Listing 2 shows more examples of string tests. Check that they are as you expect.

**Listing 2. Some string tests**
<code>
[ian@pinguino ~]$ test "abc" = "def" ;echo $?
1
[ian@pinguino ~]$ [ "abc" != "def" ];echo $?
0
[ian@pinguino ~]$ [ "abc" \< "def" ];echo $?
0
[ian@pinguino ~]$ [ "abc" \> "def" ];echo $?
1
[ian@pinguino ~]$ [ "abc" \<"abc" ];echo $?
1
[ian@pinguino ~]$ [ "abc" \> "abc" ];echo $?
1
</code>


^  Table 1. Some common file tests ^^^
|  Operator  |  Characteristic  |
|  -d	| Directory  |
|  -e	| Exists (also -a)  |
|  -f	| Regular file  |
|  -h	| Symbolic link (also -L)  |
|  -p	| Named pipe  |
|  -r	| Readable by you  |
|  -s	| Not empty  |
|  -S	| Socket  |
|  -w	| Writable by you  |
|  -N	| Has been modified since last being read  |


In addition to the unary tests above, you can compare two files with the binary operators shown in Table 2.

^  Table 2. Testing pairs of files  ^^^
|  Operator  |  True if   |
|  -nt	| Test if file1 is newer than file 2. The modification date is used for this and the next comparison.  |
|  -ot	| Test if file1 is older than file 2.  |
|  -ef	| Test if file1 is a hard link to file2.  |


Several other tests allow you to check things such as the permissions of the file. See the man pages for bash for more details or use help test to see brief information on the test builtin. You can use the help command for other builtins too.

The -o operator allows you to test various shell options that may be set using set -o option, returning True (0) if the option is set and False (1) otherwise, as shown in Listing 3.

**Listing 3. Testing shell options**
<code>
[ian@pinguino ~]$ set +o nounset
[ian@pinguino ~]$ [ -o nounset ];echo $?
1
[ian@pinguino ~]$ set -u
[ian@pinguino ~]$ test  -o nounset; echo $?
0
</code>


Finally, the -a and -o options allow you to combine expressions with logical AND and OR, respectively, while the unary ! operator inverts the sense of the test. You may use parentheses to group expressions and override the default precedence. Remember that the shell will normally run an expression between parentheses in a subshell, so you will need to escape the parentheses using \( and \) or enclosing these operators in single or double quotes. Listing 4 illustrates the application of de Morgan's laws to an expression.

**Listing 4. Combining and grouping tests**
<code>
[ian@pinguino ~]$ test "a" != "$HOME" -a 3 -ge 4 ; echo $?
1
[ian@pinguino ~]$ [ ! \( "a" = "$HOME" -o 3 -lt 4 \) ]; echo $?
1
[ian@pinguino ~]$ [ ! \( "a" = "$HOME" -o '(' 3 -lt 4 ')' ")" ]; echo $?
1
</code>







==== (( and [[ ====


The test command is very powerful, but somewhat unwieldy given its requirement for escaping and given the difference between string and arithmetic comparisons. Fortunately, bash has two other ways of testing that are somewhat more natural for people who are familiar with C, C++, or Java® syntax.

The <nowiki>(( ))</nowiki> compound command evaluates an arithmetic expression and sets the exit status to 1 if the expression evaluates to 0, or to 0 if the expression evaluates to a non-zero value. You do not need to escape operators between <nowiki>(( and ))</nowiki>. Arithmetic is done on integers. Division by 0 causes an error, but overflow does not. You may perform the usual C language arithmetic, logical, and bitwise operations. The let command can also execute one or more arithmetic expressions. It is usually used to assign values to arithmetic variables.

**Listing 5. Assigning and testing arithmetic expressions**
<code>
[ian@pinguino ~]$ let x=2 y=2**3 z=y*3;echo $? $x $y $z
0 2 8 24
[ian@pinguino ~]$ (( w=(y/x) + ( (~ ++x) & 0x0f ) )); echo $? $x $y $w
0 3 8 16
[ian@pinguino ~]$ (( w=(y/x) + ( (~ ++x) & 0x0f ) )); echo $? $x $y $w
0 4 8 13
</code>

As with <nowiki>(( ))</nowiki>, the <nowiki>[[ ]]</nowiki> compound command allows you to use more natural syntax for filename and string tests. You can combine tests that are allowed for the test command using parentheses and logical operators.

**Listing 6. Using the** <nowiki>[[</nowiki> **compound**

<code>
[ian@pinguino ~]$ [[ ( -d "$HOME" ) && ( -w "$HOME" ) ]] &&  
>  echo "home is a writable directory"
home is a writable directory

</code>

The <nowiki>[[</nowiki> compound can also do pattern matching on strings when the = or != operators are used. The match behaves as for wildcard globbing as illustrated in Listing 7.

**Listing 7. Wildcard tests with** <nowiki>[[</nowiki>

<code>
[ian@pinguino ~]$ [[ "abc def .d,x--" == a[abc]*\ ?d* ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def c" == a[abc]*\ ?d* ]]; echo $?
1
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* ]]; echo $?
1
</code>

You can even do arithmetic tests within <nowiki>[[</nowiki> compounds, but be careful. Unless within a <nowiki>((</nowiki> compound, the < and > operators will compare the operands as strings and test their order in the current collating sequence. Listing 8 illustrates this with some examples.

**Listing 8. Including arithmetic tests with** <nowiki>[[</nowiki>

<code>
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || (( 3 > 2 )) ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || 3 -gt 2 ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || 3 > 2 ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || a > 2 ]]; echo $?
0
[ian@pinguino ~]$ [[ "abc def d,x" == a[abc]*\ ?d* || a -gt 2 ]]; echo $?
-bash: a: unbound variable
</code>


===== Conditionals =====

While you could accomplish a huge amount of programming with the above tests and the && and || control operators, bash includes the more familiar "if, then, else" and case constructs. After you learn about these, you will learn about looping constructs and your toolbox will really expand.



==== If, then, else statements ====

The bash if command is a compound command that tests the return value of a test or command ($?) and branches based on whether it is True (0) or False (not 0). Although the tests above returned only 0 or 1 values, commands may return other values. Learn more about these in the LPI exam 102 prep: Shells, scripting, programming, and compiling tutorial.

The if command in bash has a then clause containing a list of commands to be executed if the test or command returns 0, one or more optional elif clauses, each with an additional test and then clause with an associated list of commands, an optional final else clause and list of commands to be executed if neither the original test, nor any of the tests used in the elif clauses was true, and a terminal fi to mark the end of the construct.

Using what you have learned so far, you could now build a simple calculator to evaluate arithmetic expressions as shown in Listing 9.

**Listing 9. Evaluating expressions with if, then, else**

<code>
[ian@pinguino ~]$ function mycalc ()
> {
>   local x
>   if [ $# -lt 1 ]; then
>     echo "This function evaluates arithmetic for you if you give it some"
>   elif (( $* )); then
>     let x="$*"
>     echo "$* = $x"
>   else
>     echo "$* = 0 or is not an arithmetic expression"
>   fi
> }
[ian@pinguino ~]$ mycalc 3 + 4
3 + 4 = 7
[ian@pinguino ~]$ mycalc 3 + 4**3
3 + 4**3 = 67
[ian@pinguino ~]$ mycalc 3 + (4**3 /2)
-bash: syntax error near unexpected token `('
[ian@pinguino ~]$ mycalc 3 + "(4**3 /2)"
3 + (4**3 /2) = 35
[ian@pinguino ~]$ mycalc xyz
xyz = 0 or is not an arithmetic expression
[ian@pinguino ~]$ mycalc xyz + 3 + "(4**3 /2)" + abc
xyz + 3 + (4**3 /2) + abc = 35
        
</code>

The calculator makes use of the local statement to declare x as a local variable that is available only within the scope of the mycalc function. The let function has several possible options, as does the declare function to which it is closely related. Check the man pages for bash, or use help let for more information.

As you saw in Listing 9, you need to make sure that your expressions are properly escaped if they use shell metacharacters such as (, ), *, >, and <. Nevertheless, you have quite a handy little calculator for evaluating arithmetic as the shell does it.

You may have noticed the else clause and the last two examples in Listing 9. As you can see, it is not an error to pass xyz to mycalc, but it evaluates to 0. This function is not smart enough to identify the character values in the final example of use and thus be able to warn the user. You could use a string pattern matching test such as

<code>
[[ ! ("$*" == *[a-zA-Z]* ]]
</code>

(or the appropriate form for your locale) to eliminate any expression containing alphabetic characters, but that would prevent using hexadecimal notation in your input, since you might use 0x0f to represent 15 using hexadecimal notation. In fact, the shell allows bases up to 64 (using base#value notation), so you could legitimately use any alphabetic character, plus _ and @ in your input. Octal and hexadecimal use the usual notation of a leading 0 for octal and leading 0x or 0X for hexadecimal. Listing 10 shows some examples.

**Listing 10. Calculating with different bases**

<code>
[ian@pinguino ~]$ mycalc 015
015 = 13
[ian@pinguino ~]$ mycalc 0xff
0xff = 255
[ian@pinguino ~]$ mycalc 29#37
29#37 = 94
[ian@pinguino ~]$ mycalc 64#1az
64#1az = 4771
[ian@pinguino ~]$ mycalc 64#1azA
64#1azA = 305380
[ian@pinguino ~]$ mycalc 64#1azA_@
64#1azA_@ = 1250840574
[ian@pinguino ~]$ mycalc 64#1az*64**3 + 64#A_@
64#1az*64**3 + 64#A_@ = 1250840574

</code>

Additional laundering of the input is beyond the scope of this tip, so use your calculator with care.

The elif statement is very convenient. It helps you in writing scripts by allowing you to simplify the indenting. You may be surprised to see the output of the type command for the mycalc function as shown in Listing 11.

**Listing 11. Type mycalc**

<code>
[ian@pinguino ~]$ type mycalc
mycalc is a function
mycalc ()
{
    local x;
    if [ $# -lt 1 ]; then
        echo "This function evaluates arithmetic for you if you give it some";
    else
        if (( $* )); then
            let x="$*";
            echo "$* = $x";
        else
            echo "$* = 0 or is not an arithmetic expression";
        fi;
    fi
}

</code>

Of course, you could just do shell arithmetic by using <nowiki>$((</nowiki> expression <nowiki>))</nowiki> with the echo command as shown in Listing 12. You wouldn't have learned anything about functions or tests that way, but do note that the shell does not interpret metacharacters, such as *, in their normal role when inside <nowiki>((</nowiki> expression <nowiki>))</nowiki> or <nowiki>[[</nowiki> expression <nowiki>]]</nowiki>.

**Listing 12. Direct calculation in the shell with echo and** <nowiki>$(( ))</nowiki>

<code>
[ian@pinguino ~]$  echo $((3 + (4**3 /2)))
35

</code>


====== More Conditionals ======
Bash Conditional Expressions

Conditional expressions are used by the [[ compound command and the test and [ builtin commands.

Expressions may be unary or binary. Unary expressions are often used to examine the status of a file. There are string operators and numeric comparison operators as well. If the file argument to one of the primaries is of the form /dev/fd/N, then file descriptor N is checked. If the file argument to one of the primaries is one of /dev/stdin, /dev/stdout, or /dev/stderr, file descriptor 0, 1, or 2, respectively, is checked.

Unless otherwise specified, primaries that operate on files follow symbolic links and operate on the target of the link, rather than the link itself.






^  Table 3. More tests  ^^^
|  Operator  |  Characteristic  |
|  -a file  | True if file exists.  |
|  -b file  | True if file exists and is a block special file.  |
|  -c file  | True if file exists and is a character special file.  |
|  -d file  | True if file exists and is a directory.  |
|  -e file  | True if file exists.  |
|  -f file  | True if file exists and is a regular file.  |
|  -g file  | True if file exists and its set-group-id bit is set.  |
|  -h file  | True if file exists and is a symbolic link.  |
|  -k file  | True if file exists and its "sticky" bit is set.  |
|  -p file  | True if file exists and is a named pipe (FIFO).  |
|  -r file  | True if file exists and is readable.  |
|  -s file  | True if file exists and has a size greater than zero.  |
|  -t fd  | True if file descriptor fd is open and refers to a terminal.  |
|  -u file  | True if file exists and its set-user-id bit is set.  |
|  -w file  | True if file exists and is writable.  |
|  -x file  | True if file exists and is executable.  |
|  -O file  | True if file exists and is owned by the effective user id.  |
|  -G file  | True if file exists and is owned by the effective group id.  |
|  -L file  | True if file exists and is a symbolic link.  |
|  -S file  | True if file exists and is a socket.  |
|  -N file  | True if file exists and has been modified since it was last read.  |
|  file1 -nt file2  | True if file1 is newer (according to modification date) than file2, or if file1 exists and file2 does not.  |
|  file1 -ot file2  | True if file1 is older than file2, or if file2 exists and file1 does not.  |
|  file1 -ef file2  | True if file1 and file2 refer to the same device and inode numbers.  |
|  -o optname  | True if shell option optname is enabled. The list of options appears in the description of the -o option to the set builtin (see The Set Builtin).  |
|  -z string  | True if the length of string is zero.  |
|  -n string  | True if the length of string is non-zero.  |
|  string1 == string2  | True if the strings are equal. ‘=’ may be used in place of ‘==’ for strict posix compliance.  |
|  string1 != string2  | True if the strings are not equal.  |
|  string1 < string2   | True if string1 sorts before string2 lexicographically in the current locale.  |
|  string1 > string2   | True if string1 sorts after string2 lexicographically in the current locale.  |
|  arg1 OP arg2  | OP is one of ‘-eq’, ‘-ne’, ‘-lt’, ‘-le’, ‘-gt’, or ‘-ge’. These arithmetic binary operators return true if arg1 is equal to, not equal to, less than, less than or equal to, greater than, or greater than or equal to arg2, respectively. Arg1 and arg2 may be positive or negative integers.   |

====== Date calculation ======

Following example check the expiring date for a domain and prints out how long until renewal is required. If the there is less than 30 days left it will send a mail alert:

<file>
#!/bin/bash
TO="user@example.com"
FROM="root@example.com"

date2stamp () {
    date --utc --date "$1" +%s
}

stamp2date (){
    date --utc --date "1970-01-01 $1 sec" "+%Y-%m-%d %T"
}

dateDiff (){
    case $1 in
        -s)   sec=1;      shift;;
        -m)   sec=60;     shift;;
        -h)   sec=3600;   shift;;
        -d)   sec=86400;  shift;;
        *)    sec=86400;;
    esac
    dte1=$(date2stamp $1)
    dte2=$(date2stamp $2)
    diffSec=$((dte2-dte1))
    if ((diffSec < 0)); then abs=-1; else abs=1; fi
    echo $((diffSec/sec*abs))
}
mail (){
DOMAIN="$1"
OWNER="$2"
DAYSLEFT="$3"
TO="$4"
FROM="$5"

cp /root/domain_check.txt /tmp/domain_check.$$

sed -e "s/DOMAIN/$DOMAIN/g" -e "s/OWNER/$OWNER/g" -e "s/DAYSLEFT/$DAYSLEFT/g" -e "s/TO/$TO/g" -e "s/FROM/$FROM/g" /tmp/domain_check.$$ | sendmail -i -f $FROM -- $TO

}

OWNER=$(whois $1 | grep -i owner | awk '{print ($2,$3)}')
EXPIREDATE=$(whois $1 | grep -i expires | awk '{print $2'})
TODAY=$(date "+%Y-%m-%d")
DAYSLEFT=$(dateDiff -d $TODAY $EXPIREDATE)

echo $1 is owned by $OWNER for the next $DAYSLEFT

if [ $DAYSLEFT -lt 3000 ]
then
	mail $1 "$OWNER" $DAYSLEFT $TO $FROM
fi
</file>

And mail template file domain_check.txt:

<file>
From: "din System Administrator" <FROM>
MIME-Version: 1.0
To: TO
Subject: =?ISO-8859-1?Q?Dom=E6ne_Alarm_for_DOMAIN!!?=
Content-Type: multipart/alternative;
 boundary="------------020602090302090208010701"

This is a multi-part message in MIME format.
--------------020602090302090208010701
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Husk forny domænet DOMAIN

Der er DAYSLEFT dage tilbage

Ejer: OWNER


--------------020602090302090208010701
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
  </head>
  <body text="#000000" bgcolor="#ffffff">
    <div align="center"><big><big><big><small>Husk forny dom&aelig;net
              DOMAIN<br>
              <br>
            </small></big></big></big>
      <div align="left">Der er DAYSLEFT dage tilbage<br>
        <br>
        Ejer: OWNER<br>
        <br>
      </div>
      <big><big><big></big></big></big></div>
  </body>

</html>

--------------020602090302090208010701--
</file>

====== Extract device name by 'by-id' ======

<code>
ls -l /dev/disk/by-id | grep "usb-HD_Elements_1023_575884314193303030383613-0:0"

# lrwxrwxrwx 1 root root  9 2011-03-11 13:33 usb-HD_Elements_1023_575884314193303030383613-0:0 -> ../../sdc

DEVICENAME=$(ls -l /dev/disk/by-id/ | grep "usb-HD_Elements_1023_575884314193303030383613-0:0" | awk '{print $10}' | sed -e 's/\.\.\/\.\.\///')
echo $DEVICENAME

# sdc
</code>

====== Case example with diskspace ======

<file>
#!/bin/bash
# Check for Disk Usage
space=`df -h | awk '{print $5}' | grep % | grep -v Use | sort -n | tail -1 | cut -d "%" -f1 -`
case $space in
[1-6]*)
Message="Disk Space is OK"
;;
[7-8]*)
Message="Disk Space Filling Up  There's a partition that is $space % full."
;;
9[1-8])
Message="Disk Space is Critical ...  One partition is $space % full."
;;
99)
Message="Disk Emergency!!!!!!!!!!!  There's a partition at $space %!"
;;
*)
Message="NO DISK SPACE LEFT........"
;;
esac
echo $Message
</file>

====== Speedtester based on filesize growth ======
First commandline parameter is the path to the file we want to watch.
Second one is how many cycles we want to test the speed.

<file>
#!/bin/bash
x=$2
while [ $x -ne 0 ]
do
    min=$(ls -l "$1" | awk '{print $5'})
    sleep 1
    max=$(ls -l "$1" | awk '{print $5'})
    echo "($max-$min) / 1024" | bc 
    let x--
done
</file>




====== Remove empty lines with sed ======


I have text files with tons of empty lines, how do I get rid of those in one second?
 

 
 
For html or php files that I'll post on the net, I like to get rid of empty lines. Usually, they are leftovers of search and replace and I have never been able to get any of the replace functions of the editors I use to get rid of them.
Now, sed is your friend... Written a long time ago for Unix systems, sed has been ported to a variety of operating systems, including Linux. It's is a non-interactive editor that works from a command line, with no GUI, so you do not waste time opening up windows, and clicking all over the place.
So, open up a console and move into the directory where your file resides (cd MyDirectory). And here we go with the two lines that'll do the job

<code>
sed '/^$/d' myFile > tt 
mv tt myFile
</code>


Here is what happens:

sed '/^$/d' myFile removes all empty lines from the file myFile and outputs the result in the console,

> tt redirects the output into a temporary file called tt,

mv tt myFile moves the temporary file tt to myFile.

Now, you may have 100 html files to correct at the same time. That's where foreach comes in... Let's say you want to correct all files ending with .html, here is what you should do:
open up a console, move into the directory where your html files reside, type the following commands:

<code>
foreach file (*html)
sed '/^$/d' $file > tt
mv tt $file
end
</code>

Finished!




----
---- 

^ **Sources:** ^      
| http://www-128.ibm.com/developerworks/linux/library/l-bash-test.html?ca=dgr-lnxw02LinuxBashTest    |
| http://www.gnu.org/software/bash/manual/bashref.html    |
| http://www.unix.com/tips-tutorials/31944-simple-date-time-calulation-bash.html    |
| http://bashshell.net/shell-scripts/case-statement/ |
| http://soft.zoneo.net/Linux/remove_empty_lines.php|