Two ways of using scripts :
1) sh
2 bash

1) sh is available in any linux distrib. That implements the POSIX standard.
Advantage : strong compatibility
Drawback: cumbercome syntax (ex: compared elements has to be inside double quotes) and features limitations (ex: not regex possible in comparison, not array type…)

2) bash is available in almost any linux distrib. At the origin, that implements the POSIX standard but with the time bash has added multiple syntax improvements and features not compatible any more with the sh standard.
Advantage : better language than sh
Drawback: may not be installed on some distribs.
About that drawback, that was true some years ago ( for example in the 2000’s beginning). Today, that is not true : bash is very spread.
So but if you install a very particular/restricted linux distrib, you will have that available or at at least installable.

Bash syntax

bash syntax is if [[ ]] (while sh is if [ ]).
An advantage of bash syntax is that word-splitting and pathname expansion are not applied operands, so no need to enclose all operands with « » characters.
That is needed just for literals.
So we could write :

if [[ $result = "0" ]]; then
  exit
fi

As well as, we could chain logical operators in the same expression :

if [[ $1 = "cool"  &&  $2 = "red" ]]; then
    echo "Cool Red Beans"
fi

With sh, we would have written :

if [ "$1" = "cool" ] &&  [ "$2" = "red" ]; then
    echo "Cool Red Beans"
fi

Command evaluation

result=`myCommand bar ...` and result=$(myCommand bar ...) are equivalent.
These evaluate a linux expression.
The single difference is that $(myCommand bar ...) may be nested such as result=$(foo $(bar)) while backtick evaluations cannot.

Valid a bash file syntax

It validates the general syntax but will not check any path validity referenced in the script.
bash -n myFile.sh

Logic operators precedence

As for most of languages, operators with the same precedence are left-associative.
But && and || have the same precedence.
So the order of them is always which matters.

Program arguments

By calling : foo_app arg1 arg2 arg3 :
$0 equals to « foo_app »
$1 equals to « arg1 »
$2 equals to « arg2 »
$3 equals to « arg3 »
$@ stores all args. So here $@ equals to p »arg1 arg2 arg3″

Test the exit code of a command

If the command returns has a successful exit code (0), the expression is evaluated to true otherwise to false.
Test a successful exit code :

if ls fooDir; then    
    echo "the dir exists"
fi

Test a non-successful exit code :

if ! ls fooDir; then    
     echo "the dir does not exist"
fi

Function with retry in bash

The function expects 2 params : the nb of try and the nb of seconds to wait between the tries.
Here we use the function return with the output way (echo).
– no output means success, any output means failure.
– the output is the error message.
Here the function and how to use it :

function foo_with_retry(){
    nb_try=$1
    wait_sec=$2
    for i in $(seq 1 $nb_try);
      do
        # we get only the http response code of curl
        local foo_code_response=$(curl -w %{http_code} -k --silent -L -o /dev/null   
                                 "https://foo.com/test")
        if (( foo_code_response >= 200 && foo_code_response <= 299 )); then
            echo ""
            return 0
        fi
        sleep "${wait_sec}s"
    done
    echo "Test failed after ${nb_try} tries ! Last server response code=${foo_code_response}"
}
 
 
# calling the function
error_msg=$(foo_with_retry 3 60)
if [[ -n $error_msg ]]; then
   echo "$error_msg"
   # failure processings
else
  echo "Test passed"
   # success processings
fi

Operations on processes

Kill a process and wait for its termination with a timeout

kill -9 $foo_pid 
timeout 10s tail -f --pid=$foo_pid /dev/null
if [[ $? -ne 0 ]]; then
  echo "don't manage to kill the process $foo_pid"
fi

Arithmetic operations

Sum/Multiply/Substract/Divide two numbers
General syntax : $((FOO_NUMBER_OR_VAR SIGN FOO_NUMBER_OR_VAR))
Ex:
– echo the sum of two number variables: echo $(($FOO + $BAR)) - echo the sum of a variable and a constant: echo $(($FOO + 15))

String comparison (literals, parameters/variables)

if [[ $1 = "cool" ]]
then
    echo "Cool Beans"
elif [[ $1 = "neat" ]]
then
    echo "Neato cool"
else
    echo "Not Cool Beans"
fi

string comparison : logical and/or

if [[ $1 = "cool" && $2 = "red" ]]
then
    echo "Cool Red Beans"
fi

string comparison : wildcard

if [[ $1 != *"-ool" ]]
then
    echo "Cool Beans"
fi

string comparison : one line

if [[ "$1" = "cool" ]]; then echo "Cool Beans"; elif [[ "$1" = "neat" ]]; then echo "Neato cool"; else echo "Not cool Bean"; fi

numbers comparison by using an arithmetic context ((…)).

May not work with sh, work with bash
Possible comparison :
==, !=, >, >=, <, <=
Example :

if (( a > b ))
then
    ...
fi

We could also combine arithmetic contexts with OR/AND such as :

if (( a > b && a < 1000 ))
then
    ...
fi

numbers comparison (POSIX compatible)

fooVar=5
if [ $fooVar -eq 5 ]; then
   echo "fooVar equals 5"
fi
if [ $fooVar -ne 10 ]; then
   echo "fooVar not equals 10"  
fi
if [ $fooVar -gt 4 ];then
   echo "fooVar is > 4"
fi
if [ $fooVar -lt 6 ];then
   echo  "fooVar is < 6"
fi

string emptiness test

fooVar=...
if [[ -n $fooVar ]] ; then
  echo "not empty"
fi
if [[ -z $fooVar ]] ; then
  echo "empty"
fi

string emptiness (spaces excluded) test

We combine the -z (empty test) with the string value without its whitespaces :

if [[ -z "${fooVar// }" ]] ; then
 echo "empty or only spaces"
fi

Toggles for script:

set -x : to enable debug directly in the script
bash -x ./foo.sh : to enable debug outside the script

-e : exit script on some specific errors.
BEWARE:
The shell does not exit if the command that fails is :
– part of the command list immediately following a while or until keyword
– part of the test in an if statement
– part of any command executed in a && or || list except the command following the final && or ||
– any command in a pipeline but the last
– or if the command’s return status is being inverted with !.
Source: href= »https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html#:~:text=The%20shell%20does%20not%20exit,last%2C%20or%20if%20the%20command’s »>GNU documentation

To disable the flag for a command: we must chain the command with && or || and another command.
For example a canonical way to do that :

ret=0
myCommand || ret=$?
if (( $ret == 0 )); then	
   echo "myCommand was successful"
fi

As alternative, if the command outputs some text only for the nominal case and besides we need that text value in further commands, we could use the command output to identify the case :

foocmdOutput=$(fooCmd that may return a not 0 exit code) || echo ""
if [[ -n $foocmdOutput ]]; then
  echo "fooCmd has something"
fi

compound comparison

-a : logical and
exp1 -a exp2 returns true if both exp1 and exp2 are true.

-o : logical or
exp1 -o exp2 returns true if either exp1 or exp2 is true.

Test the existence/nonexistence of a file/directory

File existence with a file named « myFile » :

if [[ -f myFile ]] ; then
  ...
fi

Variant with a variable to define the file :

if [[ -f $myFile ]] ; then
  ...
fi

File nonexistence :

if [[ ! -f myFile ]] ; then
  ...
fi

For directory, use the flag -d instead.

The test command

Syntax : test -FLAG EXPRESSION
If the expression is true, 0 exit code is returned, other that is different from 0.
It allows to check file types or to compare values.
The advantage : not need to use conditional expression to exit a script.

WARNING :
When we use variable in the expression, we have to enclose them with » because blank value may produce unexpected result.
For example : test -d dir returns an 0 exit code if dir exist and is a directory but without passing any dir param such as test -d also returns 0 !

Flag examples :
( EXPRESSION )
EXPRESSION is true

! EXPRESSION
EXPRESSION is false

-n STRING
the length of STRING is nonzero
example:test -n "${field_to_extract}"

-z STRING
the length of STRING is zero

STRING1 = STRING2
the strings are equal

STRING1 != STRING2
the strings are not equal

-e FILE
FILE exists

-d FILE
FILE exists and is a directory

-f FILE
FILE exists and is a regular file

User-defined function in Bash

Return an exit code

The return keyword used in a function doesn’t return a value to the caller but returns the exit code value of the function.
Returning an exist code can be helpful if a numeric value match to the caller function need.
But beware a non zero exit code of a function is considered by bash as an error beyond the scope of the function.
It means that if you enabled the -e flag to stop the bash at first error, a return function with non zero value will stop the whole script.
Example :

function doThat(){
  echo "doThat()"
  if [[ $1 == "ok" ]]; then
    # do anything ...
    return 0
  fi
 return 1
}
 
set -e
doThat "ok"
echo "exit code of doThat=$?"
echo "Printed"!
 
doThat "other"
echo "never printed"

Output

doThat()
exit code of doThat=0
Printed!
doThat()

Return a value to the caller

Bash has no official way for a functions to return a value to its caller.
But there is some tricks.
One of them is writing the result of the function in the std output.
Example (here the local variable is only used to make things clean):

function getFoo(){
   local foo # local var
   foo="hello"
   echo "$foo"
}
 
gFoo=$(getFoo)
echo "foo=$foo"
echo "gfoo=$gFoo"

Set a global var in a function

By default a variable is global.
Here we set the foo variable only if the input param is ok.
If not the case, we exit the script.

function setFoo(){
  # global var foo
  if [[ "$1" == "a" ]]; then
     foo="hello"
  elif [[ "$1" == "b" ]]; then
     foo="hola"
  else
    echo "Error : cannot value foo"
    exit 1
  fi
}
 
setFoo a
echo "foo=$foo"
setFoo b
echo "foo=$foo"
setFoo c
echo "foo=$foo"

Iteration

while Iteration

General syntax :

while [[ conditions ]]; do
  echo ...
  run...
done

for Iteration

General syntax

One line:

for word in words; do commands; done

Multi line:

for word in words; do 
   commands
done

Here words is a sequence of words that is expanded if required and we iterate on each token with word as current variable.

*Iterate on a sequence of words
Example with a sequence of numbers :

values=$(seq 1 5)
for value in $values; do echo $value; done
#output
1
2
3
4
5

Example with a sequence of strings space-separated:

values="dog cat lion"
for value in $values; do echo $value; done
#output
dog
cat
lion

* Inlining the sequence of numbers in the for expression:
Example : iterate 10 times

for i in $(seq 1 10); do
  echo $i
done

*Iterate on a variable split by a separator
Here the separator is - but it could be another thing.
Here the idea is to create a sequence of words (strings here) white-separated as seen above. Here : // means global replace.
Warn : To not use with elements containing spaces. Indeed with that solution an element containing a space will be considered as two token in the loop.

foo=abc-def-ghi
for i in ${foo//-/ }
do
    echo "$i"
done

An alternative is storing the result of the split into an array :

myVar="foo;bar"
myArr=(${myVar//;/ })
echo ${myArr[0]}
echo ${myArr[1]}

*Iterate on an array of string

declare -a animals=("lion" "dog" "cat" "mouse" 
                    "bull" "aligator")
 
for animal in "${animals[@]}"
do   
   echo "current animal : $animal"   
done

*Iterate on files and directories (no recursive)
Example : Iterate on no hidden files and directory of the current directory

for file in ./*; do
  if [[ -f $file ]]; then
    # it is a file
  fi
  if [[ -d $file ]]; then
    # it is a directory
  fi
done

Variant with the directory to iterate as a variable:

dir="/etc"
for file in $dir/*; do  
     # ...
done

Variant by including even hidden files :

# enable bash to includes filenames beginning with a ‘.’ in the results of filename expansion
shopt -s dotglob
dir="/etc"
for file in $dir/*; do
    # ...
done

Deletion of directories inside a script

Example of a safe deletion :

# can be / but could be /etc/ or any sensitive dir such as /foo-important-dir/
ROOT_FOLDER="/" 
 
# CHILD_FOLDER is a variable designed to be subdirectory of ROOT_FOLDER
# We must be sure that it is before deleting it !
 
# we strip all whitespaces of CHILD_FOLDER
CHILD_FOLDER="${CHILD_FOLDER// /}"
# we assert that CHILD_FOLDER is not empty or an empty string but reference something
if [[ "${BUILD_DIRECTORY}/${CHILD_FOLDER}" = "${BUILD_DIRECTORY}/" ]]; then
	echo 'ERROR : CHILD_FOLDER has to be set'
	exit 1
fi
 
# Ok : it is safe to delete folder referencing $CHILD_FOLDER now
rm -rf ${BUILD_DIRECTORY}/${CHILD_FOLDER}

Misc command, variables

dirname : strip last component from file name
ex:
dirname foo/bar/file
output : foo/bar
Inside scripts, we use that as :
RELATIVE_BASEDIR=$(dirname "$0")

readlink path : print absolute path of a symbolic link or a file/folder
Flags :
-f, –canonicalize : all but the last component must exist
-e, –canonicalize-existing : all components must exist
-m, –canonicalize-missing : no requirements on components existence

Inside scripts, we may use that as :
RELATIVE_BASEDIR=$(dirname "$0")
BASEDIR=$(readlink -f $RELATIVE_BASEDIR)
$0 variable :
In shell : « bash » value
In script : the exact path of the script that was invoked.
Ex:
The script is invoked from ./ so $0 equals the script name.
The script is invoked from a parent folder such as foo/bar.sh so $0 equals foo/bar.sh.

Read a line from the std input and split it into fields

– read with a prompt message and store the input in a single var :
read -p "Please, ...: " varToAssignTheInput

– read with a (p)rompt message, a interactive shell for (e)diting and an (i)nitial value and store the input in a single var :
read -e -p "Please, ...: " -i "fooInitialValue" varToAssignTheInput

Rename files with an extension to another one

For example we want to rename with the .srt, extension to the .srt extension. Since we want just to strip the last character of the original extension substring is enough:
find -name '*.srt,' -exec bash -c 'file="{}";x="${file:0:-1}"; echo "${x}"; mv "${file}" "${x}" ' \;

Write Bash Scripts