Shell scripting - cheatsheet

Shell scripting - cheatsheet
Page content

Here’s a cheatsheet for shell scripting including some tips and tricks, where and how to use it.

Introduction

There are many different shells out there - sh, bash, zsh, fish and csh just to name some popular ones. Each has its variances and scripts that work on bash may not work on sh or fish. These snippets use code that targets my personal shell choice zsh but TRY to remain POSIX compatible.

Please note: This cheatsheet is not meant for beginners. You should always try to understand what is happening before you copy and paste.

Basics

Header line

#!/usr/bin/env bash
#!/bin/sh
#!/usr/bin/env zsh

suppress all output (including errors)

echo "suppressed" 2>&1 > /dev/null

if then else (see if then else reference )

# [[...]] would be non POSIX, even if it is preferred for bash/zsh!
if [ -d "${DIR}" ]; then
  echo "${DIR} exists"
elif [ -d "${DIR2}" ]; then
  echo "${DIR2} exists" 
else
  echo "${DIR} and ${DIR2} missing"
fi

# short version
[ -d "${DIR}" ] && ( echo "${DIR} exists" ) || ( echo "${DIR} missing" );

# if NOT (negation)
[ ! -d "${DIR}" ] && ( echo "${DIR} missing" );

case

day=$(date +%w) 
case "${day}" in
  0|\
  7) echo "Weekend - yay!" 
    ;;
  6) echo "Almost made it!" 
    ;;
  *)   
    echo "Oh no..."   
    ;;
esac

loops - for, while, continue and break

for i in 1 2 3 4 5; do
   echo "$i"
done

i=0
while true; do 
  i=$((i+1))
  if [ $i -lt 2 ] && (continue)
  echo $i
  [ $i -gt 3 ] && (break)
done

functions

function append { 
  echo "${1}${2}" # returning strings only with echo
  return 0 # real return value stored in $?
}
VALUE=$(append "123" "456")
echo $?

Files

write output to files

#truncate file (empty file is kept)
> /tmp/test.txt

# overwrite existing contents
echo "test-overwrite" > /tmp/file.txt

# append content
echo "test-append" >> /tmp/file.txt

# append multiple lines
cat <<EOF >> /tmp/file.txt
multiple
lines
EOF

# overwrite contents with multiple lines and sudo permissions
sudo tee /tmp/file.txt > /dev/null <<EOT
multiple
lines
EOT

# append multiple lines with sudo permissions
sudo tee -a /tmp/file.txt > /dev/null <<EOT
multiple
lines
EOT

search file contents

# search for pattern in file
grep 'pattern' file

# search for pattern recursively in my_directory
grep -R 'pattern' my_directory

# search for multiple patterns
grep 'pattern1\|pattern2' file

# search for NON matches (invert search)
grep -v 'pattern' file

# search without printing found matches
grep -q 'pattern' file && (echo "pattern found")

inplace editiing with sed

# comment out matching line (dtparam=audio=on => #dtparam=audio=on)
FILE_TO_REPLACE="/boot/config.txt"
sed -i 's/^dtparam=audio=on/#dtparam=audio=on/g' "${FILE_TO_REPLACE}"

# replace matching line (value=hello => value=replaced)
FILE_TO_REPLACE="/tmp/test.txt"
sed -i 's/^value=.*/value=replaced/g' "${FILE_TO_REPLACE}"

# append line AFTER match (dtoverlay=hifiberry-dac after #dtparam=audio=on) 
FILE_TO_REPLACE="/boot/config.txt"
sed -i '/^#dtparam=audio=on/a dtoverlay=hifiberry-dac/' "${FILE_TO_REPLACE}"

# insert a line BEFORE match (dtoverlay=hifiberry-dac before #dtparam=audio=on)
FILE_TO_REPLACE="/boot/config.txt"
sed -i '/^#dtparam=audio=on/i dtoverlay=hifiberry-dac/' "${FILE_TO_REPLACE}"

# insert multiline before a match (using `\n`)
FILE_TO_REPLACE="/boot/config.txt"
sed -i '/^#dtparam=audio=on/i dtoverlay=hifiberry-dac\n#another=value/' "${FILE_TO_REPLACE}"

mkdir if not exists

[ -d "${DIR}" ] || (mkdir -p "${DIR}")

change dir, run command and change back

cd /tmp/ && touch "testfile.txt" && cd -

loop over files in directory

SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
FILES=/data/*
for f in $FILES; do
  echo "$f"
done
IFS=$SAVEIFS # restore $IFS

loop over files with find

find . -print0 | while read -d $'\0' file; do
  echo -v "$file"
done

find duplicate files

# linux only (unix uniq works different)
# skip 0 byte files (! -empty)
# only files, not dirs (-type f)
# create md5 checksum (md5sum {})
# sort by md5 checksum
# compare first 32 chars (only hash, exclude filename) of only duplicate lines and all of them (uniq -w32 -dD)
find "." ! -empty -type f -exec md5sum {} + | sort | uniq -w32 -dD

Math

Arithmetic (increment / decrement / calculate)

# simple operations (only integers)
i=0
i=$((i+1))
i=$((i-1))

# more complex (including floats)
var=$(echo "1+2"|bc)

String manipulation

concat strings

ASTRING="a-string-with-$VARIABLE"

heredoc simple

cat <<EOF
testing
EOF

heredoc with pipes

(
cat <<EOF
testing
EOF
) | grep 'testing'

match text with heredoc

# (result: 1 for match, 0 for no match)
grep -oc 'yes' <<< "Available: yes"

squeeze multiple spaces

tr -s ' ' <<< "this  is   a    test"

left pad a string (with spaces)

PADLEN=15
echo "to_pad" | sed -e :a -e "s/^.\{1,$PADLEN\}\$/ &/;ta" ; 

Misc

Count lines (e.g. jpg files in dir)

ls -l *.jpg | wc -l

find examples

use a function for find -exec

export -f my_function
find . -name '*.jpg' -exec bash -c 'myfunction "$0"' {} \;

find multiple extensions

find . -iregex '.*\.\(jpg\|svg\|gif\|png\|jpeg\)$' 

find invert match (e.g. files that are not mp3)

find . -name '*' -type f -not -path '*.mp3'

Utility reference

NameDescriptionExample
awkpattern scanning and string manipulation (complex!)awk '{print}' cooking.txt
catconcatenate files and print on the standard outputcat/tmp/file.txt
cpcopy files and directoriescp a.txt b.txt
curltransfer a URL.curl -O http://example.com/file.jpg
cutcut out selected portions of each line of a filecut -d ' ' -f 2 <<< "Hello World"
diffcompare files line by linediff a.txt b.txt
duestimate file space usagedu -sh *
echodisplay a line of textecho "test"
findsearch for files in a directory hierarchyfind . -name '*.mp3
grepPrint lines matching a patterngrep 'search' file.txt
killsend a signal to a processkill -HUP 801
lslist directory contentsls -a /tmp
mvmove files and directoriesmove a.txt b.txt
pastemerge lines of filespaste id.txt name.txt email.txt
patchapply a diff file to an originalpatch < insert_lines.patch
printfformat and print dataprintf "Open: %s\nClosed: %s\n" "34" "65"
rsyncfile transfer utilityrsync -a src/ destination/
rmremove files or directoriesrm /tmp/file-to-remove.txt
sedstream editor (e.g. for string replacements)sed 's/Hello/Hi/' <<< "Hello World"
seqprint a sequence of numbersseq 5
sleepdelay for a specified amount of timesleep 2
sortsort lines of text filessort file.txt
teeread from input and write to output and files`ls
trtranslate characterstr -s 'Hello' 'Hi' <<< 'Hi'
unameprint system informationuname -a
wcprint newline, word, and byte counts for each file`ls -l
wgetThe non-interactive network downloader.wget http://example.com/file.jpg
whichlocate a program file in the user’s pathwhich sh
xargsappend arguments when pipe support is missing`ls *.txt

if then else reference

For a complete reference see man test.

ParameterDescription
[ -e "${FILE}" ]file exists
[ -s "${FILE}" ]file exists and not empty
[ -w "${FILE}" ]file is writable by current user
[ -d "${DIR}" ]file is directory and exists
[ -h "${LINK}" ]file is symlink and exists
[ -n "${STR}" ]string is NOT empty
[ -z "${STR}" ]string is empty
[ "${STR1}" = "${STR2}" ]string STR1 equals STR2
[ "${STR1}" != "${STR2}" ]string STR1 NOT equals STR2
[ ${INT1} -eq "${INT2}" ]integer INT1 equals INT2
[ ${INT1} -ne "${INT2}" ]integer INT1 NOT equals INT2
[ ${INT1} -lt "${INT2}" ]integer INT1 lower than INT2
[ ${INT1} -le "${INT2}" ]integer INT1 lower or equal than INT2
[ ${INT1} -gt "${INT2}" ]integer INT1 greater than INT2
[ ${INT1} -ge "${INT2}" ]integer INT1 greater or equal than INT2
Andreas Fuhrich avatar
About Andreas Fuhrich
I’m a professional software developer and tech enthusiast from Germany. On this website I share my personal notes and side project details. If you like it, you could support me on github - if not, feel free to file an issue :-)