Shell scripting - cheatsheet
Here’s a cheatsheet for shell scripting including some tips and tricks, where and how to use it.
Introduction
There are many different shells out there - sh
, bash
, zsh
, fish
and csh
just to name some popular ones. Each has its variances and scripts that work on bash
may not work on sh
or fish
. These snippets use code that targets my personal shell choice zsh
but TRY to remain POSIX compatible.
Please note: This cheatsheet is not meant for beginners. You should always try to understand what is happening before you copy and paste.
Basics
Header line
#!/usr/bin/env bash
#!/bin/sh
#!/usr/bin/env zsh
suppress all output (including errors)
echo "suppressed" 2>&1 > /dev/null
if then else (see if then else reference )
# [[...]] would be non POSIX, even if it is preferred for bash/zsh!
if [ -d "${DIR}" ]; then
echo "${DIR} exists"
elif [ -d "${DIR2}" ]; then
echo "${DIR2} exists"
else
echo "${DIR} and ${DIR2} missing"
fi
# short version
[ -d "${DIR}" ] && ( echo "${DIR} exists" ) || ( echo "${DIR} missing" );
# if NOT (negation)
[ ! -d "${DIR}" ] && ( echo "${DIR} missing" );
case
day=$(date +%w)
case "${day}" in
0|\
7) echo "Weekend - yay!"
;;
6) echo "Almost made it!"
;;
*)
echo "Oh no..."
;;
esac
loops - for, while, continue and break
for i in 1 2 3 4 5; do
echo "$i"
done
i=0
while true; do
i=$((i+1))
if [ $i -lt 2 ] && (continue)
echo $i
[ $i -gt 3 ] && (break)
done
functions
function append {
echo "${1}${2}" # returning strings only with echo
return 0 # real return value stored in $?
}
VALUE=$(append "123" "456")
echo $?
Files
write output to files
#truncate file (empty file is kept)
> /tmp/test.txt
# overwrite existing contents
echo "test-overwrite" > /tmp/file.txt
# append content
echo "test-append" >> /tmp/file.txt
# append multiple lines
cat <<EOF >> /tmp/file.txt
multiple
lines
EOF
# overwrite contents with multiple lines and sudo permissions
sudo tee /tmp/file.txt > /dev/null <<EOT
multiple
lines
EOT
# append multiple lines with sudo permissions
sudo tee -a /tmp/file.txt > /dev/null <<EOT
multiple
lines
EOT
search file contents
# search for pattern in file
grep 'pattern' file
# search for pattern recursively in my_directory
grep -R 'pattern' my_directory
# search for multiple patterns
grep 'pattern1\|pattern2' file
# search for NON matches (invert search)
grep -v 'pattern' file
# search without printing found matches
grep -q 'pattern' file && (echo "pattern found")
inplace editiing with sed
# comment out matching line (dtparam=audio=on => #dtparam=audio=on)
FILE_TO_REPLACE="/boot/config.txt"
sed -i 's/^dtparam=audio=on/#dtparam=audio=on/g' "${FILE_TO_REPLACE}"
# replace matching line (value=hello => value=replaced)
FILE_TO_REPLACE="/tmp/test.txt"
sed -i 's/^value=.*/value=replaced/g' "${FILE_TO_REPLACE}"
# append line AFTER match (dtoverlay=hifiberry-dac after #dtparam=audio=on)
FILE_TO_REPLACE="/boot/config.txt"
sed -i '/^#dtparam=audio=on/a dtoverlay=hifiberry-dac/' "${FILE_TO_REPLACE}"
# insert a line BEFORE match (dtoverlay=hifiberry-dac before #dtparam=audio=on)
FILE_TO_REPLACE="/boot/config.txt"
sed -i '/^#dtparam=audio=on/i dtoverlay=hifiberry-dac/' "${FILE_TO_REPLACE}"
# insert multiline before a match (using `\n`)
FILE_TO_REPLACE="/boot/config.txt"
sed -i '/^#dtparam=audio=on/i dtoverlay=hifiberry-dac\n#another=value/' "${FILE_TO_REPLACE}"
mkdir if not exists
[ -d "${DIR}" ] || (mkdir -p "${DIR}")
change dir, run command and change back
cd /tmp/ && touch "testfile.txt" && cd -
loop over files in directory
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
FILES=/data/*
for f in $FILES; do
echo "$f"
done
IFS=$SAVEIFS # restore $IFS
loop over files with find
find . -print0 | while read -d $'\0' file; do
echo -v "$file"
done
find duplicate files
# linux only (unix uniq works different)
# skip 0 byte files (! -empty)
# only files, not dirs (-type f)
# create md5 checksum (md5sum {})
# sort by md5 checksum
# compare first 32 chars (only hash, exclude filename) of only duplicate lines and all of them (uniq -w32 -dD)
find "." ! -empty -type f -exec md5sum {} + | sort | uniq -w32 -dD
Math
Arithmetic (increment / decrement / calculate)
# simple operations (only integers)
i=0
i=$((i+1))
i=$((i-1))
# more complex (including floats)
var=$(echo "1+2"|bc)
String manipulation
concat strings
ASTRING="a-string-with-$VARIABLE"
heredoc simple
cat <<EOF
testing
EOF
heredoc with pipes
(
cat <<EOF
testing
EOF
) | grep 'testing'
match text with heredoc
# (result: 1 for match, 0 for no match)
grep -oc 'yes' <<< "Available: yes"
squeeze multiple spaces
tr -s ' ' <<< "this is a test"
left pad a string (with spaces)
PADLEN=15
echo "to_pad" | sed -e :a -e "s/^.\{1,$PADLEN\}\$/ &/;ta" ;
Misc
Count lines (e.g. jpg files in dir)
ls -l *.jpg | wc -l
find
examples
use a function for find -exec
export -f my_function
find . -name '*.jpg' -exec bash -c 'myfunction "$0"' {} \;
find multiple extensions
find . -iregex '.*\.\(jpg\|svg\|gif\|png\|jpeg\)$'
find invert match (e.g. files that are not mp3)
find . -name '*' -type f -not -path '*.mp3'
Utility reference
Name | Description | Example |
---|---|---|
awk | pattern scanning and string manipulation (complex!) | awk '{print}' cooking.txt |
cat | concatenate files and print on the standard output | cat/tmp/file.txt |
cp | copy files and directories | cp a.txt b.txt |
curl | transfer a URL. | curl -O http://example.com/file.jpg |
cut | cut out selected portions of each line of a file | cut -d ' ' -f 2 <<< "Hello World" |
diff | compare files line by line | diff a.txt b.txt |
du | estimate file space usage | du -sh * |
echo | display a line of text | echo "test" |
find | search for files in a directory hierarchy | find . -name '*.mp3 |
grep | Print lines matching a pattern | grep 'search' file.txt |
kill | send a signal to a process | kill -HUP 801 |
ls | list directory contents | ls -a /tmp |
mv | move files and directories | move a.txt b.txt |
paste | merge lines of files | paste id.txt name.txt email.txt |
patch | apply a diff file to an original | patch < insert_lines.patch |
printf | format and print data | printf "Open: %s\nClosed: %s\n" "34" "65" |
rsync | file transfer utility | rsync -a src/ destination/ |
rm | remove files or directories | rm /tmp/file-to-remove.txt |
sed | stream editor (e.g. for string replacements) | sed 's/Hello/Hi/' <<< "Hello World" |
seq | print a sequence of numbers | seq 5 |
sleep | delay for a specified amount of time | sleep 2 |
sort | sort lines of text files | sort file.txt |
tee | read from input and write to output and files | `ls |
tr | translate characters | tr -s 'Hello' 'Hi' <<< 'Hi' |
uname | print system information | uname -a |
wc | print newline, word, and byte counts for each file | `ls -l |
wget | The non-interactive network downloader. | wget http://example.com/file.jpg |
which | locate a program file in the user’s path | which sh |
xargs | append arguments when pipe support is missing | `ls *.txt |
if then else reference
For a complete reference see man test
.
Parameter | Description |
---|---|
[ -e "${FILE}" ] | file exists |
[ -s "${FILE}" ] | file exists and not empty |
[ -w "${FILE}" ] | file is writable by current user |
[ -d "${DIR}" ] | file is directory and exists |
[ -h "${LINK}" ] | file is symlink and exists |
[ -n "${STR}" ] | string is NOT empty |
[ -z "${STR}" ] | string is empty |
[ "${STR1}" = "${STR2}" ] | string STR1 equals STR2 |
[ "${STR1}" != "${STR2}" ] | string STR1 NOT equals STR2 |
[ ${INT1} -eq "${INT2}" ] | integer INT1 equals INT2 |
[ ${INT1} -ne "${INT2}" ] | integer INT1 NOT equals INT2 |
[ ${INT1} -lt "${INT2}" ] | integer INT1 lower than INT2 |
[ ${INT1} -le "${INT2}" ] | integer INT1 lower or equal than INT2 |
[ ${INT1} -gt "${INT2}" ] | integer INT1 greater than INT2 |
[ ${INT1} -ge "${INT2}" ] | integer INT1 greater or equal than INT2 |