open atlas
↑ Back to track
Command line CLI · 08 · 03

Loops and arguments

for iterates over a list; while iterates until a condition fails. $@ expands to all positional arguments as separate words; $* joins them. read processes stdin line by line. shift discards $1 and renumbers the rest.

CLI Middle ◷ 22 min
Level
FoundationsJuniorMiddleSenior

A script that processes one file is a start. A script that processes every file in a directory — correctly, even when filenames contain spaces or start with dashes — is what engineers actually ship. Loops and argument handling are where shell scripting earns its keep: they turn a one-liner into a reusable tool. But the pitfalls cluster here too. $* vs "$@" looks like a trivia question until you pass a filename with a space and watch your script silently corrupt data. read without -r mangles backslashes. shift done wrong drops arguments on the floor. This lesson covers the correct forms.

Goal

After this lesson you can write for loops over lists and globs, write while read loops over stdin, access script arguments with $1, $#, "$@", use shift to consume arguments, and understand why "$@" is always preferred over $*.

1

for item in list; do ... done iterates over a whitespace-separated list or a glob. The list is word-split and glob-expanded before the loop starts.

for color in red green blue; do
  echo "color: $color"
done

# Iterate over files — the glob expands before the loop
for f in /var/log/*.log; do
  echo "Processing $f"
  wc -l "$f"
done

When the glob matches nothing, $f holds the literal string /var/log/*.log by default. Add shopt -s nullglob at the top of your script to make non-matching globs expand to nothing instead.

For a numeric range, use brace expansion or C-style syntax:

for i in {1..5}; do echo "$i"; done
for (( i=0; i<5; i++ )); do echo "$i"; done
2

$1, $2, … hold positional arguments. $0 is the script name. $# is the count. "$@" expands each argument as a separate quoted word. Always use "$@" — never $* — when forwarding arguments to another command.

#!/usr/bin/env bash
echo "Script: $0"
echo "Arg count: $#"
echo "First arg: $1"

# Forward all arguments safely
rsync -av "$@" /backup/

"$@" with an argument my file.txt produces one word: my file.txt. $* (unquoted) or "$*" (quoted) joins all arguments into a single string — fine for printing, wrong for forwarding.

# With args: "file one.txt" "file two.txt"
for f in "$@"; do cp "$f" /backup/; done   # correct — 2 iterations
for f in $@;  do cp "$f" /backup/; done   # WRONG — splits on spaces → 4 iterations
3

shift discards $1 and renumbers remaining arguments: old $2 becomes new $1, etc. Use it to consume arguments one by one in option-parsing loops.

#!/usr/bin/env bash
verbose=0
output=""

while [[ $# -gt 0 ]]; do
  case "$1" in
    -v|--verbose) verbose=1; shift ;;
    -o|--output)  output="$2"; shift 2 ;;
    --)           shift; break ;;
    -*)           echo "Unknown option: $1" >&2; exit 1 ;;
    *)            break ;;
  esac
done

echo "verbose=$verbose output=$output remaining=$*"

shift 2 discards two arguments at once — needed after options that take a value (-o value). After parsing flags, "$@" holds the remaining positional arguments.

4

while read -r line; do ... done processes stdin line by line. The -r flag prevents read from interpreting backslashes. Without it, a line ending in \ is joined with the next line — a silent data corruption.

# Read lines from a file
while IFS= read -r line; do
  echo "Line: $line"
done < /etc/hosts

# Pipe into while read
find /var/log -name "*.log" | while IFS= read -r logfile; do
  echo "Found: $logfile"
done

IFS= (empty IFS) before read prevents stripping of leading/trailing whitespace from each line. Always pair IFS= read -r when reading file content faithfully.

5

while with a command condition loops until the command exits non-zero.

# Poll until a port opens (useful in startup scripts)
until nc -z localhost 5432 2>/dev/null; do
  echo "Waiting for postgres..."
  sleep 2
done
echo "Postgres is up"

# Process a queue file
while [[ -s queue.txt ]]; do
  item=$(head -1 queue.txt)
  process "$item"
  sed -i '1d' queue.txt
done

until is the inverse of while: it loops while the condition is non-zero (failure) and stops when it becomes zero (success). Both while and until evaluate the exit code of their condition command.

Worked example

Batch-compress log files passed as arguments.

#!/usr/bin/env bash
if [[ $# -eq 0 ]]; then
  echo "Usage: $0 file [file ...]" >&2
  exit 1
fi

for f in "$@"; do
  if [[ ! -f "$f" ]]; then
    echo "Skipping (not a file): $f" >&2
    continue
  fi
  gzip -v "$f" && echo "Compressed: ${f}.gz"
done

"$@" in the for loop preserves filenames with spaces. continue skips to the next iteration without aborting. && after gzip ensures the success message only prints if compression actually worked.

Why this works

On macOS bash 3.2, all loop constructs and "$@" behavior are identical to modern bash. One macOS-specific nuance: sed -i requires an explicit backup suffix (sed -i '' '1d' file) — the GNU sed form sed -i '1d' file does not work on macOS. If your script must run on both, use sed -i.bak '1d' file and remove the backup afterward, or install GNU sed via Homebrew.

Common mistake

A subtle trap with while read and pipes: each stage of a pipeline runs in a subshell. Variables set inside cmd | while read -r line; do VAR=...; done are lost after the loop ends — the shell that ran the loop was a subshell that exited. Fix: use process substitution instead of a pipe: while IFS= read -r line; do ...; done < <(cmd). This runs cmd in a subshell but the while loop itself runs in the current shell, so variable assignments persist.

Check yourself
Quiz

A script receives two arguments: 'hello world' and 'foo'. Which expansion produces exactly 2 loop iterations?

Recap

for item in list iterates over words or glob matches; always quote "$f" inside the loop body. "$@" expands each argument as a separate quoted word — the only safe way to forward arguments. $# gives the count; $1/$2 access individual arguments; shift consumes them one by one for option parsing. while IFS= read -r line reads stdin line by line without mangling backslashes or trimming whitespace. Variables set inside a piped while loop are lost — use process substitution < <(cmd) to keep them in the current shell.

Practice

Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.

recallapplystretch0 of 4 done

Something unclear?

Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.

shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources3
expand
  1. 01
  2. 02
  3. 03

Trademarks belong to their respective owners. Editorial reference only.