open atlas
↑ Back to track
Command line CLI · 06 · 03

sed substitution

sed processes text line by line. The s/old/new/g command substitutes a regex pattern with a replacement string. Address prefixes (line numbers, /regex/) restrict which lines are edited. -i rewrites files in-place — portability between GNU and BSD sed requires care.

CLI Middle ◷ 22 min
Level
FoundationsJuniorMiddleSenior

You have a config file with a hostname that needs to change. Or a log export where every timestamp uses the wrong format. Or a script that references an old API endpoint across fifty lines. Opening an editor works for one file; sed works for a thousand files in a pipeline. sed — the Stream EDitor — reads text line by line, applies a set of editing commands, and prints the result. Its most-used command by far is s/old/new/g: find a regex, replace it with a string, repeat for every occurrence on the line. Everything else in sed is about controlling which lines the substitution applies to.

By the end of this lesson you will be able to perform global substitutions, restrict edits to specific lines or address ranges, delete lines matching a pattern, and safely edit files in-place.

Goal

After this lesson you can use sed 's/old/new/g' for global substitution, use addresses to restrict which lines are edited, delete lines with the d command, and use sed -i (with BSD portability in mind) to rewrite files in-place.

1

s/REGEX/REPLACEMENT/FLAGS is the core sed command. The / characters are delimiters — you can use any character in their place, which is useful when the pattern contains slashes.

echo "hello world" | sed 's/world/earth/'
# hello earth

Without any flags, s replaces only the first match on each line.

The g flag replaces all matches on the line.

echo "aaa bbb aaa" | sed 's/aaa/zzz/g'
# zzz bbb zzz

Without g, only the first aaa would become zzz.

Alternate delimiter: When the pattern contains /, use a different delimiter to avoid escaping.

sed 's|/old/path|/new/path|g' config.txt

Using | as the delimiter. Any non-alphanumeric character works: s#old#new#g, s,old,new,g.

2

Addresses restrict which lines receive the command. Without an address, a command applies to every line.

  • N — line number: 3s/old/new/ applies only to line 3.
  • $ — last line: $d deletes only the last line.
  • /REGEX/ — lines matching a pattern: /^#/d deletes comment lines.
  • N,M — line range: 2,5s/foo/bar/ edits lines 2 through 5.
  • /START/,/END/ — range between two patterns: /BEGIN/,/END/s/x/y/ edits lines from the first match of BEGIN to the next match of END.
# Replace only in lines 10 through 20
sed '10,20s/debug/info/g' app.log

# Replace only in lines that contain 'ERROR'
sed '/ERROR/s/localhost/prod-server/g' config.log

Address + command is a powerful combination: you can make surgical changes without touching lines that look similar but are in the wrong section.

3

The d command deletes lines. Combined with an address, it removes specific lines from the output.

# Delete all comment lines (starting with #)
sed '/^#/d' nginx.conf

# Delete blank lines
sed '/^[[:space:]]*$/d' file.txt

# Delete lines 1 through 5 (e.g. a CSV header + metadata)
sed '1,5d' report.csv

# Delete the last line
sed '$d' file.txt

d prints nothing for the matched line and immediately moves to the next line — no further commands run on a deleted line.

Negation with !: addr!cmd applies cmd to lines that do NOT match the address.

# Keep only lines containing 'ERROR' (delete everything else)
sed '/ERROR/!d' app.log

This is equivalent to grep 'ERROR' but demonstrates sed’s address system.

4

-i edits files in-place. Instead of printing to stdout, sed -i rewrites the file directly. This is essential for scripted config management and mass rename operations.

# GNU sed (Linux)
sed -i 's/old_hostname/new_hostname/g' /etc/hosts

# Rename across multiple files at once
sed -i 's/deprecated_func/new_func/g' src/*.py

Always test without -i first — pipe to stdout and visually verify the substitution before modifying files.

-i supports an optional backup suffix: sed -i.bak 's/old/new/g' file saves the original as file.bak. Without a suffix (GNU sed) or with an empty string suffix (BSD sed), no backup is made.

5

Capture groups with \(...\) and backreferences \1. Standard sed uses BRE (Basic Regular Expressions) where group syntax is \(...\) and backreferences are \1, \2.

# Swap first and second colon-separated fields
echo "alice:admin" | sed 's/\([^:]*\):\([^:]*\)/\2:\1/'
# admin:alice

\([^:]*\) captures “everything up to the colon” as group 1. \([^:]*\) captures the second field as group 2. The replacement \2:\1 swaps them.

For extended regex with cleaner syntax (+, ?, |, groups without backslash), use -E:

echo "2025-06-19" | sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3\/\2\/\1/'
# 19/06/2025
Worked example

Strip ANSI color codes from a log file so it’s readable in a plain text viewer.

ANSI escape sequences look like \e[31m (red) or \e[0m (reset). The pattern is ESC[ followed by numbers and letters.

sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' colored.log > clean.log

\x1b is the hex escape for the ESC character (0x1B). \[[0-9;]*[a-zA-Z] matches the rest of the sequence. The g flag removes all occurrences per line.

Mass-replace an API endpoint across a codebase:

find . -name '*.js' -exec sed -i 's|https://api.old.com|https://api.new.com|g' {} +

find locates all .js files; -exec sed -i ... {} + runs sed -i on each batch. The | delimiter avoids escaping the / in the URLs.

Why this works

BSD sed (macOS) requires a backup extension argument after -i — even an empty string must be provided explicitly. GNU sed (Linux) treats -i with no argument as “no backup”. This means sed -i 's/old/new/g' file works on Linux but fails on macOS with sed: 1: "file": invalid command code f. The portable form: sed -i '' 's/old/new/g' file (macOS) vs sed -i 's/old/new/g' file (Linux). For scripts that must run on both, detect the platform or use perl -i -pe 's/old/new/g' file, which behaves identically everywhere.

Common mistake

Forgetting the g flag is the most common sed mistake. sed 's/foo/bar/' replaces only the first occurrence on each line — if a line contains foo foo, the second foo survives unchanged. Use g unless you specifically want only the first match. Another trap: sed -i without a prior dry run on stdout. Always test sed 's/pattern/replacement/g' file | head -20 before adding -i — a bad regex with -i and no backup overwrites the file with garbage.

Check yourself
Quiz

You run: sed '/^#/d' config.txt. What does this command do?

Recap

sed 's/REGEX/REPLACEMENT/g' substitutes all matches on each line. Without g, only the first match per line changes. Addresses (line numbers, /regex/, N,M ranges) restrict which lines a command applies to. The d command deletes matched lines. -i rewrites files in-place — always dry-run first; on macOS use sed -i '', on Linux sed -i. Capture groups (\(...\) in BRE, (...) with -E) and backreferences (\1, \2) enable field-swap and reformatting operations. Use an alternate delimiter like | when the pattern or replacement contains slashes.

Practice

Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.

recallapplystretch0 of 4 done

Something unclear?

Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.

shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources2
expand
  1. 01
  2. 02

Trademarks belong to their respective owners. Editorial reference only.