open atlas
↑ Back to track
Command line CLI · 03 · 04

Counting and slicing

wc counts lines, words, and bytes; head and tail slice the top or bottom N lines; sort orders a stream; uniq -c collapses consecutive duplicates into a count. Together they form the classic frequency-analysis pipeline: sort | uniq -c | sort -rn | head.

CLI Junior ◷ 25 min
Level
FoundationsJuniorMiddleSenior

You can now connect commands with pipes and filter lines with grep. But what do you do when you want to count things, grab only the first ten lines of output, or find out which words appear most often in a log file? A small set of composable tools handles all of these: wc, head, tail, sort, and uniq. Combined in a pipeline, these five commands can answer questions about any text stream — from “how many errors are in this log?” to “what are the top 10 IP addresses hitting this server?”

By the end of this lesson you will be able to use each tool individually and compose them into the classic frequency-analysis pipeline.

Goal

After this lesson you can use wc -l to count lines, head -n N and tail -n N to slice output, sort to order a stream alphabetically or numerically, uniq -c to count consecutive duplicates, and compose sort | uniq -c | sort -rn | head to find the most frequent items in any stream.

1

wc counts lines, words, and bytes. The name stands for “word count” but it counts more than words.

wc /etc/passwd

Output:

 45  90 2751 /etc/passwd

Three numbers: line count, word count, byte count. Flags let you select one:

wc -l /etc/passwd     # line count only
wc -w /etc/passwd     # word count only
wc -c /etc/passwd     # byte count only

wc -l is by far the most common: “how many lines does this produce?”

ls /etc | wc -l

Count the number of entries in /etc.

2

head and tail slice the top or bottom N lines. By default both show 10 lines. Use -n N to change the count.

head -n 5 /etc/passwd     # first 5 lines
tail -n 5 /etc/passwd     # last 5 lines

tail -f follows a file as it grows — invaluable for watching logs in real time:

tail -f /var/log/syslog

New lines appear as they are appended. Press Ctrl+C to stop.

Both tools work with pipes:

ls /etc | head -n 10    # first 10 entries
ls /etc | tail -n 10    # last 10 entries
3

sort orders lines alphabetically by default. It reads all input, sorts it, then writes it out. Key flags:

  • -r — reverse order (Z→A, or largest→smallest with -n)
  • -n — numeric sort (so “10” sorts after “9”, not before “2”)
  • -k N — sort by field N (splitting on whitespace)
  • -u — unique: output each value only once (like sort | uniq)
ls /etc | sort           # alphabetical
ls /etc | sort -r        # reverse alphabetical
cat numbers.txt | sort -n   # numeric order

sort -n is critical when you want to rank by count: without it, “10” would sort before “2” because “1” < “2” lexicographically.

4

uniq -c collapses consecutive identical lines into one, prefixed with the count. The key word is consecutiveuniq only detects adjacent duplicates. That is why sort must come first: sort brings all identical lines together so uniq can count them.

printf 'apple\nbanana\napple\ncherry\nbanana\napple\n' | sort | uniq -c

After sort the stream is: apple apple apple banana banana cherry.

uniq -c output:

      3 apple
      2 banana
      1 cherry

The number is left-padded with spaces by default. The format is <count> <value>.

5

The classic frequency pipeline: sort | uniq -c | sort -rn | head. This four-stage pattern finds the most common items in any stream:

  1. sort — group identical items together.
  2. uniq -c — count each group.
  3. sort -rn — sort numerically in reverse (highest count first).
  4. head -n 10 — take the top 10.
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -n 10

This extracts the IP field (field 1 in nginx logs), counts each IP’s appearances, and shows the top 10 — the clients making the most requests. The same four-stage pattern works on any text: HTTP status codes, user agents, error messages, git commit authors, etc.

Worked example

Find the most frequent HTTP status codes in a web server log.

Nginx access logs have the status code in field 9 (in combined log format). Extract it and run the frequency pipeline:

awk '{print $9}' /var/log/nginx/access.log \
  | sort \
  | uniq -c \
  | sort -rn \
  | head -n 5

Typical output:

  52341 200
   3201 304
    891 404
    102 301
     18 500

At a glance: 52k successful responses, 3k not-modified, 891 not-found, 18 server errors. The entire analysis runs in a few seconds on a multi-GB log file because the pipeline streams — it never loads the whole file into memory at once.

Now count lines in the log before and after filtering:

wc -l /var/log/nginx/access.log
grep ' 500 ' /var/log/nginx/access.log | wc -l

Two numbers: total requests, and how many were server errors. The ratio is your error rate.

Why this works

On macOS, sort is BSD sort and uniq is BSD uniq. They accept the same flags (-r, -n, -k, -c) as their GNU counterparts. One subtle difference: BSD sort -k splits on whitespace including leading spaces, while GNU sort sometimes handles edge cases differently. For the sort | uniq -c | sort -rn | head pattern there is no practical difference — it works identically on Linux and macOS.

Common mistake

Running uniq -c without sort first is the most common mistake with this pipeline. uniq only collapses adjacent duplicate lines. If the input is unsorted, identical items scattered throughout the stream will each appear as separate count-1 entries instead of being merged. Always: sort | uniq -c, never just uniq -c on unsorted data.

Check yourself
Quiz

You run: cat words.txt | sort | uniq -c | sort -rn | head -n 3. What does sort -rn do in this pipeline?

Recap

wc -l counts lines in a stream. head -n N and tail -n N slice the first or last N lines. sort orders a stream; sort -rn sorts numerically in descending order. uniq -c counts consecutive identical lines — always pipe through sort first. The four-stage pattern sort | uniq -c | sort -rn | head finds the most frequent items in any text stream and is one of the most useful shell idioms you will use as an engineer. It works on log files, git history, HTTP status codes, or any line-oriented data.

Practice

Start at the top. Tasks go easiest → hardest: recall a fact, apply it to a case, then a senior-level stretch. Open one, attempt it, then reveal.

recallapplystretch0 of 4 done

Something unclear?

Ask a question about this lesson. Questions are anonymous and go straight to the author to make the lesson better.

shortcuts expand
search
K
prev piece
k
next piece
j
cycle tier
t
this menu
?
sources5
expand
  1. 01
  2. 02
  3. 03
  4. 04
  5. 05

Trademarks belong to their respective owners. Editorial reference only.