systems · starter · 4d
CLI text tool
Build a small command-line tool that reads text from stdin or a file, transforms it (filter, count, slice), and composes with pipes — the Unix way, no framework.
Deliverable
A Node CLI (shebang + chmod +x) that reads from stdin or a file arg, supports --filter / --count / --slice flags, handles large files via streams, exits with correct codes, and composes with | and >.
Milestones
0/5 · 0%- 01A CLI that runs
Make a file that runs as a CLI: a shebang (#!/usr/bin/env node), chmod +x, and a single command that reads an argument or stdin and writes to stdout. Run it as ./tool hello and as echo hello | ./tool — both should work. Exit codes matter: 0 on success, non-zero on missing input or bad flag, so a pipe like ./tool --bad 2>/dev/null; echo $? shows the error code. No argument-parsing library yet — just process.argv and process.stdin so you see what every library hides.
Definition of done- The file has a shebang, is executable, and works both as ./tool arg and as echo hi | ./tool.
- Bad input or unknown flag exits non-zero and the exit code is testable via echo $?.
Self-review
Show ./tool arg and echo | ./tool both produce output and bad flag → non-zero exit. A reviewer checks shebang + chmod and correct exit codes.
- 02Stdin or file, same code
Let the tool read from either stdin (when piped) or a file path argument (when given), like cat, grep, and sort do. The rule is: if a file argument is present, read that file; otherwise read stdin. Handle both uniformly as a readable stream so the rest of the tool doesn't care where the bytes came from. Handle errors: a missing file → stderr + non-zero exit, not a crash or a silent empty output. Prove it: cat file.txt | ./tool --count and ./tool --count file.txt produce the same count.
Definition of done- Both cat file | ./tool and ./tool file produce the same output; a missing file writes to stderr and exits non-zero.
- The input is consumed as a stream (not readFileSync of the whole file) so large files don't blow the heap.
Feeds fromSelf-review
Show piped vs file arg same output and missing file → stderr + non-zero. A reviewer checks the source is a stream, not a sync whole-file read.
- 03Flags: filter, count, slice
Add three flags that cover the core Unix text patterns: --filter <pattern> keeps only lines containing the substring (like grep), --count prints only the number of matching (or total) lines (like wc -l), and --slice <start>:<end> keeps lines start..end (like sed -n). Parse them by hand from process.argv — no yargs/commander — so you handle --help, unknown flag → stderr + non-zero, and flag ordering. The flags compose: --filter foo --slice 1:5 means 'filter then slice the filtered output'. Validate: --slice with a bad range (negative, reversed) → 400-style stderr + non-zero, not a silent wrong slice. Measure: filter a 100k-line fixture and show the output is correct and fast.
Definition of done- --filter keeps matching lines, --count prints the count, --slice keeps the line range; flags compose in order and --help prints usage.
- Bad ranges or unknown flags → stderr + non-zero; filtering a 100k-line fixture is correct and streamed, not buffered fully.
Self-review
Show --filter, --count, --slice individually and composed, plus bad flag → stderr. A reviewer checks hand-parsed argv, --help, and correct error codes.
- 04Streams for large files
Make it handle a large file without holding it all in memory. Process stdin/file as a line-by-line stream (split on \n, handle the trailing partial line) so a 500 MB file never exceeds a few MB of heap. The naive readFileSync + split would OOM or GC-pressure on large inputs; streaming keeps memory flat. Handle the edge: a line longer than the chunk boundary must be reassembled before filtering, not split mid-line. Prove it: generate a 10k-line fixture and also a 100 MB fixture (or simulate with a large stream), run the tool, and show heap stays bounded (or at least that the line count is correct and the process doesn't crash). Explain why a line-buffer, not a chunk-buffer, is the right granularity.
Definition of done- Large files are processed line-by-line via streams with correct handling of chunk-boundary lines; a 100 MB fixture does not OOM or miscount lines.
- You can explain why line-buffer (not chunk-buffer) is correct and where readFileSync would fail.
Self-review
Show correct line count on a large fixture and explain chunk-boundary handling. A reviewer checks the input is a line stream, not a whole-file read.
- 05Compose with pipes and redirects
Prove the tool is a real Unix citizen: it composes with other tools via pipes and redirects with no special code. Demonstrate: cat file | ./tool --filter foo | sort | uniq -c (pipe out), ./tool --filter foo file > out.txt (redirect), and ./tool --filter foo file | ./tool --count (pipe to self). Each flag's output is line-oriented text on stdout, errors on stderr, and the exit code signals success — so > and | work exactly like they do for cat/grep/wc. Add a tiny test harness that spawns the CLI (child_process.spawn) and asserts on stdout/stderr/exit code for a few cases — your first CLI tests without a framework.
Definition of done- Piping to sort/uniq and redirecting to a file work; piping the tool to itself (--filter | --count) is correct.
- A spawn-based test asserts on stdout/stderr/exit code for at least 3 cases (success, filter, bad flag).
Self-review
Show pipe to sort/uniq, redirect to file, and pipe to self, plus spawn tests. A reviewer checks stdout is line text, stderr is errors, and exit codes are correct.
Starter
fallowlone/skein-projects
projects/cli-text-tool
- README.md
- src/tool.ts
- test/tool.test.ts
npx degit fallowlone/skein-projects/projects/cli-text-tool cli-text-tool Implement the stubs, then run the tests until they pass: bun test
Fork the repo and push your work — the grade workflow runs the suite plus static checks on your own runners.
Rubric
| Junior | Mid | Senior | |
|---|---|---|---|
| Unix contract (stdin/file, stdout/stderr, exit codes) | Reads only stdin or only a file, not both; errors go to stdout or the process crashes instead of exiting non-zero. | Reads stdin when piped and a file when given, writes results to stdout and errors to stderr, and exits non-zero on bad input. | Exit codes distinguish usage error (2) vs file-not-found (1) vs success (0), and the tool is pipe-safe (no extra formatting that breaks sort/uniq). |
| Streaming & large files | Reads the whole file with readFileSync and splits; works on small fixtures but OOMs or miscounts on large files or chunk-boundary lines. | Processes input as a line stream with correct chunk-boundary reassembly; a large fixture is counted correctly without holding it all in memory. | Can explain why line-buffer is the right granularity, where a chunk-buffer would split a line mid-filter, and how backpressure (pause/resume) prevents the writer from outrunning the reader. |
| Flags & composition | One flag works in isolation; combining flags or piping to another tool gives wrong output or ignores the second flag. | Flags compose in order (--filter then --slice) and piping to sort/uniq or redirecting to a file works; --help prints usage. | Flags are hand-parsed with correct error handling (unknown flag → stderr + non-zero), and a spawn-based test asserts on stdout/stderr/exit for multiple cases without a test framework. |
Reference walkthrough (spoiler)
Why stdin OR file: Unix tools are filters — they transform a byte stream regardless of source. Accepting both makes the tool composable: echo hi | tool and tool file do the same thing, and tool file | sort | uniq -c chains without temp files.
Why streams: readFileSync loads the whole file into a single Buffer/String before processing. A 500 MB file needs 500 MB of heap plus GC pressure; a line stream needs only the current line. The chunk-boundary bug — a line split across two chunk callbacks — is the classic streaming defect and must be handled by buffering the tail.
Make it senior
- Add --regex for JS RegExp filtering and --invert to keep non-matching lines, with escaping for special chars.
- Add a progress bar on stderr for large files (lines processed) that does not pollute stdout when piped.