Skip to main content

Command Composition, Redirection, and Pipes

SUPPORTING CONCEPT - Turns separate commands into actual workflows

Concept Cluster Navigation​

Cluster 01: Shell Mastery

What This Concept Is​

The shell becomes powerful when commands stop being isolated. Redirection, pipes, and small filters let you take the output of one program and feed it into the next step.

Why It Matters Here​

Without composition, you use the shell like a faster launcher. With composition, you use it as a programmable environment for logs, test results, file discovery, report generation, and automation.

Concrete Example​

You want to count how many Python files mention TODO:

find . -name "*.py" -print0 | xargs -0 grep -n "TODO" | wc -l

Redirection is the other half of the story:

python run_tests.py > test.log 2> test.err

That keeps normal output and error output separate for later inspection.

Common Confusion / Misconception​

The common mistake is building pipelines by guesswork. Ask:

  1. What does this command read from?
  2. What does it write to?
  3. Is the next command expecting lines, filenames, or raw text?

How To Use It​

Key patterns:

cmd > out.txt
cmd >> out.txt
cmd 2> err.txt
cmd > out.txt 2>&1
cmd1 | cmd2 | cmd3
cmd | tee out.txt

Use tee when you want to both view and save a stream.

Check Yourself​

  1. Can you explain the difference between > and >>?
  2. Why does stderr not automatically move through a pipe?
  3. What would you use tee for during debugging?

Mini Drill or Application​

Using a directory with text files or source files, complete all of these:

  1. Save a file listing to files.txt
  2. Save only error output from a failing command to errors.txt
  3. Count how many lines contain import
  4. Use tee to both view and save the output of a command

Then explain what stream moved where in each case.

Read this only if stuck​

  • Start with Reference and Selective Reading for the curated shell, editor, and environment sources.
  • If you need a second explanation, skim Missing Semester and then retry the commands from this page.
  • Prefer one concrete terminal action over more passive reading.

Video and Lecture References​

Depth Path​

Professional Integration​

Pipelines are one of the first real productivity multipliers in engineering. People who can inspect a codebase, slice logs, and summarize outputs directly in the shell ship faster and debug faster.

Cluster Integration Check​

  • You can redirect stdout and stderr intentionally
  • You can explain what a pipeline is passing between stages
  • You can use tee when you need both visibility and saved output
  • You can build a small multi-stage command without guessing

Ready to advance: Move to Cluster 2 and start Modal Editing with Vim.