Module 4: CI/CD Pipelines & Release Engineering
Primary focus: delivery as code -- build, test, ship, and roll back on purpose
Selective support: Pro Git and Git from the bottom up for branching and versioning discipline; The Linux Command Line for shell fluency in runners and release scripts
This guide is the primary teacher. You do not need to read the source books front-to-back to complete this module. You do need to become operationally strong at pipelines, deployment strategies, release hygiene, and the safety rails around all of it.
Scope of This Module
This module is where the code leaves your laptop and reaches real users without drama.
What it covers in depth:
- the delivery mindset: small, frequent, reversible changes instead of heroic releases
- branching strategies and when trunk-based beats GitFlow -- and when it does not
- DORA metrics as honest feedback on the health of your delivery system
- the anatomy of a CI pipeline: build once, test at multiple levels, promote an immutable artifact
- pipeline-as-code in GitHub Actions and GitLab CI, with YAML that actually validates
- CD strategies: rolling, blue-green, canary, and the rollback criteria that make them safe
- feature flags and dark launches to decouple deploy from release
- progressive delivery and automated rollback driven by metrics
- release engineering: semantic versioning, changelogs, artifact registries, signing, provenance
- database migrations alongside code using the expand/contract pattern
- pipeline security: secrets management, OIDC to the cloud, least-privilege runners
- environments, approvals, change management, and what a "production deploy" audit trail looks like
- observability of delivery itself -- pipeline metrics and deployment markers
What it deliberately does not try to finish here:
- cloud platform fundamentals (that was Module 1) or IaC deep dives (Module 2)
- Kubernetes primitives beyond what a pipeline needs (Module 3 owns that)
- full SRE / incident response theory (Module 5 carries it further)
- enterprise change-management frameworks beyond the core ideas
If you can write YAML that deploys but cannot explain what would stop a bad deploy, the module is not done.
Before You Start
Answer these closed-book before starting the main path:
- What is the difference between continuous integration, continuous delivery, and continuous deployment?
- Why is a long-lived feature branch that merges once a month a delivery risk even if the code works?
- What does "build once, promote everywhere" mean and why does it matter?
- Name one deployment strategy that lets you abort mid-release without downtime.
- Why are rollback criteria the most important part of a canary rollout?
Diagnostic Interpretation
4-5 solid answers -- you are ready for the full path.
2-3 solid answers -- continue, but expect extra time in clusters 2 and 3.
0-1 solid answers -- revisit Module 3 container basics and the Git track (branching, tags) first.
What This Module Is For
Delivery is the bottleneck where good engineering becomes real value -- or where it quietly dies. Later work repeatedly asks:
- can we ship this change today, safely, with a rollback plan?
- if production breaks at 3am, can we revert in one command?
- how do we ship a breaking DB migration without an outage window?
- how do we let a single team deploy twenty times a day without stepping on each other?
- how do we prove the artifact in production is the one we built and tested?
This module builds the pipeline, release, and safety-rail discipline that every production engineering role expects.
Concept Map
How To Use This Module
Work in order. Later clusters assume the earlier delivery habits are stable.
Cluster 1: Delivery as a Discipline
| Order | Concept | Type | Focus |
|---|---|---|---|
| 1 | The Goal: Small, Frequent, Reversible Changes | PRIMARY | Why batch size dominates every other delivery tradeoff |
| 2 | Trunk-Based Development vs GitFlow | PRIMARY | Branching models, integration frequency, and when each is right |
| 3 | DORA Metrics and the Four Keys | PRIMARY | Lead time, deploy frequency, change-fail rate, MTTR -- and their limits |
Cluster mastery check: Can you explain why a team with small batches and a fast rollback path almost always out-delivers a team with more review stages?
Cluster 2: The CI Pipeline
| Order | Concept | Type | Focus |
|---|---|---|---|
| 4 | Build-Once, Promote-Everywhere | PRIMARY | Immutable artifacts and environment-specific configuration |
| 5 | Test Strategy in CI | PRIMARY | Unit, integration, contract, smoke -- what belongs where |
| 6 | Pipeline as Code | PRIMARY | GitHub Actions and GitLab CI workflows that actually validate |
Cluster mastery check: Can you draw a CI pipeline that produces one artifact, runs the right tests in the right order, and explain why each gate is there?
Cluster 3: The CD Pipeline
| Order | Concept | Type | Focus |
|---|---|---|---|
| 7 | Deployment Strategies: Rolling, Blue-Green, Canary | PRIMARY | Three safe ways to replace a running version |
| 8 | Feature Flags and Dark Launches | PRIMARY | Decoupling deploy from release |
| 9 | Progressive Delivery and Rollback Discipline | PRIMARY | Metric-gated rollouts with automated rollback |
Cluster mastery check: Given a service, can you pick a deployment strategy, state the rollback trigger, and estimate blast radius?
Cluster 4: Release Engineering
| Order | Concept | Type | Focus |
|---|---|---|---|
| 10 | Semantic Versioning, Changelogs, Release Notes | PRIMARY | How version numbers communicate compatibility |
| 11 | Artifacts, Registries, Signing, Provenance | PRIMARY | What ships, where it lives, and how we trust it |
| 12 | Database Migrations Alongside Code | PRIMARY | Expand/contract pattern and backward-compatible rollouts |
Cluster mastery check: Can you cut a release, bump the version correctly, and ship a schema change without a downtime window?
Cluster 5: Quality Gates and Safety
| Order | Concept | Type | Focus |
|---|---|---|---|
| 13 | Pipeline Security | PRIMARY | Secrets, OIDC to the cloud, least-privilege runners |
| 14 | Environments, Approvals, and Change Management | PRIMARY | Gated promotions, audit trails, and break-glass procedures |
| 15 | Observability in Delivery | SUPPORTING | Pipeline metrics and deployment markers in monitoring |
Cluster mastery check: Can you explain what a long-lived static cloud credential in CI really buys an attacker, and what you would replace it with?
Then work these practice pages:
| Order | Practice path | Focus |
|---|---|---|
| 1 | Pipeline Design Lab | Building a CI pipeline from scratch with real YAML |
| 2 | Deployment Strategy Workshop | Choosing and sequencing rollouts for real services |
| 3 | Release and Migration Clinic | Versioning, changelog, and expand/contract migration |
| 4 | CI/CD Katas | Four repeatable coding katas |
Use Module Quiz after the concept and practice path. Use Reference and Selective Reading and Learning Resources for targeted reinforcement.
Learning Objectives
By the end of this module you should be able to:
- Defend the "small, frequent, reversible" delivery stance against ceremonial release processes.
- Choose trunk-based or GitFlow-style branching based on concrete team and product constraints.
- Report DORA metrics for a system honestly, including at least one named shortcoming.
- Design a CI pipeline that builds one artifact and promotes it across environments.
- Classify tests into unit, integration, contract, and smoke, and place each at the right pipeline stage.
- Author a GitHub Actions or GitLab CI workflow in YAML that validates and runs.
- Pick rolling, blue-green, or canary for a given service and state the rollback criteria.
- Use feature flags to separate deploy from release and describe the flag-debt problem.
- Design a progressive rollout with automated rollback from error-rate or latency thresholds.
- Apply semantic versioning correctly, including what forces a major bump.
- Produce signed artifacts with provenance metadata and explain what SLSA levels 1-3 guarantee.
- Apply expand/contract to ship a breaking schema change without downtime.
- Replace long-lived cloud secrets in CI with OIDC-based short-lived credentials.
- Design environment promotion with protection rules and approvals that are not theater.
- Emit deployment markers and pipeline metrics into observability tooling.
Outputs
- a working CI workflow file (Actions or GitLab) that builds, tests, and publishes an artifact
- a CD rollout plan document for one service with strategy, rollback criteria, and dashboards
- a release checklist covering version bump, changelog, artifact, signing, migration, and rollback
- one expand/contract migration sequence for a concrete rename-a-column scenario
- one OIDC-to-cloud sample workflow replacing a static key
- one "what broke in delivery" post-mortem of a real or simulated incident
- a mistake log of at least 10 delivery errors such as
merged a long-lived branch,no rollback plan,shipped migration before code,static AWS key in CI - one short memo on how DORA metrics in your project would change under trunk-based development
Completion Standard
You have completed Module 4 when all of these are true:
- you prefer smaller changes because you can name the failure modes of larger ones
- you can read a CI YAML file and explain each job, why it is there, and what it produces
- you can pick a deployment strategy for a given service and defend the choice
- you can write a rollback trigger as a concrete metric threshold, not a vibe
- you ship DB changes in at least two steps and never couple them to code in one commit
- you do not put long-lived cloud credentials into CI, and can explain why
- you treat the pipeline itself as production: versioned, reviewed, monitored, and owned
If your "deploy" is still a checklist someone follows by hand, the module is not complete.
Reading Policy
- Concept pages are the main path. Most use validated external docs, not book chunks, because this is a practice-heavy topic.
- Local Pro Git and Git from the bottom up chunks are selectively used where Git branching or tagging semantics matter (clusters 1 and 4).
- The Linux Command Line chunks are optional for runner scripting.
Read only if stuckmeans try the concept page and drill first.
Suggested Weekly Flow
| Day | Work |
|---|---|
| 1 | Cluster 1 concepts and one DORA self-report for a real or sample project |
| 2 | Cluster 2 concepts and one CI pipeline authored end-to-end |
| 3 | Cluster 3 concepts and a written deployment plan with rollback criteria |
| 4 | Cluster 4 concepts and a tagged release with changelog and signing |
| 5 | Cluster 5 concepts and OIDC replacement of a long-lived secret |
| 6 | Practice pages 1-2, katas started |
| 7 | Practice pages 3-4, quiz, mistake-log cleanup |
Reference
If you need exact links into the local chunked books or curated external resources, use Reference and Selective Reading and Learning Resources.
Rich Learning Pages
Worked Examples | Guided Labs | Case Studies | Mistake Clinic | Reading Guide | Capstone Thread