Docker vs containerd vs CRI-O: Where Is the Line?

What This Concept Is

"Docker" is a brand that covered four distinct pieces of software that used to ship together and have since been separated:

• Docker CLI / Docker Engine: the user-facing docker command and its REST API. A developer tool, not something Kubernetes ever spoke to directly in production.
• containerd: a daemon that manages the container lifecycle, pulls images, and owns snapshotters (overlay, etc.). Now a CNCF graduated project.
• runc: the OCI runtime that actually calls clone(), unshares namespaces, writes cgroup files, and execs the entrypoint.
• shim processes: one per container, keeping stdio and exit codes alive after containerd restarts.

CRI-O is a separate daemon that plays the same role as containerd from Kubernetes' perspective. It was built specifically to implement the Container Runtime Interface (CRI), the gRPC contract the kubelet uses to talk to a runtime.

Since Kubernetes 1.24 the kubelet no longer ships with a "dockershim" translator; it speaks CRI directly to containerd or CRI-O. Docker Engine is still useful for local development but is no longer in the cluster data path on modern nodes.

Why It Matters Here

If you think "Docker" is one thing, three real questions will confuse you:

1. "Why did Kubernetes 1.24 remove Docker?" (It did not remove containers. It removed the dockershim translator; the node still runs containerd under the hood.)
2. "Why does kubectl exec work but docker exec on the node shows nothing?" (Because the workloads are running under containerd, not dockerd.)
3. "Why does a runc CVE matter even if I use CRI-O?" (Both containerd and CRI-O call runc; the actual namespace and cgroup setup is almost always runc.)

Concrete Example

The runtime stack on a typical Kubernetes node:

Same boundaries, different names:

Layer	Docker world	Kubernetes world
User-facing CLI	docker	kubectl + crictl
Daemon	dockerd + containerd	containerd or cri-o
OCI runtime	runc	runc (or crun, kata-runtime)
Per-container helper	shim	shim

A single crictl ps on a node gives you the containerd-level view of all pods' containers, which is what the kubelet sees.

The CRI Contract

The Container Runtime Interface is a gRPC service, defined in cri-api, with two services the runtime must implement:

• RuntimeService -- lifecycle: RunPodSandbox, CreateContainer, StartContainer, StopContainer, RemoveContainer, ListContainers, ExecSync, Attach.
• ImageService -- image management: PullImage, ListImages, RemoveImage, ImageStatus.

The kubelet is a CRI client. containerd (via the cri plugin) or CRI-O is the server. If you understand these two services, you understand the entire kubelet-to-runtime boundary. crictl is a CLI that speaks the same CRI API, which is why it is the right tool for node-level debugging.

Common Confusion / Misconception

"Kubernetes 1.24 deprecated containers."

It deprecated dockershim, which was an adapter inside the kubelet. Containers still run; they run via containerd (or CRI-O). Images built with docker build still work in the cluster, because they are OCI-compatible images, which is what matters, not the tool that built them.

A second confusion: "I can just run docker ps on the node to debug a pod." On a modern node that runs containerd, docker usually is not installed. Use crictl ps and crictl logs instead -- those are the equivalent commands at the CRI boundary.

How To Use It

When reading an incident or a CVE, place it on the diagram above:

• Registry or pull problem -> containerd/CRI-O
• OCI config assembly problem -> containerd/CRI-O + image config
• Namespace/cgroup setup failure -> runc (or its substitute)
• "My docker command works locally but pods behave differently" -> different runtime, different defaults (seccomp, cgroup driver, cgroupns mode)

You rarely care about the boundary until something goes wrong; when it does, knowing who owns the layer is the difference between a one-line fix and an hour of guessing.

Failure Modes Mapped to Layers

Symptom	Likely layer	Why
ErrImagePull with HTTP 401	containerd / CRI-O	The runtime daemon did the pull; credentials are its concern
CreateContainerError: failed to generate spec	containerd / CRI-O	It is assembling the OCI config.json and rejected
RunContainerError: OCI runtime create failed	runc	Namespace/cgroup setup failed; usually a seccomp or permissions issue
Container gets SIGKILL with no exit message	runtime + kernel	Memory cgroup invoked OOMKiller; not a container-software bug
kubectl exec works, docker exec does not	It's a CRI cluster	Docker Engine is not in the data path

Put a failure on the right row before you start reading logs.

Image Builders Are Separate Again

An often-overlooked consequence of the Docker split: the tool that builds an image no longer has to be the tool that runs it. On a cluster without dockerd you may still build images with:

• docker build on a developer laptop (BuildKit under the hood)
• buildah / podman build in rootless CI
• kaniko or img inside the cluster itself (runs as a Pod, no daemon)
• nerdctl build against a local containerd

All produce OCI images. What the cluster consumes is the image, not the tool that built it. This is why "we dropped Docker" is operationally boring: the ergonomics of docker build are independent of whether the runtime on the node is dockerd, containerd, or cri-o.

Check Yourself

1. Which process actually creates the namespaces: containerd, runc, or kubelet?
2. What did Kubernetes 1.24 actually change?
3. If crictl ps shows a container but kubectl get pod does not, where is the divergence most likely?
4. Why do cluster CVEs often mention runc even when the cluster runs CRI-O?

Mini Drill or Application

On a running cluster node (or a kind node via docker exec), run:

crictl ps
crictl inspect <container-id>
crictl version

Identify the CRI runtime in use and read the generated OCI config for one container. Write down three fields that came from the Pod spec and three fields that came from the image config. Then describe what each field is telling runc to do.

Repeat for another container in a different pod. Note whether the two containers share a pod sandbox (they will, if they are in the same pod). Explain what "pod sandbox" means at the containerd layer versus the Kubernetes API layer.

Alternate OCI Runtimes

runc is the default, but it is not the only option. Swapping the OCI runtime changes the isolation guarantees without touching containerd/CRI-O:

• crun -- a C rewrite of runc, faster to start and with lower memory overhead.
• kata-runtime -- each container runs in a lightweight VM. Full kernel isolation, at the cost of a few hundred milliseconds of startup.
• gvisor (runsc) -- a userspace kernel that implements most Linux syscalls, trapping container syscalls away from the host kernel.
• youki -- a Rust implementation of the OCI runtime spec.

Kubernetes exposes this through a RuntimeClass resource. A Pod may set spec.runtimeClassName: kata to be scheduled only on nodes whose containerd/CRI-O is configured with the kata handler. This is how multi-tenant platforms get stronger isolation for untrusted workloads without maintaining separate clusters.

Node-Level Debugging with crictl and ctr

Two CLIs matter on a modern Kubernetes node:

• crictl speaks the CRI gRPC API. It is the supported debugging tool, because the CRI is the contract the kubelet uses. crictl ps, crictl logs, crictl inspect, crictl pods all operate at the pod-sandbox abstraction the kubelet manages.
• ctr (containerd's native CLI) speaks containerd's API directly. It sees every container containerd knows about, including infra and build artifacts that do not correspond to Pods. It is useful for runtime-level problems, but it is not aware of pod sandboxes the way crictl is.

Rule of thumb: when a Pod is behaving strangely on one node, crictl is the first tool; when containerd itself is behaving strangely, ctr comes next.

Why This Split Happened

The Docker-as-monolith era ended for a reason: a scheduler needs a small, testable contract with the runtime; a developer tool needs ergonomic CLI, a build system, and a desktop experience. Bolting both to the same daemon made each worse.

Splitting gave:

• containerd / CRI-O -- graduated CNCF projects that the kubelet can depend on without pulling Docker's full surface area.
• runc -- a tiny OCI runtime that security teams can audit and swap (for crun, gVisor, kata) without touching the higher layers.
• BuildKit / buildah / kaniko -- image builders that do not need a privileged daemon running on every developer laptop.

The architectural lesson is generic: when a layer serves two different customers (schedulers vs humans), split it along the contract boundary. This is an ADR worth writing when you face the equivalent inside your own system.

Read This Only If Stuck

• Linux Command Line: Sending signals to processes with kill -- shim processes preserve signal delivery and exit codes to the kubelet.
• Linux Command Line: Putting a process in the background / signals -- PID 1 semantics in a pod sandbox.
• Kubernetes: Container Runtimes -- supported CRI runtimes, cgroup driver requirements, and install paths.
• Kubernetes blog: Don't Panic: Kubernetes and Docker -- the authoritative explanation of the 1.24 change.
• Kubernetes: Container Runtime Interface (CRI) -- the gRPC contract the kubelet speaks to any runtime.
• containerd documentation -- architecture, plugins, and the cri plugin that serves the CRI API.
• CRI-O documentation -- configuration, runtime handlers, and how RuntimeClass wires alternate OCI runtimes.
• Kubernetes: RuntimeClass -- how Pods select kata/gVisor/crun via a per-node handler.
• runc repository (opencontainers/runc) -- the reference implementation doing the namespace/cgroup work.
• crictl reference on Kubernetes docs -- the supported node-level debugging CLI at the CRI boundary.