Skip to main content

Volumes, PersistentVolumes, and StorageClasses

What This Concept Is​

Kubernetes storage is three layers, each with a different lifetime:

  • Volume: a directory mounted inside a Pod. Tied to the Pod's lifecycle. Examples: emptyDir (node-local scratch, gone when the Pod dies), configMap / secret (projected content), hostPath (an arbitrary host path -- mostly anti-pattern).
  • PersistentVolume (PV): a cluster-scoped resource representing a piece of real storage (cloud disk, NFS export, Ceph RBD, CSI-backed volume). Lifetime is independent of any Pod.
  • PersistentVolumeClaim (PVC): a namespaced request for storage ("I want 10Gi, ReadWriteOnce"). The claim binds to a PV that satisfies it.

Automation is provided by a StorageClass: a parameterized template that tells a dynamic provisioner (typically a CSI driver) how to create a new PV when a PVC asks for one. "Dynamic provisioning" means you do not pre-create PVs; you create a StorageClass and let PVCs summon PVs.

Three access modes a PV can advertise:

ModeMeaning
ReadWriteOnce (RWO)Mounted as read-write by a single node
ReadOnlyMany (ROX)Mounted read-only by many nodes
ReadWriteMany (RWX)Mounted as read-write by many nodes (rare; requires NFS/CephFS-like backend)

Why It Matters Here​

The volume model is what lets a StatefulSet survive Pod restarts and node failures without losing data. Getting it wrong typically shows as:

  • pods stuck Pending because PVCs cannot bind (no matching PV or no StorageClass)
  • pods stuck ContainerCreating because the node cannot mount the attached disk
  • data loss because the developer used emptyDir for what should have been a PV
  • surprises at scale-up because the workload assumed RWX but the disk is RWO

Concrete Example​

A claim and a Pod that uses it:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-data
spec:
  accessModes: ["ReadWriteOnce"]
  storageClassName: gp3
  resources:
    requests:
      storage: 20Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: postgres
spec:
  containers:
    - name: db
      image: postgres:16
      volumeMounts:
        - { name: data, mountPath: /var/lib/postgresql/data }
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: postgres-data

With StorageClass gp3 pointing at the EBS CSI driver, applying this:

  1. The PVC is created in Pending.
  2. The provisioner sees the PVC, calls the CSI driver to create a 20Gi EBS gp3 volume.
  3. A PV is created and bound to the PVC.
  4. The Pod is scheduled only to a node in the same AZ as the volume.
  5. The CSI attach and mount hooks make the disk visible as /var/lib/postgresql/data inside the container.

When the Pod is deleted, the PVC remains; when the PVC is deleted, what happens to the underlying volume is controlled by the PV's persistentVolumeReclaimPolicy (Delete or Retain).

Common Confusion / Misconception​

"emptyDir is persistent."

It is not. emptyDir lives for the lifetime of the Pod. When the Pod is deleted or the node is replaced, the data is gone. Even a Pod restart on the same node preserves the directory, but a Pod deletion does not.

A second confusion: "ReadWriteOnce means one pod at a time." It means one node at a time. Two Pods on the same node can both mount the same RWO volume read-write. This matters during rolling updates where old and new Pods overlap on a node.

A third confusion: "A PVC is the disk." A PVC is a request. The disk is whatever the StorageClass's provisioner creates, represented by the PV. Deleting the PVC does not necessarily delete the disk -- that depends on the reclaim policy.

How To Use It​

Decision flow:

Operational checks:

  1. kubectl get sc -- which StorageClasses exist? Which is default (storageclass.kubernetes.io/is-default-class: "true")?
  2. kubectl get pv,pvc -A -- what is bound, what is pending, what is orphaned?
  3. kubectl describe pvc <name> -- why has it not bound yet?

Check Yourself​

  1. What is the difference between a Volume, a PersistentVolume, and a PersistentVolumeClaim?
  2. What does ReadWriteOnce actually restrict?
  3. Who creates the underlying disk when a PVC has no matching PV?

CSI and Dynamic Provisioning​

The Container Storage Interface (CSI) is the pluggable contract between Kubernetes and storage backends. A CSI driver provides:

  • a controller plugin that calls the storage API to create, delete, attach, detach, and snapshot volumes
  • a node plugin that runs on every node to mount the volume into a Pod's filesystem

When a PVC is created against a StorageClass that names a CSI provisioner, the sequence is:

  1. external-provisioner sidecar sees the PVC and calls the CSI controller's CreateVolume.
  2. Controller plugin talks to the cloud/storage API, creates the disk, returns a volume ID.
  3. A PV is created and bound to the PVC.
  4. When the Pod is scheduled, external-attacher calls ControllerPublishVolume to attach the disk to the chosen node.
  5. The node plugin calls NodeStageVolume and NodePublishVolume to mount the filesystem into the Pod's volume path.

CSI unified what had been a pile of "in-tree" volume plugins compiled into Kubernetes itself. In modern clusters virtually all persistent storage is CSI.

Mini Drill or Application​

On a cluster with a default StorageClass, apply the Postgres example. Run:

kubectl get pvc postgres-data
kubectl describe pvc postgres-data
kubectl get pv

Delete the Pod. Re-create it. Verify the data persisted. Then delete the PVC and observe what happens to the PV depending on its reclaim policy.

Common PVC Bind Problems and Their Fix​

Symptom in describe pvcLikely causeFix
no persistent volumes available for this claim and no storage class is setMissing storageClassName and no default StorageClass on the clusterset storageClassName explicitly or mark a default SC
waiting for a volume to be created (stuck)Dynamic provisioner is installed but the CSI driver pod is unhealthycheck the driver's DaemonSet and controller pods in kube-system
volume "pvc-..." conflict, ZONE mismatchPod landed in a different zone than the EBS/PDset volumeBindingMode: WaitForFirstConsumer on the SC
PVC bound but Pod stuck ContainerCreating: FailedMountNode plugin can't attach (IAM, quota, firewall)inspect node plugin logs, check cloud-provider IAM / attach quotas
PV in Released state, new PVC won't bindReclaim policy Retain after PVC deletioneither kubectl edit pv to clear claimRef, or provision a new PV

Dynamic provisioning hides complexity until something goes wrong. When it does, these five cases cover the large majority of real production incidents.

Read This Only If Stuck​