Skip to main content

Idempotency and Convergence in IaC

What This Concept Is​

Two closely related properties that IaC tools promise and sometimes fail to deliver.

Idempotent: applying the same operation twice produces the same end state as applying it once. No duplicate rows, no second instance, no append.

Convergent: repeatedly applying an operation from different starting states eventually brings the system to the declared desired state.

Terraform combines both. terraform apply is idempotent on a matching system (no-op on the second run) and convergent on a drifted system (plan proposes the delta, apply reduces it to zero).

These are properties of the tool and configuration together, not just the tool. You can write Terraform that is neither, usually by reaching for local-exec provisioners that run unprotected shell.

Why It Matters Here​

Every production apply is an operation on a system that is not in a known clean state. Engineers rerun failed pipelines, bots retry, two people hit "Apply" in the UI within seconds. If the workflow is not idempotent and convergent, retrying breaks things.

Idempotency also underpins drift detection. A second run that does nothing is not just a comfort -- it is how you know the system matches the config. If your apply always shows changes, you have no signal for actual drift.

Concrete Example​

Idempotent by construction (resource blocks):

resource "aws_iam_role" "ci" {
  name               = "ci-runner"
  assume_role_policy = data.aws_iam_policy_document.ci.json
}

First apply creates the role. Second apply refreshes state, sees no diff, does nothing. Third apply after a drift (someone edited the trust policy in the console) proposes a single-field update and converges.

Easily non-idempotent (local provisioner):

resource "null_resource" "append_hosts" {
  provisioner "local-exec" {
    command = "echo '10.0.0.5 api' >> /etc/hosts"
  }
}

Terraform thinks this "succeeded" and remembers it. But the shell appended. Rerun from a fresh state (CI cache wiped) and you append again. The tool is idempotent; the configuration is not.

Convergent but not immediate (eventual consistency):

Some cloud resources (IAM propagation, DNS, Azure RBAC) take seconds to minutes before the next API read sees them. apply sometimes completes before the system has fully converged. This is why you occasionally see spurious diffs on the very next plan, which then resolve on the third run. Convergence is still holding -- just on a delay.

Common Confusion / Misconception​

"If apply succeeds, the system matches my config." Usually yes, but eventual-consistency windows and ignore_changes lifecycle rules can leave the real world looking different from the config. Always plan before and after to confirm.

"Idempotent means safe." It means rerunnable. A destroy plan is perfectly idempotent; it will happily delete your database twice if the state says to.

"Provisioners are how you make Terraform imperative." They are, and that is the problem. Treat local-exec / remote-exec as last-resort escape hatches, not features. Anything you put in them loses idempotency guarantees.

How To Use It​

  1. Prefer native resources over provisioners. If a provider resource does it, use the resource.
  2. When writing provisioners, make the underlying command idempotent (use install not cp --force, use apt-get install -y not echo >>).
  3. After every apply, run terraform plan again. Expect "No changes." Investigate if you see drift.
  4. In CI, treat "plan shows changes on unchanged code" as a bug. It is either drift, eventual consistency, or a poorly-written resource.

Check Yourself​

  1. Give one Terraform snippet that is idempotent and one that is not, using the same resource type.
  2. Why does Terraform refresh state at the start of plan?
  3. You apply, see "Apply complete," and then immediately plan again and see drift. Give two possible explanations.

Mini Drill or Application​

In your sandbox, apply a config that creates an S3 bucket with a tag. Run apply three times in a row. Confirm runs 2 and 3 are no-ops.

Now add a provisioner "local-exec" { command = "date >> /tmp/applies.log" } and repeat. Observe that the log grows each apply even though no cloud change is happening. Write one sentence on why the bucket is idempotent and the provisioner is not, using the definitions above.

See also (external)​

Source Backbone​

Infrastructure-as-code details are tool-specific, but these local books provide the operational backbone for shell, Git, and change discipline.