Encryption: At-Rest, In-Transit, and KMS Envelope Encryption
What This Concept Is
Encryption protects data by making it unreadable without a key. Cloud systems care about two lifecycles:
- In-transit encryption -- data moving between parties, protected by TLS. This stops passive network observers and on-path attackers from reading the bytes.
- At-rest encryption -- data stored somewhere (disk, database, object storage, backup), protected by keys held outside the storage medium. This stops someone who got a copy of the bytes (stolen disk, backup leak, wrong-bucket upload) from reading them.
Neither one is a substitute for the other. TLS does not protect a database dump; disk encryption does not protect a request in flight; both together still do not protect an authenticated request that the application itself is happy to answer.
Envelope encryption is how cloud KMS services (Google Cloud KMS, AWS KMS, Azure Key Vault) actually do at-rest encryption without making the KMS the bottleneck. The idea is a two-level key hierarchy:
- a DEK (data encryption key) encrypts the data
- a KEK (key encryption key) encrypts the DEK
- the encrypted data and the encrypted DEK are stored together
- only the KEK lives in the KMS, and only the KMS can unwrap the DEK
The Google Cloud envelope-encryption doc makes the pattern explicit: generate a DEK locally (e.g. AES-256), encrypt data with it, use the KEK (in KMS) to wrap the DEK, store the wrapped DEK with the ciphertext.
Why It Matters Here
Cloud systems store huge amounts of data across many services. Calling a KMS once per record would cost a fortune and would add latency to every read. Envelope encryption lets you do the expensive thing (KMS call) once per file or per short batch, and the cheap thing (symmetric encryption) for the actual bytes.
Envelope encryption is also what makes key rotation cheap. Rotating the KEK does not require re-encrypting your data -- you only re-wrap the DEKs. That changes what is practical operationally.
Concrete Example: DEK / KEK Flow
ENCRYPTION PATH
plaintext (big file)
|
generate DEK |
(e.g. AES-256)|
v
+-------+-------+
| Encrypt |
| plaintext |
| with DEK |
+-------+-------+
|
v
ciphertext
|
+------+ wrap DEK with KEK in KMS
| KEK +-------------------------+
+------+ |
v
wrapped_DEK
|
+---------------+-----------+
| Store together: |
| - ciphertext |
| - wrapped_DEK |
| - key_id / metadata |
+---------------------------+
DECRYPTION PATH
load (ciphertext, wrapped_DEK, key_id)
|
v
unwrap wrapped_DEK via KMS
using KEK(key_id)
|
v
DEK
|
v
decrypt ciphertext with DEK
|
v
plaintext
In this flow:
- the plaintext DEK exists only briefly in process memory
- the KEK never leaves the KMS
- key rotation = rotate KEK, re-wrap DEKs; ciphertext is unchanged
- access control = permissions on the KMS KEK, audited on every unwrap
Common Confusion / Misconception
"HTTPS is enabled, so we are encrypted." HTTPS protects data in transit between the client and the load balancer (and further, if mTLS is used internally). It does not protect anything at rest. A TLS-only deployment with an unencrypted RDS snapshot leaking is a disclosure incident.
"Disk-level encryption is data protection." Alone, it is theft protection. Against an attacker who can ask the application for data, disk encryption does not stop that, because the application will happily decrypt on every read. Envelope encryption + KMS access logs are what make at-rest encryption enforce access in addition to protect -- because a Decrypt call is attributable to an identity and can be denied by key policy.
"Rotate keys every 90 days." Know which rotation you are doing. Rotating the KEK is cheap: re-wrap existing DEKs. Rotating the DEK for billions of records is not, and is usually not necessary if the KEK wrapping is sound. NIST SP 800-57 distinguishes these as key-hierarchy operations with different cryptoperiods.
"KMS is just a store for keys." It is an access-control and audit boundary. The value of a KMS is that (a) the KEK never leaves the HSM, (b) every Decrypt call is logged with the caller's identity, and (c) the KEK's key policy + IAM decides who can use it. A KMS used only for key storage, with broad kms:Decrypt on *, is a glorified file server.
"Customer-managed keys (CMKs) always beat provider-managed." CMKs give you control of the key policy and rotation schedule, but also the responsibility. If the CMK is deleted or becomes inaccessible, the data is gone -- permanently. Use CMK when you need key-level isolation or regulatory separation of duties; use provider-managed when you primarily want encryption without the operational overhead.
How To Use It
For any system that stores meaningful data, answer:
- At-rest strategy. Managed by the cloud? Customer-managed key in KMS? Bring-your-own-key? What is the key hierarchy (how many DEKs, one KEK per what -- per service, per tenant, per data class)?
- Envelope encryption. For anything non-trivial, yes. If a single record unwrap requires a KMS round-trip, rethink; DEK per file or per batch is usually right.
- Access list. What identities can call
Decrypton the KEK? That IAM/key-policy list is the real access control list for the data -- not the DB user table. - Audit. Every unwrap must be logged (AWS CloudTrail, GCP Cloud Audit Logs, Azure Activity Log) with caller identity, resource, and request ID. Route these into your security log pipeline (Concept 11).
- In-transit scope. Is TLS terminated at the edge only, or is mTLS used service-to-service? The unencrypted hop inside the VPC is where insider/sidecar attackers hide.
- Key rotation. What is the KEK rotation cadence, and who tests that re-wrap works end-to-end? Rotating a key that cannot be rotated is the standard way to discover your backups were tied to the old key.
- Key destruction. What is the deletion policy for the KEK once data is retired? "Crypto-shredding" is a real technique: delete the KEK, and the ciphertext is unrecoverable.
Check Yourself
- Why is envelope encryption more efficient than calling KMS on every record?
- What does rotating the KEK actually change? What does it not change?
- What does TLS protect that disk encryption does not? What does disk encryption protect that TLS does not?
Mini Drill or Application
Draw the DEK/KEK diagram from memory, no peeking. Then add a second path showing what happens when the KEK is rotated. Verify yourself by comparing to the diagram above.
See also (external)
- Google Cloud KMS: Envelope Encryption -- the clearest short explanation of the DEK/KEK model with explicit best practices.
- AWS KMS Developer Guide: Concepts -- KMS keys, grants, key policies, and the envelope-encryption flow on AWS.
- AWS Well-Architected: Security Pillar (Data Protection) -- encryption patterns tied to AWS KMS, including customer-managed keys and grants.
- NIST SP 800-57 Part 1 (Recommendations for Key Management) -- the canonical treatment of key hierarchies, cryptoperiods, and rotation rationale.
- Building Secure and Reliable Systems, Ch. 8: Design for Resilience -- compartmentalization and blast-radius thinking that directly informs key-hierarchy design.
- Google Tink -- a cryptographic library that hides most envelope-encryption footguns behind a small API; useful as a reference implementation of sane defaults.
Depth Path
Source Backbone
Security and observability require official docs, but these books provide the systems and reliability backbone behind the practices.
- Building Secure and Reliable Systems - primary book backbone for security/reliability tradeoffs.
- Software Engineering at Google - support for operational engineering and process.
- The Linux Command Line - support for operational investigation and automation.