Module 3: Replication & Partitioning: Guided Labs

Lab 1: Replication-Lag Timeline

Simulate a leader and two followers with controllable apply delay. Execute a write followed by reads routed to each replica. Record commit, replication, apply, and read times. Add a session token or leader-pinning mechanism and prove read-after-write for that session.

Evidence: timeline, failing test, passing test, and a statement of what consistency is not guaranteed.

Lab 2: Quorum and Conflict Clinic

Evaluate at least four (N,R,W) configurations during normal operation and one-node failure. Then inject concurrent writes and implement explicit version comparison and reconciliation. Include a case where quorum intersection exists but naive “last arrival wins” loses intent.

Evidence: availability/overlap table and conflict traces.

Lab 3: Partition a Skewed Dataset

Compare range, hash, and directory-based partitioning using a generated tenant workload with one hot tenant. Measure maximum shard load, scatter/gather count, and movement required when adding a shard. Design a hot-key remedy and migration protocol.

Evidence: generator, distribution results, migration checkpoints, and rollback condition.

Lab 4: Rebalance Without Double-Counting

Move a partition while writes continue. Specify snapshot position, change capture, idempotency key, validation, cutover, and old-owner retirement. Test retries and a worker crash at every stage.

Exit check: the design states ownership at every instant and provides evidence that no acknowledged write is lost or applied twice semantically.