Module 5: Distributed Systems Fundamentals: Guided Labs

Lab 1: Failure-Detector Simulator

Build a three-node heartbeat simulator with configurable delay, loss, pause, and crash. Measure detection time and false suspicions at several timeout values. Add suspicion states and recovery behavior rather than treating timeout as proof of death.

Evidence: event traces, results table, and an explicit latency/accuracy tradeoff.

Lab 2: Logical-Time Trace

Generate a multi-process history with local events and messages. Assign Lamport timestamps, then vector timestamps. For every selected pair, classify happened-before, reverse happened-before, or concurrent. Include a pair that Lamport order places sequentially even though vectors show concurrency.

Evidence: event graph and checked timestamp table.

Lab 3: Raft Election and Replication

Implement a deterministic simulator or small in-memory cluster for terms, votes, log matching, append entries, and majority commit. Inject a leader partition, competing candidates, stale logs, and follower recovery. Assert election safety and log-matching invariants after every transition.

Evidence: state traces, invariant assertions, and tests for stale-leader fencing.

Lab 4: Retry-Safe Distributed API

Wrap one state-changing operation with deadlines, bounded exponential backoff, jitter, idempotency keys, and a retry budget. Inject lost responses so the server succeeds while the client believes it timed out. Verify one semantic effect.

Exit check: the writeup distinguishes safety from liveness and states which assumptions each guarantee requires.