Skip to main content

Server Architectures: Iterative, Forking, Threaded, Event-Driven

What This Concept Is​

Once you can accept a connection, you must decide how many connections to handle at once and how. Four classical patterns:

  • Iterative -- one connection at a time. accept one, serve it to completion, close, repeat. Trivial, but one slow client stalls everyone.
  • Forking (process-per-connection) -- accept, then fork a child to serve that connection while the parent returns to accept. Isolation is strong (child crash does not take the parent down). Memory and startup cost per connection is high.
  • Threaded (thread-per-connection) -- same as forking but cheaper: spawn a thread instead of a process. Threads share memory, so you must lock shared state. Still limited to thousands, not tens of thousands, of connections.
  • Event-driven -- one (or few) threads run an event loop on top of select / poll / epoll / kqueue / IOCP. Each fd reports readiness; the server services whichever are ready. Scales to many tens of thousands of connections per thread.

Modern servers (nginx, Envoy, Node.js, Go's net/http) are usually event-driven under the hood, sometimes multiplied by a small pool of threads.

Why It Matters Here​

The choice defines the server's throughput, latency tail, and failure modes. A service that handles 100 concurrent clients happily might collapse at 10,000 because a thread-per-connection design ran out of stack memory. An event-driven server handles 10,000 connections but introduces its own pain: one slow callback blocks every other connection.

Concrete Example​

Same echo service, three sketches.

Forking:

for (;;) {
    int c = accept(s, NULL, NULL);
    pid_t pid = fork();
    if (pid == 0) { serve(c); close(c); _exit(0); }
    close(c);                  // parent
}

Threaded (pthreads):

for (;;) {
    int *c = malloc(sizeof *c);
    *c = accept(s, NULL, NULL);
    pthread_t t;
    pthread_create(&t, NULL, serve_thread, c);
    pthread_detach(t);
}

Event-driven (epoll sketch):

int ep = epoll_create1(0);
struct epoll_event ev = { .events = EPOLLIN, .data.fd = s };
epoll_ctl(ep, EPOLL_CTL_ADD, s, &ev);

for (;;) {
    struct epoll_event events[64];
    int n = epoll_wait(ep, events, 64, -1);
    for (int i = 0; i < n; i++) {
        if (events[i].data.fd == s) accept_and_register(ep);
        else                         handle_ready_client(events[i].data.fd);
    }
}

The event-driven version never blocks on one client; it advances only the fds whose kernel buffers are actually ready.

Common Confusion / Misconception​

"Threads are always better than fork because threads are cheap." They are cheaper, but they share address space, so a bug in one handler can corrupt another. Forking used to be the default for a reason: isolation.

"Event-driven is always fastest." It is only faster if your per-connection work is mostly I/O-bound. CPU-heavy handlers in a single event loop block everything. Real systems combine an event loop with a thread pool for CPU work.

How To Use It​

Choose by answering three questions:

  1. How many concurrent connections must this hold? (Hundreds -> threads. Tens of thousands -> event-driven.)
  2. How CPU-bound is per-request work? (High -> thread or process pool.)
  3. How critical is isolation between requests? (Very -> forking; moderate -> threads; low -> shared event loop.)

Default for new services: event-driven with a worker-pool escape hatch.

Check Yourself​

  1. Why does a forking server need to reap zombie children, and what signal handles this?
  2. What does "C10K problem" refer to, and which architecture made it tractable?
  3. Why can a single slow handler degrade an event-driven server worse than a threaded one?

Mini Drill or Application​

  1. Implement an echo server three times: iterative, threaded, and epoll-based (or kqueue/select on macOS).
  2. Use ab or a small script to open 100, then 1,000, then 10,000 concurrent connections to each.
  3. Record how each variant behaves: response time, CPU usage, and failure mode. Note what breaks first in each.

Read This Only If Stuck​