Skip to main content

Async/Await and Event-Loop Concurrency Models

What This Concept Is​

Async/await is cooperative concurrency on a single thread. Instead of many preemptable threads sharing memory, a single thread runs an event loop that executes tasks (coroutines). A task runs until it voluntarily suspends at an await point, at which time the event loop resumes another ready task.

Key structural facts:

  • Between two await points, code runs uninterruptedly on the event loop thread.
  • The event loop is driven by I/O readiness (epoll, kqueue, io_uring, IOCP) and timers.
  • A CPU-bound loop with no await points stalls the entire event loop and blocks all other tasks.
  • Because only one task runs at a time, data races on memory between tasks do not exist at the CPU instruction level -- but logical races absolutely do.

Examples: Node.js's single-threaded event loop, Python's asyncio, Rust's tokio, C#'s Task, Kotlin's coroutines, Go's goroutines (similar but with M:N threading).

Why It Matters Here​

Async is the dominant concurrency model for network servers because one event-loop thread handles tens of thousands of idle connections cheaply, compared to the 1-2 MB of stack per OS thread.

It matters to this module because async changes which races are possible without eliminating concurrency reasoning:

  • No instruction-level race on shared data. You may mutate a dict without a lock.
  • Logical races persist. Two coroutines that both await db.read() and then update a cache can interleave their updates at the await point and produce the same kind of lost-update bug you saw in concept 2.
  • A blocking call inside an async function (a time.sleep, a heavy for-loop, a filesystem stat) freezes everything. The equivalent of "holding a lock too long" in threaded code is "going too long between awaits."

Async replaces the lock-contention problem with the fair scheduling problem: how do we keep the event loop responsive?

Concrete Example​

A logical race in Python asyncio:

cache = {}

async def get_user(uid):
    if uid in cache:
        return cache[uid]
    # no lock needed: we are single-threaded
    u = await db.fetch_user(uid)   # await -> yields to event loop
    cache[uid] = u
    return u

Two coroutines calling get_user(42) concurrently both find uid absent, both await the DB, both write to the cache. One DB query is wasted, and any side effect of the fetch (logging, metrics, cache stampede protection) runs twice. A lock is not the right fix in async -- use a dict of asyncio.Future promises so the second caller awaits the first caller's result:

inflight: dict[int, asyncio.Future] = {}

async def get_user(uid):
    if uid in cache: return cache[uid]
    if uid in inflight: return await inflight[uid]
    inflight[uid] = fut = asyncio.get_event_loop().create_future()
    try:
        u = await db.fetch_user(uid)
        cache[uid] = u
        fut.set_result(u)
        return u
    finally:
        del inflight[uid]

A performance trap in Node.js:

app.get('/hash', (req, res) => {
    const h = bcrypt.hashSync(req.body.pw, 12);   // 300ms of CPU, blocks
    res.json({h});
});

The synchronous hash stalls the event loop for every request. All other pending requests wait. The fix is to offload CPU work to a thread pool (bcrypt.hash(...) async form) so the event loop remains responsive.

Common Confusion / Misconception​

"Async/await means no concurrency bugs." Instruction-level data races go away; logical races and liveness bugs do not. You still need coordination for one-at-a-time semantics, batching, ordering, and fairness.

"Async is always faster than threads." Only for I/O-bound workloads. For CPU-bound work, threads (or a worker pool, or multiple processes) are still required. Frameworks like tokio and Go combine both: M:N threading with cooperative coroutines.

"I can await inside a critical section." You can in async code, but an await inside a mutex or a critical logical section creates a new class of bugs: the state is visible to other tasks while your coroutine is suspended. Use asyncio.Lock explicitly and know what await does inside it.

"Go is async/await." Go is M:N scheduling with preemption. Multiple OS threads run multiple goroutines; goroutines preempt at function calls (and, since Go 1.14, asynchronously). So Go code can have true data races -- use sync.Mutex.

How To Use It​

When choosing a concurrency model:

  1. I/O-heavy, many connections -> async / event loop.
  2. CPU-heavy, embarrassingly parallel -> threads or processes.
  3. Mixed -> async event loop with offloaded worker pool for CPU work.
  4. In async, never block the event loop. Profile with a loop-latency metric in production.
  5. Treat every await point as a potential scheduling boundary; reason about what other tasks can observe there.

Check Yourself​

  1. Why does async avoid instruction-level races but not logical races?
  2. What is the cost of a synchronous bcrypt.hashSync in a Node.js request handler?
  3. When is async not the right tool even for a server?

Mini Drill or Application​

Write the get_user cache race in Python asyncio. Simulate the DB with asyncio.sleep(0.2). Run 100 concurrent requests for the same user and count how many DB fetches happen. Then apply the inflight-future fix and show that only one DB fetch happens.

Read This Only If Stuck​