Skip to main content

Shared Memory Between Processes

What This Concept Is​

Normally each process has a private address space: pointers in one process have no meaning in another. Shared memory is a region that is mapped into two or more processes simultaneously. A write through the pointer in one process becomes visible to the other, with no syscall per read or write.

POSIX gives two practical ways to get shared memory:

  1. shm_open + mmap. shm_open("/name", O_CREAT | O_RDWR, 0600) returns an fd to an in-kernel memory object. ftruncate(fd, size) sets its size. mmap(..., MAP_SHARED, fd, 0) maps it. Any process that opens the same name and maps it sees the same bytes.
  2. mmap with MAP_SHARED | MAP_ANONYMOUS before fork. The mapping is inherited by the child; parent and child see the same region. Simpler when the sharing is just between a parent and its descendants.

System V IPC (shmget, shmat) is older and still in use, but POSIX shm is cleaner and is the modern default.

Why It Matters Here​

Shared memory is the lowest-overhead inter-process communication available. Pipes and sockets copy bytes through the kernel; shared memory does not copy at all. High-performance systems (databases, message brokers, game engines with worker processes) use it when they need many MB/s of throughput between processes.

But -- and this is the whole reason shared memory is a SUPPORTING concept rather than a primary one -- shared memory has no built-in synchronization. The moment two processes write the same region, you own every race condition they create. You must layer in either process-shared mutexes (pthread_mutex_t in the region, with PTHREAD_PROCESS_SHARED), semaphores (sem_open), or atomics (Concept 12).

Concrete Example​

A parent and child that share a counter:

#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdio.h>
#include <stdatomic.h>

int main(void) {
    atomic_int *counter = mmap(NULL, sizeof *counter,
                               PROT_READ | PROT_WRITE,
                               MAP_SHARED | MAP_ANONYMOUS, -1, 0);
    atomic_init(counter, 0);

    pid_t pid = fork();
    if (pid == 0) {
        for (int i = 0; i < 100000; i++) atomic_fetch_add(counter, 1);
        _exit(0);
    }
    for (int i = 0; i < 100000; i++) atomic_fetch_add(counter, 1);

    waitpid(pid, NULL, 0);
    printf("final = %d\n", atomic_load(counter));  /* 200000 */
    munmap(counter, sizeof *counter);
    return 0;
}

Without atomic_fetch_add, the program would routinely print a number less than 200000 -- the classic lost-update race. The mapping must be MAP_SHARED; MAP_PRIVATE would give each process its own copy-on-write page after the first write.

Common Confusion / Misconception​

"I use MAP_SHARED, so my increment is safe." The sharing is about memory, not about atomicity. *counter += 1 compiles to load/add/store on most architectures, and two processes interleaving those three operations can lose an update. Share memory, but still use atomics or a mutex.

"I can share a C++ std::map between processes." Usually no. Any data structure containing pointers is dangerous in shared memory because the pointers are virtual addresses, and the two processes may map the region at different addresses. mmap(addr, ..., MAP_FIXED, ...) can pin the address, but it is fragile. For cross-process structures, use relative offsets or flat arrays of POD.

Another trap: forgetting to shm_unlink a named segment. Unlike an anonymous mmap, a shm_open segment lives until explicitly unlinked or the system reboots, even if every process has exited. Test code that creates /myshm and never unlinks leaks kernel resources you can inspect in /dev/shm.

How To Use It​

A disciplined shared-memory design:

  1. Decide whether you need named sharing (shm_open) or parent-child sharing (anonymous + fork). Prefer the latter when it fits.
  2. Lay out the region as a single POD struct: a header (version, size), a synchronization primitive, then data.
  3. Initialize the primitive in exactly one process (the creator). Others attach and use it.
  4. Use atomics for simple counters; use a process-shared pthread_mutex_t for anything more; use a ring buffer with two cursors for high-throughput streaming.
  5. Have a cleanup path that calls munmap and, for named segments, shm_unlink.

Check Yourself​

  1. Why is MAP_SHARED not sufficient to make *counter += 1 safe across processes?
  2. What is the difference between POSIX shared memory (shm_open) and anonymous MAP_SHARED?
  3. Why should shared structures avoid absolute pointers?

Mini Drill or Application​

Do all four:

  1. Run the example program above. Replace atomic_fetch_add with plain (*counter)++. Run 20 times and observe the distribution of final values.
  2. Rewrite it using a process-shared pthread_mutex_t placed inside the region. You must set PTHREAD_PROCESS_SHARED on the attr.
  3. Modify the example to use a named segment (shm_open("/demo", ...)). Verify via ls /dev/shm that the segment exists, and that it disappears after shm_unlink.
  4. In one sentence, explain when you should not reach for shared memory as an IPC choice.

Read This Only If Stuck​