Skip to main content

Stack Frames, Calling Conventions, Activation Records

What This Concept Is​

Every function call creates a new stack frame (also called an activation record) on the thread's call stack. The frame holds everything the call needs beyond what fits in registers:

  • saved previous frame pointer (%rbp on x86-64)
  • saved return address (pushed by the call instruction)
  • register spills the compiler could not keep in registers
  • local variables that had their address taken or did not fit in registers
  • arguments beyond what the register-based part of the calling convention carries

The calling convention is the contract between caller and callee: which registers carry arguments, which return values, which registers the callee must preserve, and how the stack is aligned. On Linux x86-64 the convention is System V AMD64: first six integer / pointer arguments in %rdi, %rsi, %rdx, %rcx, %r8, %r9; return value in %rax; the stack must be 16-byte-aligned before call.

Why It Matters Here​

Stack frames are where:

  • buffer overflows corrupt return addresses (Cluster 4)
  • debuggers reconstruct backtraces
  • tail-call optimization reuses one frame for many calls
  • recursive depth limits bite you
  • performance work lives, because calls are not free

Concrete Example​

Consider:

int add(int a, int b) {
    int sum = a + b;
    return sum;
}

int main(void) {
    int r = add(3, 4);
    return r;
}

A simplified x86-64 stack (high addresses at top, stack grows down):

    higher addresses
   +---------------------+  <- where main's caller's frame ends
   | ret addr for main   |
   +---------------------+  <- rbp in main
   | saved rbp of caller |
   | local int r         |
   +---------------------+  <- rsp right before calling add
   | ret addr for add    |   (pushed by call add)
   +---------------------+  <- rbp in add
   | saved rbp (main's)  |
   | local int sum       |
   +---------------------+
    lower addresses

Arguments 3 and 4 came in via registers %edi, %esi. When add returns, ret addr for add is popped, %rbp is restored, and execution continues in main with the return value in %eax.

Common Confusion / Misconception​

"Local variables live in registers." They often do, but whenever you take the address of a local, the compiler must spill it to the stack. &local forces a memory home.

"The stack grows up." On x86-64 and ARM64, the stack grows downward (new frames at lower addresses). The language does not mandate this, but all common ABIs do.

"return &local; works as long as I read it quickly." The moment the function returns, the frame is gone. Any pointer into it is dangling even if the bytes happen to still read as expected.

How To Use It​

Whenever function-call behavior matters:

  1. Read the compiler's output with gcc -S -O1 -fno-omit-frame-pointer foo.c -o foo.s.
  2. Identify the prologue (push rbp; mov rbp, rsp; sub rsp, N) and epilogue (leave; ret or mov rsp, rbp; pop rbp; ret).
  3. Count which locals are in registers and which are in the stack frame.
  4. For deep recursion, track the frame size; stack overflow is a fixed limit (commonly 8 MB per thread on Linux).

Check Yourself​

  1. What does call add physically do, and what does ret physically do?
  2. What is the difference between caller-saved and callee-saved registers?
  3. Why is %rbp useful for debuggers even when an optimizer does not strictly need it?

Mini Drill or Application​

#include <stdio.h>

int square(int x) { return x * x; }

int main(void) {
    int r = square(9);
    printf("%d\n", r);
    return 0;
}

Compile and inspect assembly:

gcc -Wall -Wextra -O0 -S -fno-omit-frame-pointer -o square.s square.c

Read square.s line by line. Identify the prologue, epilogue, where x sits, and how the return value is produced. Run under a debugger: gdb ./square, break square, run, then info frame, backtrace, and disassemble.

Read This Only If Stuck​