Debugging with gdb: Breakpoints, Watchpoints, Core Dumps

What This Concept Is

gdb is an interactive debugger for C, C++, and other compiled languages. It lets you stop a running program at chosen points, inspect memory and registers, change values, step one instruction at a time, and revive dead programs from a core dump for post-mortem analysis.

Core operations:

Category	Command	What it does
run	run [args]	Start the program under the debugger
break	break main / break file.c:42	Stop when control reaches that location
watch	watch counter	Stop whenever the value of counter changes
step	step (into calls), next (over calls), cont	Resume execution
inspect	print expr, info locals, info args, backtrace, frame N	Look around
modify	set var x = 42	Change values on the fly
attach	attach <pid>	Debug an already-running process
post-mortem	gdb ./prog core	Open a core dump

Compile with -g (DWARF debug info) and ideally -O0 or -Og so the debugger can map every line of source to instructions.

Why It Matters Here

Every systems engineer eventually has a bug that cannot be reasoned about on paper. The trace is long, the invariant is subtle, the repro is flaky. gdb turns the running program into an observable specimen: you can pause it at a suspected boundary, check the exact value of every variable, walk up the stack to see who called the current function, and step forward one line at a time.

Core dumps are equally important. When a server crashes in production, you cannot always attach a debugger. If the kernel wrote a core file, you can investigate the crash hours later from a different machine and still see every stack frame and every register at the moment of death.

Concrete Example

A program with a planted bug:

#include <stdio.h>
#include <string.h>

void corrupt(char *s) {
    strcpy(s, "this string is way too long for a 16-byte buffer");
}

int main(void) {
    char buf[16];
    corrupt(buf);
    printf("%s\n", buf);
    return 0;
}

Compile with symbols, enable core dumps, run:

$ cc -g -O0 -o bug bug.c
$ ulimit -c unlimited
$ ./bug
Segmentation fault (core dumped)
$ gdb ./bug core
(gdb) bt
#0  0x... in __strcpy_avx2 () at strcpy.S:...
#1  0x... in corrupt (s=0x7ffc...) at bug.c:4
#2  0x... in main () at bug.c:9
(gdb) frame 2
(gdb) print buf
$1 = "this string is w"
(gdb) print sizeof(buf)
$2 = 16

The backtrace pins the crash to strcpy in corrupt. Switching to the main frame and printing buf shows the 16-byte buffer was overrun by the long string literal.

To interact with the program rather than post-mortem:

$ gdb ./bug
(gdb) break corrupt
(gdb) run
(gdb) info args        # s = 0x7ffc...
(gdb) watch *s         # break when *s changes
(gdb) next             # step through strcpy

Common Confusion / Misconception

"gdb on my program is useless because optimization scrambles the lines." Partially true -- heavy optimization (-O2, -O3) causes "value optimized out" and inlined frames. Compile test builds with -O0 -g or -Og -g. For production binaries where you need the optimized code but want to debug, keep separate debug-info files (-g + objcopy --only-keep-debug).

"Core dumps are disabled on my system." They often are. ulimit -c unlimited in the shell enables them; /proc/sys/kernel/core_pattern controls where they go. On systemd systems, coredumpctl list shows recent ones.

"A watchpoint is just a breakpoint on a line." No. A watchpoint triggers whenever the value of an expression changes -- on any line, anywhere in the program. That is exactly what you want when "some function clobbers my variable, but I don't know which one." Hardware watchpoints are fast; software watchpoints (when hardware runs out) are 100-1000× slower.

"printf debugging is always faster than gdb." Often, yes, for reproducible bugs. gdb pays off for: non-reproducible crashes, memory corruption, large programs where you do not know where to add prints, and anything involving a core dump from production.

How To Use It

A reliable debugging routine:

1. Compile with -g -O0 for the first run. Add -fsanitize=address if the bug smells like memory corruption.
2. Can you run and reproduce? If yes, set a breakpoint near the symptom, step back with bt.
3. Can you not reproduce but have a core? gdb ./prog core, bt, frame N, info locals.
4. If a variable takes on a bad value and you do not know where: watch var from main.
5. For multi-threaded bugs: info threads, thread <n>, thread apply all bt.
6. Keep the commands in .gdbinit or a -x script.gdb file so the session is reproducible.

Check Yourself

1. What is the difference between a breakpoint and a watchpoint?
2. What does -g change in the compiled binary?
3. How do you enable core dumps on a typical Linux system?

Mini Drill or Application

Do all four:

1. Write and compile the bug.c above. Enable core dumps, run, open the core with gdb, and produce a backtrace.
2. Use watch *s inside corrupt to see each byte overwrite. Count the number of writes before the program crosses into the stack canary / saved frame pointer.
3. Write a small program that crashes at a random iteration (e.g., after 1-1000 increments). Attach gdb to the running process with attach <pid> and inspect state.
4. In one sentence, explain when you would reach for gdb over printf debugging.

Read This Only If Stuck

• K&R 7.6: Error Handling -- stderr and exit -- pairs with core-dump investigation of program termination
• GDB User Manual
• Man page: man 1 gdb
• Man page: man 5 core -- core dump format and where they go
• GDB Cheat Sheet (Stanford CS)