Stack Buffer Overflows
Imagine a login program that copies a username into a fixed-size array. It never checks the input's length. A careless call like `strcpy(buffer, input)` trusts the caller completely. If the input is longer than the buffer, extra bytes spill into adjacent memory. This includes, on the stack, the address the program will jump to. That jump happens when the current function returns. This single design flaw was first weaponized publicly in 1988. It remains one of the most consequential bugs in computing history. Stack buffer overflows matter for a key reason. They convert a simple memory-safety mistake into full control-flow hijacking. An attacker who can overwrite a return address can redirect execution. They can point it to code of their own choosing. This might be shellcode, a library function, or a chain of existing instructions. This is the foundational exploit primitive behind decades of worms. It also underlies remote-code-execution CVEs and game-console jailbreaks. That is why modern compilers, operating systems, and hardware all cooperate. Together they work to make stacks harder to corrupt. This article builds on Memory-Safety Vulnerabilities. It focuses specifically on the stack. We examine how function calls lay out data in memory. We show how an oversized write corrupts a saved return address. We then explain how that corruption becomes a hijacked `EIP`/`RIP`. We work through the mechanics with a concrete stack-frame diagram. We also provide a byte-level worked example. Finally, we cover the classic mitigations that shaped the resulting arms race. These include stack canaries, non-executable stacks, and ASLR. The article also foreshadows where this leads next. Once direct shellcode injection is blocked, attackers pivot to a new approach. They begin reusing existing code instead. This is the subject of Code Injection Attacks and Return-Oriented Programming. Understanding the raw stack overflow is essential groundwork. It is the prerequisite for understanding why those later techniques exist at all.
- A stack buffer overflow happens when a write to a stack-allocated array exceeds its bounds and corrupts adjacent memory.
- The critical target is the saved return address, which controls where the CPU jumps after the current function ends.
- Overwriting it lets an attacker redirect execution — classically to injected shellcode placed in the same buffer.
- Exploitation requires knowing (or guessing) the buffer's stack address and the offset to the return address.
- Defenses include stack canaries, non-executable stack pages (`DEP`/`NX`), ASLR, and safe-language/bounds-checked APIs.
- These defenses pushed attackers toward code-reuse techniques, foreshadowing Return-Oriented Programming.
An unbounded write into a stack buffer can overwrite the saved return address, letting an attacker redirect the CPU's next jump to code of their choosing.
The idea
A stack buffer overflow is a memory-safety violation in which data written to a fixed-size buffer allocated on the call stack extends past the buffer's boundary. Because the stack also holds bookkeeping data for function calls — saved registers, the caller's frame pointer, and crucially the return address — an overflow that reaches far enough corrupts that bookkeeping. The return address is the instruction pointer value the CPU restores via a `ret` instruction when the current function finishes; it tells the processor where to resume execution in the caller. The exploit primitive follows directly from stack layout. When a function is called, the caller pushes arguments and a return address, then the callee's prologue reserves space for local variables (including any buffers) by decrementing the stack pointer. On most common architectures the stack grows downward (toward lower addresses), while array indexing and typical buffer-filling loops write upward (toward higher addresses). A local buffer therefore sits at a lower address than the saved return address, and an overflow that writes past the end of the buffer marches directly toward — and eventually over — that saved address. Overwrite it with an attacker-chosen value, and when the function executes `ret`, control jumps there instead of back to the legitimate caller. The classic full exploit chain has three parts: (1) a vulnerable copy — an unbounded write such as `strcpy`, `gets`, `sprintf` without a length limit, or a manual loop lacking a bounds check; (2) a payload, historically machine code ("shellcode") placed inside the oversized input itself; and (3) a hijacked return address pointing back into the buffer (or elsewhere) so that execution flows into the payload. Understanding this chain precisely — not just "overflow bad" — is what lets you reason about which mitigations block which step, and why attackers evolved toward reusing existing code instead of injecting new code, the subject of Code Injection Attacks and Return-Oriented Programming.
How it works
Step 1 — Understand the frame layout. Each function call creates a stack frame containing (from low to high address, x86-family, growth downward): local variables/buffers, the saved frame pointer (old `EBP`/`RBP`), the return address, then the caller's arguments. The exact layout depends on calling convention and compiler, but the ordering "locals below saved return address" is the structural fact that makes the attack possible.
Step 2 — Identify the unbounded write. The vulnerability is any operation that copies attacker-controlled bytes into a fixed buffer without checking that the byte count fits. Typical culprits: ``` char buffer[64]; strcpy(buffer, user_input); // no length check gets(buffer); // cannot bound input at all sprintf(buffer, "%s", user_input);// unbounded format copy for (i = 0; user_input[i]; i++) // hand-rolled loop, no bound on i buffer[i] = user_input[i]; ```
Step 3 — Compute the offset to the return address. The attacker (or analyst) determines how many bytes lie between the start of the buffer and the saved return address — this includes buffer padding, alignment, and the saved frame pointer. This offset is usually found via source inspection, disassembly, or a cyclic pattern technique: send a uniquely-marked byte sequence, crash the program, then read which 4 or 8 bytes ended up in the instruction pointer to locate the exact offset.
Step 4 — Craft the payload. The overflow input is built as: ``` [ NOP sled ] [ shellcode ] [ padding ] [ new return address ] ``` The NOP sled (a run of no-op instructions) increases the chance that a slightly-imprecise guessed address still lands inside executable filler that slides into the shellcode. The new return address is chosen to point back into the buffer itself (commonly into the NOP sled), so that when `ret` executes, the CPU jumps into attacker-supplied code instead of returning to the caller.
Step 5 — Trigger the return. The vulnerable function completes its epilogue and executes `ret`, which pops the (now-corrupted) return address off the stack into the instruction pointer. Execution resumes at the attacker's chosen address. ```pseudocode function vulnerable(user_input): buffer = allocate_on_stack(64 bytes) // fixed-size local array copy_unbounded(buffer, user_input) // no bounds check performed! return // pops corrupted return address function copy_unbounded(dest, src): i = 0 while src[i] != NUL_TERMINATOR: dest[i] = src[i] // writes past dest's 64 bytes i = i + 1 // once len(src) > 64 dest[i] = NUL_TERMINATOR // attacker's constructed input, conceptually: payload = NOP_SLED(N bytes) + SHELLCODE(M bytes) + PADDING(to reach exact offset of saved return address) + TARGET_ADDRESS(points into NOP_SLED or SHELLCODE) send(payload) to vulnerable() // on ret: CPU jumps to TARGET_ADDRESS instead of legitimate caller ```
Complexity & correctness
Complexity of the exploit search. Finding the correct offset to the return address is a linear scan over the input length, `O(n)` where `n` is the buffer's distance to the return slot; with a cyclic-pattern probe it takes exactly one crash to read off the offset, since the crashing address itself encodes the position. Locating a usable payload address to jump to, absent leaks or disabled ASLR, is a brute-force guess over the address space entropy — with `k` bits of randomization the expected number of attempts is `O(2^k)`, which is why increasing ASLR entropy is a direct, provable defense (more on this below).
Correctness of the mechanism (why it reliably works absent defenses). The argument rests on an invariant of the calling convention: the return address is always stored at a fixed, statically-determinable offset from the base of the callee's locals, for a given compiler, architecture, and optimization level. This is not incidental — it is required for `call`/`ret` to work at all, since the CPU must know unambiguously where to find the address to restore. Consequently: 1. If the copy operation writes `L` bytes into a buffer of capacity `C` with no check that `L ≤ C`, and `L` is attacker-controlled, then the attacker controls exactly which bytes land at every offset beyond `C`, including the offset holding the return address. 2. Because memory is byte-addressable and the write path is contiguous, the attacker can place both the jump target and the code to jump to (the shellcode) within the same controlled input, satisfying both requirements for hijacking in a single write. 3. The `ret` instruction is unconditional trust: it pops whatever bytes currently occupy that stack slot into the instruction pointer, with no validation. This absence of a check is exactly what stack canaries and shadow stacks retrofit. This is why the attack is deterministic once the offset and target address are known, rather than probabilistic — the only probabilistic element in the classic (non-leaked) exploit is guessing an address inside the target address space, since compilers and the OS don't verify return-address integrity.
Honest tradeoffs of the defenses this bug provoked:
| Defense | What it stops | What it does not stop |
|---|---|---|
| Stack canaries | Overwrites that must cross a canary byte before reaching the return address | Overflows that overwrite return address without touching the canary (e.g., via a pointer write), or that leak/predict the canary |
| Non-executable stack (`NX`/`DEP`) | Direct injected-shellcode execution on the stack | Redirection to existing executable code — this is precisely what motivates Return-Oriented Programming |
| ASLR | Deterministic guessing of shellcode/return addresses | Information leaks that reveal addresses at runtime; low-entropy 32-bit spaces are still brute-forceable |
| Bounds-checked copy functions (`strncpy`, `strlcpy`) | The overflow itself, at the source | Nothing, if programmers still miscompute the length argument |
None of these are complete; each closes one link in the exploit chain described above, which is exactly why real hardened systems compose several of them.
Worked example
Consider this simplified vulnerable C function on a 32-bit x86 stack (addresses illustrative, growing downward): ```c void vulnerable(char *input) { char buffer[8]; strcpy(buffer, input); // no bounds check } ``` Assume the compiler lays out the frame with `buffer` at address `0xBFFFF000`, no padding for simplicity, and the saved return address at `0xBFFFF00C` (8 bytes for `buffer`, 4 bytes for a saved frame pointer, then the return address — a plausible, if simplified, layout).
| Offset from `buffer` start | Address | Contents (normal run) |
|---|---|---|
| 0–7 | `0xBFFFF000`–`0xBFFFF007` | `buffer[8]` |
| 8–11 | `0xBFFFF008`–`0xBFFFF00B` | saved frame pointer |
| 12–15 | `0xBFFFF00C`–`0xBFFFF00F` | return address (e.g. `0x08048456`) |
The attacker sends a 20-byte input: ``` Bytes 0-7 : "AAAAAAAA" (fills buffer[8]) Bytes 8-11 : "BBBB" (overwrites saved frame pointer) Bytes 12-15 : 0x11223344 (little-endian: 44 33 22 11) (overwrites return address) Bytes 16-19 : padding / part of a real target if input is longer ``` Trace of `strcpy(buffer, input)` writing byte-by-byte:
| `i` | Byte written | Destination address | Effect |
|---|---|---|---|
| 0 | `'A'` | `0xBFFFF000` | inside buffer — normal |
| 7 | `'A'` | `0xBFFFF007` | last valid buffer byte |
| 8 | `'B'` | `0xBFFFF008` | first overflow byte — corrupts saved frame pointer |
| 11 | `'B'` | `0xBFFFF00B` | frame pointer fully corrupted |
| 12 | `0x44` | `0xBFFFF00C` | first byte of return address overwritten |
| 15 | `0x11` | `0xBFFFF00F` | return address now reads `0x11223344` |
When `vulnerable` executes its epilogue and `ret`, the CPU pops `0x11223344` from `0xBFFFF00C` into the instruction pointer. If that address is unmapped, the process crashes with a segmentation fault — the everyday symptom of this bug during ordinary testing. If instead the attacker set that value to the address of a NOP sled placed earlier in the same input (say `0xBFFFF000`, the start of the buffer, now itself containing a sled followed by shellcode instead of `"AAAAAAAA"`), execution jumps back into the buffer and the shellcode runs with the privileges of the vulnerable process. This is the mechanism that converts "text got copied into the wrong place" into "attacker-chosen code now executing" — the essential idea that Code Injection Attacks generalizes across other memory-safety bugs.
Common pitfalls & traps
| Pitfall | Why it happens | Consequence |
|---|---|---|
| Off-by-one in bounds checks | Using `<=` instead of `<` against buffer capacity, or forgetting the NUL terminator's byte | One-byte overflow — often enough to corrupt an adjacent saved byte or canary byte |
| Trusting `strncpy`/`snprintf` blindly | These bound the copy but callers still miscompute the length argument (e.g., using `sizeof(pointer)` instead of `sizeof(array)`) | False sense of safety; overflow still possible |
| Assuming stack layout is portable | Compiler, architecture, optimization level, and padding/alignment all change frame layout | An exploit offset computed on one build fails silently or crashes differently on another |
| Ignoring the frame pointer and canary in offset math | Focusing only on "buffer size" without accounting for saved registers between buffer and return address | Miscalculated payload lands return-address bytes in the wrong slot |
| Treating stack canaries as sufficient | Canaries only detect corruption that must pass through them sequentially | Overflows via a separate pointer write, or leaks that reveal the canary value, bypass them entirely |
| Disabling protections "just for this test build" | `-fno-stack-protector`, disabling ASLR/NX for debugging convenience | Shipping a build with mitigations off, or forming exploit-dev habits that don't transfer to hardened targets |
Pioneers & history
The stack buffer overflow's public debut is inseparable from the Morris Worm, and its formal, widely read explanation is inseparable from a single 1996 paper that turned an obscure technique into common knowledge among both attackers and defenders.
Origins · 1 minRobert Tappan Morris and how this was discovered
The Morris Worm's stack overflow was one vector among several, but it is the event that made memory-corruption bugs a matter of national infrastructure concern rather than academic curiosity. The 1996 Aleph One paper then did for offensive security education what a good textbook does for any field: it gave a generation of researchers and attackers a precise, reproducible mental model of frame layout and return-address hijacking, directly shaping both exploit development and the defensive mitigations — canaries, `NX`, ASLR — that followed within a decade.
Where this matters
Stack buffer overflows are the ancestral case study behind essentially every modern control-flow-integrity mechanism shipped in production systems. Compilers (`GCC`, `Clang`, `MSVC`) insert stack canaries by default in optimized and unoptimized builds alike; operating systems mark stack pages non-executable and randomize memory layout at process start; CPU vendors have added hardware features such as shadow stacks (Intel CET, ARM's pointer authentication) specifically because software mitigations alone proved bypassable. Embedded and IoT firmware — often built without these protections for cost or legacy reasons — remains a persistent target for exactly this class of bug, and CVE databases continue to log remote-code-execution reports rooted in an unbounded `strcpy`, `memcpy`, or hand-written copy loop. Capture-the-flag competitions and vulnerability-research training pipelines still start here, precisely because every later technique — heap exploitation, format-string attacks, and especially Return-Oriented Programming as a response to `NX`-protected stacks — is best understood as a variation on, or evolution from, this foundational primitive.
Ask the tutor: How does a return-oriented programming exploit still work even when the stack is fully non-executable?
What exactly is a "return address" and why is it the attacker's target?
The return address is the memory location the CPU jumps back to once the currently executing function finishes, restored by the `ret` instruction. It is pushed onto the stack automatically at call time and popped automatically at return time, with no built-in check that it hasn't changed in between. Because it sits at a fixed, predictable offset from local buffers within a given compiled function, an unbounded write into those buffers can reach and overwrite it deterministically. Overwriting it is attractive because it gives the attacker a single, reliable lever to redirect the entire process's execution, rather than needing to corrupt data used later in some indirect way.
Does using `strncpy` instead of `strcpy` completely fix this vulnerability?
Not automatically. `strncpy` bounds the number of bytes copied, but only if the length argument passed to it correctly reflects the destination buffer's real capacity. A common real-world bug is passing the source string's length, or a `sizeof` computed on a pointer rather than the actual array, which reintroduces an unbounded effective copy. `strncpy` also does not guarantee NUL-termination if the source is exactly as long as the limit, which can cause separate over-read bugs later. Bounds-checked functions reduce risk only when paired with correctly computed, destination-based size arguments — the function alone is not the safety guarantee.
Why does ASLR only make exploitation harder rather than impossible?
ASLR randomizes the base addresses of the stack, heap, and shared libraries at process start, so an attacker cannot know in advance where their shellcode or return target will land. But it does not remove the underlying overflow, and randomization is finite — on 32-bit systems entropy is low enough that brute-force guessing across repeated crashes can succeed within a feasible number of attempts. Additionally, any bug that leaks a memory address (an information disclosure) can defeat ASLR entirely by telling the attacker exactly where things are, after which the original overflow becomes just as exploitable as if ASLR were absent.
How does a non-executable stack (NX/DEP) stop this attack, and what replaced injected shellcode once it did?
NX marks stack memory pages as non-executable, so even if an attacker successfully overwrites the return address to point into the stack, the CPU refuses to fetch instructions from that page and the process terminates instead of running shellcode. This closed off direct shellcode injection on the stack as a viable technique on hardened systems. Attackers responded by redirecting execution to existing executable code already loaded in memory — library functions, or short instruction sequences called "gadgets" chained together — rather than injecting new code. This code-reuse strategy is exactly what Return-Oriented Programming formalizes.
Is this vulnerability class specific to C and C++, or can it occur in any language?
It is overwhelmingly a risk in languages that expose raw memory access and manual buffer management without automatic bounds checking, primarily C and C++, and in low-level or unsafe code blocks within otherwise memory-safe languages. Languages with mandatory bounds-checked arrays and managed memory — Java, Python, Rust's safe subset, Go — prevent this specific mechanism by construction, because every array write is checked before it executes, and an out-of-bounds write raises an exception or panic instead of silently corrupting adjacent memory. This is a major reason memory-safe languages are increasingly mandated for new systems code handling untrusted input, as discussed under Memory-Safety Vulnerabilities.
What is a stack canary and how does it detect (not prevent) overflow?
A stack canary is a secret, randomly chosen value placed by the compiler between local buffers and the saved frame pointer/return address, written at function entry and checked just before the function returns. If a sequential buffer overflow writes past the buffer and through the canary's location on its way to the return address, it necessarily alters the canary value first, since the write is contiguous. The epilogue check then detects the mismatch and aborts the process before executing `ret`, rather than letting a corrupted address ever reach the instruction pointer. Canaries detect corruption after the fact; they do not stop the write itself, and they can be bypassed by non-sequential writes or a leaked canary value.
Why did this specific bug become historically so significant compared to other memory bugs?
It combines maximal impact with, historically, maximal ease: a single mistaken function call could hand an attacker arbitrary code execution with the privileges of the running process, over a network, with no authentication required. The Morris Worm demonstrated this impact at Internet scale in 1988, and the 1996 "Smashing the Stack for Fun and Profit" paper then made the technique broadly teachable, seeding a generation of exploit development. Because the exploited primitive — an unchecked write plus an unconditionally trusted return address — is so structurally simple, it also became the natural first case study for essentially every control-flow-integrity defense built afterward, from canaries through ASLR to hardware shadow stacks.