Threads and Concurrency
A web server must answer thousands of clients at once. It cannot freeze when one client is slow. If every request required a brand-new process, the operating system would drown. Memory duplication and creation overhead would be too costly. The practical fix is different. A single program can run several independent flows of control. These flows share the same memory, files, and permissions. They are called threads. Understanding threads means understanding what is shared and what is private. It also means understanding why that split makes concurrency both fast and dangerous. Threads matter because modern hardware and modern workloads both demand parallel flows of execution. Multicore CPUs sit idle unless software keeps several cores busy at once. Interactive systems must also overlap computation with waiting for disk, network, or user input. A single-threaded server blocks on one slow client. Meanwhile a hundred others wait in line. A multi-threaded server routes each client to its own thread. While one thread blocks on I/O, another runs on the CPU. This is the difference between a program that merely exists and one that scales. The big picture is a layered one. A process is a container. It holds an address space, open file descriptors, and one or more threads executing inside it. Threads inside the same process share the heap, global variables, and code segment. But each thread keeps its own stack, program counter, and register state. That sharing is the entire point. Communication between threads is nearly free — just read and write shared memory. Yet that sharing is also the entire danger. Uncoordinated reads and writes to the same memory produce race conditions. This article builds the mental model of threads as multiple flows of control inside one address space. It explains the classic thread-creation and join pattern. It works through a producer-style example with an explicit trace. It also catalogs the pitfalls that make concurrent programming famously error-prone. It sets up, without yet resolving, the follow-on questions of how threads are actually implemented. That includes implementation in user space, in the kernel, or both, and how the CPU switches between them.
- A thread is an independent flow of control; multiple threads in one process share the address space but keep private stacks, registers, and program counters.
- Threads are cheaper than processes to create and to switch between, because no new address space or page tables are needed.
- Concurrency (interleaved or parallel progress) is different from parallelism (simultaneous execution on multiple cores) — threads enable both.
- Shared mutable state without coordination causes race conditions; the shared-memory model that makes threads fast is the same model that makes them hazardous.
- The standard lifecycle is create → run → join, mirroring the fork/exec/wait pattern for processes but without copying memory.
- Threads underlie virtually all high-performance servers, GUIs, browsers, and parallel numerical software.
Threads let a program have many independent flows of execution that share one address space, buying cheap communication at the price of needing explicit synchronization.
The idea
A thread is an independent sequence of executing instructions — its own program counter, its own set of CPU registers, and its own execution stack — running inside a process. A process that contains exactly one thread is what earlier discussions of The Process Abstraction implicitly assumed: one flow of control moving through one address space. A multithreaded process instead contains several threads that all execute concurrently within that same address space, reading and writing the same heap-allocated data structures and global variables. The key distinction to internalize is what is shared versus what is private. Shared among all threads of a process: the code (text) segment, the heap, global and static variables, open file descriptors, and other process-wide resources such as signal handlers and the current working directory. Private to each thread: the stack (local variables and function call frames), the program counter, the general-purpose registers, and a small thread-local storage area if the runtime provides one. This split is precisely why creating a thread is far cheaper than creating a process with fork: `fork` must (conceptually) duplicate the entire address space, while spawning a thread only allocates a new stack and register set inside an existing address space. It is essential to separate two related but distinct ideas. Concurrency means a program is structured as multiple logically independent flows of control, whether or not they physically execute at the same instant — a single CPU core can give the illusion of concurrency by rapidly switching between threads. Parallelism means multiple flows of control genuinely execute simultaneously, which requires multiple physical CPU cores. Threads are the vehicle for both: on one core, threads provide concurrency through interleaving; on multiple cores, the same threads provide true parallelism. A correctly synchronized multithreaded program should behave identically (aside from timing and throughput) whether it runs on one core or on sixty-four.
How it works
The operating system (or a user-level threading library) exposes an API with roughly four primitives: create a thread with a starting function and argument, wait for a thread to finish and collect its result, optionally detach a thread so no one needs to wait for it, and terminate a thread's own execution. The classic pattern mirrors fork/exec/wait but stays inside one address space instead of forking a new one. ``` // Shared state, visible to every thread in this process shared int counter = 0 shared lock counter_lock function worker(id): for i in 1..1000: acquire(counter_lock) counter = counter + 1 // critical section release(counter_lock) return id // becomes the thread's "exit value" function main(): thread_handles = [] for id in 1..N: t = thread_create(worker, id) // new stack + registers, same address space thread_handles.append(t) for t in thread_handles: result = thread_join(t) // block until t finishes; reap its result print("thread finished:", result) print("final counter =", counter) // deterministic ONLY because of the lock ``` Step by step, this is what happens beneath the API: 1. `thread_create` allocates a new stack (a fixed-size region of memory, typically a few hundred KB to a few MB depending on the platform) and a fresh set of saved registers, sets the saved program counter to the entry point of `worker`, and hands the new thread's control block to the scheduler as ready. Crucially, no new page table or address space is built — the new thread reads and writes the very same heap and globals as its creator. 2. Scheduling decides, on each available CPU core, which ready thread runs next. This is exactly the same scheduling machinery used for whole processes; a thread is simply a schedulable unit that happens to share memory with siblings. (The details of how the CPU actually swaps register state between threads is the subject of Context Switching, and whether the entity being scheduled is visible to the kernel at all is the subject of User-Level vs Kernel-Level Threads.) 3. Execution and interleaving. On a single core, the running thread executes until it blocks, its time slice expires, or it yields; the scheduler then loads another ready thread's saved registers and program counter and resumes it. On multiple cores, several threads truly run at the same instant. Either way, any two threads may interleave their accesses to shared memory at essentially any granularity the hardware allows, which is why the critical section above needs a lock: without it, two threads could both read `counter`, both compute `counter + 1`, and both write back the same value, silently losing an increment. 4. `thread_join` blocks the calling thread until the named thread has terminated, then retrieves any return value, and (in most systems) reclaims the terminated thread's stack and control-block resources. This is the thread analogue of a parent calling `wait` on a child process. 5. Termination happens when the thread's entry function returns, or when it calls an explicit exit routine; some designs also allow one thread to forcibly cancel another, though this is discouraged because it can leave shared data structures in an inconsistent state if the victim held a lock at the moment of cancellation.
Complexity & correctness
Thread creation cost. Creating a thread requires allocating a stack (roughly O(1) bookkeeping plus the cost of reserving stack memory, which is typically not zero-filled eagerly) and initializing a control block — no page tables, no copying of the heap, no duplicated file-descriptor tables. This is asymptotically and practically far cheaper than `fork`, which conceptually duplicates the entire address space (mitigated in real systems by copy-on-write, but still involves duplicating page tables and process metadata). Thread creation is commonly one to two orders of magnitude faster than process creation and context switches between threads of the same process can avoid the cost of flushing address-translation caches, since the address space does not change.
Context-switch cost. Switching between two threads of the same process only needs to save and restore registers and stack pointers — the memory-management unit's mapping stays the same, so there is no need to reload page-table base registers or necessarily flush the translation lookaside buffer. Switching between threads of different processes (or between a thread and the kernel on some designs) does incur that extra cost. This asymmetry is the quantitative reason multithreading is favored for latency-sensitive concurrent workloads; the qualitative mechanics belong to Context Switching.
Space complexity. A process with `k` threads uses one copy of the address space (code, heap, globals) plus `k` private stacks, each typically reserved at some fixed maximum size `s`. Total memory overhead for stacks is `O(k · s)`, versus `O(k)` entire address spaces if the same concurrency were instead achieved with `k` separate processes — a substantial saving whenever the shared heap and code dominate.
Correctness: the interleaving argument. The subtle part of threads is not their cost but their semantics. Define the state of the program as the values of all shared variables. A multithreaded program's execution is one particular interleaving of the individual instructions of each thread (or, on true multicore hardware, one particular interleaving plus genuinely simultaneous memory operations, which is a strict superset of danger). A program is correct only if every legal interleaving produces an acceptable final state — not just the interleavings that happen to occur during testing. Consider the unlocked version of the counter increment: `counter = counter + 1` is not atomic; it decomposes into `load counter into register`, `add 1`, `store register into counter`. If threads A and B both execute this on `counter = 5` and their loads happen before either store, both compute `6` and both store `6` — one increment is lost, even though sequentially the correct answer is `7`. This is a race condition: the final value depends on the arbitrary timing of the scheduler, and among the exponentially many interleavings of `n` total increments across `t` threads, only some produce the correct sum. The lock in the pseudocode restores correctness by enforcing mutual exclusion: at most one thread executes the critical section (`counter = counter + 1`) at a time. The standard invariant used to argue this is correct: if the lock protocol guarantees that acquiring the lock can only succeed when no other thread currently holds it, and every critical-section execution is preceded by a successful acquire and followed by a release, then critical-section executions across all threads form a strict sequence with no overlap. Given that sequence, ordinary sequential reasoning applies to each critical section in turn, and by induction over the sequence, the final counter value equals the number of completed increments — exactly the same guarantee sequential code would give. This is the essential trick that all synchronization primitives (locks, semaphores, monitors) exist to provide: convert an exponential space of interleavings into a small, sequentially-reasoned set of possibilities.
The honest tradeoff. Threads are cheap and communication between them is cheap (just shared memory), but this same design pushes all correctness burden onto explicit synchronization. Processes are more expensive to create and communicate (needing explicit inter-process channels), but they buy strong isolation — one process cannot corrupt another's memory by a forgotten lock. Threads trade safety for speed of communication; that is not a flaw, it is the design point, and it is why concurrent programming discipline (locks, condition variables, careful reasoning about shared state) matters so much more inside a multithreaded process than across processes.
Worked example
Trace two threads racing on the unprotected counter increment, then again with the lock, to make the abstract danger concrete. Assume `counter` starts at `0` and each thread does exactly one increment, and the underlying load/add/store steps are visible as separate machine instructions.
Without a lock — one bad interleaving:
| Time step | Thread A | Thread B | `counter` in memory |
|---|---|---|---|
| 1 | load counter → reg_A (reads 0) | — | 0 |
| 2 | — | load counter → reg_B (reads 0) | 0 |
| 3 | reg_A = reg_A + 1 (reg_A = 1) | — | 0 |
| 4 | — | reg_B = reg_B + 1 (reg_B = 1) | 0 |
| 5 | store reg_A → counter (writes 1) | — | 1 |
| 6 | — | store reg_B → counter (writes 1) | 1 |
Two increments occurred, but the final value is `1`, not `2`. Thread B's load happened before Thread A's store was visible, so B's arithmetic used the stale value `0`. This is a lost update, and it is entirely a function of scheduling luck — a different interleaving (A fully finishes, then B starts) would have produced the correct `2`.
With the lock — the only interleavings the protocol allows:
| Time step | Thread A | Thread B | `counter` |
|---|---|---|---|
| 1 | acquire(lock) — succeeds | acquire(lock) — blocks, lock held by A | 0 |
| 2 | load, add, store (0 → 1) | still blocked | 1 |
| 3 | release(lock) | wakes, acquire(lock) — succeeds | 1 |
| 4 | (done) | load, add, store (1 → 2) | 2 |
| 5 | — | release(lock) | 2 |
The lock forces B's entire critical section to happen strictly after A's, eliminating the interleaving that caused the lost update. Generalizing the induction sketch from the previous section: with `t` threads each doing `m` increments under a correctly implemented lock, the critical sections form one sequence of `t·m` non-overlapping increments, so the final value is exactly `t·m` — matching the `N = 1000, t = N_threads` computation in the pseudocode's `main`, which is why that program prints a deterministic `final counter = 1000 · N` on every run, regardless of scheduling.
Common pitfalls & traps
| Pitfall | Why it happens | Consequence |
|---|---|---|
| Unprotected shared state | Assuming `x = x + 1` is one atomic step | Lost updates, races, nondeterministic bugs that vanish under a debugger |
| Forgetting to `join` | Treating threads like fire-and-forget calls | Main thread exits while workers are still writing shared data, or resources leak |
| Deadlock from lock ordering | Thread A locks L1 then wants L2; Thread B locks L2 then wants L1 | Both threads block forever, waiting on each other |
| Assuming single-core intuition on multicore | Testing only on one core hides races that need true simultaneity to trigger | "Works on my laptop," fails intermittently in production |
| Returning a pointer to a thread's local stack variable | Local variables live on the thread's private stack, which is invalid after the thread exits | Use-after-free, corrupted memory read by the joining thread |
| Holding a lock across a blocking I/O call | Convenient to wrap "everything" in one lock | Every other thread waiting on that lock stalls for the duration of the I/O |
Pioneers & history
The thread abstraction grew out of decades of work on making processes cheaper and concurrency more expressive, converging by the 1990s into the operating-system and library concepts used today.
Origins · 1 minPer Brinch Hansen and how this was discovered
Brinch Hansen designed the RC 4000 multiprogramming system and later formalized monitors as a structured way to organize shared data and the synchronization that protects it, directly shaping how thread-safe critical sections are taught and implemented.
Where this matters
Every major web and application server — Apache's worker model, Java application servers, database engines like PostgreSQL's and MySQL's connection-handling internals — uses threads to serve many clients concurrently without paying per-client process overhead. Graphical user interfaces keep one thread dedicated to the responsive rendering loop while background threads perform file I/O or network requests, so the interface never freezes. Web browsers use threads for parsing, rendering, networking, and JavaScript execution running in overlapping flows. Scientific and machine-learning frameworks split matrix and tensor computations across threads to exploit every core on a multicore or many-core processor. Operating system kernels themselves are internally multithreaded, handling interrupts, device drivers, and system calls as concurrent flows that must be synchronized with exactly the discipline described here. Any system where "handle many things at once, sharing state efficiently" is a requirement rests on the thread abstraction.
Ask the tutor: How do user-level thread libraries differ from kernel-managed threads in how they get scheduled onto CPUs?
Is a thread the same thing as a process?
No. A process is a container providing an address space, open files, and permissions; a thread is one flow of control that executes inside a process. A process always has at least one thread. Multiple threads inside one process share that process's address space and resources, while separate processes each get their own private address space unless they explicitly set up shared memory. The practical difference shows up in cost and safety: creating and switching between threads is cheap because no new address space is built, but that sharing means threads can corrupt each other's data if not synchronized, whereas processes are naturally isolated from one another.
Why not just use processes everywhere and skip threads entirely?
Processes give strong isolation, but that isolation has a cost: creating one duplicates address-space bookkeeping, and communicating between processes requires explicit channels like pipes or shared memory segments, which add latency and complexity. Many workloads — a web server handling thousands of connections, a GUI overlapping rendering with I/O — need cheap creation and near-free communication far more than they need isolation between the concurrent parts. Threads deliver exactly that by sharing an address space. The cost is that programmers must add explicit synchronization to prevent races, but for high-throughput, tightly-coupled concurrent work, that tradeoff usually wins.
Does using threads always make a program faster?
Not automatically. On a single CPU core, threads provide concurrency — overlapping progress, useful for hiding I/O latency — but not true simultaneous computation, since only one thread can execute per core at any instant. On multiple cores, threads can achieve real parallelism and speed up CPU-bound work, but only up to the number of cores available, and only if the work can be divided with limited contention over shared locks. Excessive locking, contended shared data structures, or workloads that are inherently sequential can make a multithreaded version slower than a well-written single-threaded one, due to synchronization overhead and cache effects.
What exactly is a race condition versus a deadlock?
A race condition is a correctness bug: unsynchronized concurrent access to shared state produces a result that depends on arbitrary timing, and some interleavings give a wrong answer. A deadlock is a liveness bug: two or more threads each hold a resource the other needs and each waits forever, so no thread ever makes progress. They are different failure modes with different symptoms — a race condition typically manifests as an occasionally-wrong value, while a deadlock manifests as the program freezing entirely — and different classic fixes: locks and atomic operations address races, while consistent lock-ordering or timeout-based acquisition address deadlocks.
Why does the stack have to be private to each thread if the heap is shared?
The stack holds local variables, function-call return addresses, and saved register state for the sequence of function calls a thread is currently inside. If two threads shared one stack, their function calls would interleave and corrupt each other's local variables and return addresses almost immediately, since each thread has its own independent call sequence. The heap and globals, by contrast, are data the programmer explicitly intends to share across threads, so keeping them common while giving each thread its own private stack is precisely what lets threads cooperate on shared data while still executing independent code paths without stepping on each other's function-call bookkeeping.
How does a program even decide which thread runs next?
That decision is made by a scheduler, the same general machinery that decides which process runs next, just applied at thread granularity. A thread can be ready, running, or blocked (for example, waiting on I/O or a lock), and the scheduler picks among ready threads according to a policy such as round robin or priority. The exact mechanics of saving one thread's registers and loading another's are the concern of Context Switching, and whether the scheduler doing this picking lives inside the kernel, inside a user-space library, or both, is the concern of User-Level vs Kernel-Level Threads.
Can threads communicate without any synchronization at all?
Technically yes — since they share memory, one thread can write a value that another later reads, with no API call in between. But without an explicit synchronization mechanism (a lock, a memory barrier, a condition variable, an atomic operation), there is no guarantee about when the writing thread's update becomes visible to the reading thread, nor any guarantee against two threads interleaving harmfully on the same variable. "It shares memory" is not the same as "it communicates correctly" — correctness requires an explicit, reasoned-about synchronization discipline layered on top of the sharing.