Memory-Mapped Files
Imagine a database engine that must scan a 50 GB index file. The naive approach opens the file and calls `read()` repeatedly into a buffer. Each block gets copied twice: once from disk into the kernel's page cache. Then again from the kernel into user space. Every access pays a system-call tax and a memory-copy tax. This happens even though the operating system already has the data sitting in RAM. Memory-mapped files eliminate this waste. They make file access look exactly like memory access. The file's bytes are placed directly into the process's address space. Reading them is just a load instruction that may trigger a Page Fault. This is one of the most elegant unifications in operating systems design. Instead of treating "files" and "memory" as separate abstractions with separate APIs, `mmap` collapses them into one mechanism. This mechanism is built on the paging hardware already present in every CPU. A memory-mapped region behaves like ordinary heap memory. The exception is that its backing store is a file on disk, a device, or another process's memory, rather than swap space. The kernel's demand-paging machinery — page tables, page faults, the page cache — does double duty as the file I/O engine. The payoff is enormous in real systems. Database engines (SQLite, LMDB, many key-value stores), dynamic linkers (loading shared libraries), and big-data frameworks all rely on `mmap`. They use it to avoid redundant copies, and to let the kernel manage caching intelligently. It also lets multiple processes share a single physical copy of a file's data. Shared library loading in every modern operating system works this way. It is implemented as a memory mapping of the `.so`/`.dll` file. Pages are shared read-only across every process running that program. This article explains precisely how `mmap` places a file into an address space. It covers how page faults are serviced from the file's data on first touch. It explains how private (copy-on-write) mappings differ from shared mappings. It also explains why this design is both time-efficient and semantically subtle. We build the mental model step by step: address space layout, the fault handler, the page cache, and the write-back path. We end with a worked trace and the classic pitfalls that trip up systems programmers.
- `mmap` places a file's contents into a process's virtual address space instead of copying it via `read`/`write`.
- Pages of the mapping are populated lazily: the first access to an unmapped page triggers a page fault, and the fault handler loads the data from the file.
- Shared mappings let multiple processes see the same physical page cache and write changes back to the file; private mappings use copy-on-write so writes stay local to one process.
- The kernel's unified page cache means memory-mapped I/O and `read`/`write` I/O on the same file are automatically kept coherent (same cache, same physical pages).
- `mmap` avoids the extra kernel-to-user copy that `read` requires, which matters enormously for large or randomly accessed files.
- Dirty pages of a mapping are written back to disk asynchronously (or on `msync`/unmap), which trades an explicit `write()` call for implicit, OS-scheduled write-back.
A memory-mapped file turns file I/O into ordinary memory access by letting the demand-paging fault handler fetch file blocks into page-cache pages that are directly mapped into the process's page table.
The idea
A memory-mapped file is a region of a process's virtual address space whose pages are backed by a file rather than by anonymous swap space. The system call, canonically `mmap(addr, length, prot, flags, fd, offset)` on POSIX systems (or `MapViewOfFile` on Windows), asks the kernel to reserve `length` bytes of virtual address space starting near `addr`, and to associate that region with the bytes of file descriptor `fd` starting at byte `offset`. Critically, `mmap` does not read the file. It only creates a mapping — an entry in the process's virtual memory area list and corresponding page-table entries marked "not present." The actual data transfer is deferred until the memory is touched, exactly as in ordinary Demand Paging. The two central mapping types are shared (`MAP_SHARED`) and private (`MAP_PRIVATE`). A shared mapping means the mapped pages are backed by, and can modify, the actual file: writes are visible to every other process that has the same file mapped, and (eventually) to `read()` calls and to the file itself on disk. A private mapping means writes trigger copy-on-write: the process gets its own private copy of the modified page, invisible to other mappers and never written back to the underlying file. Private mappings are how executables and shared libraries are loaded — every process mapping `libc.so` shares the same physical read-only code pages, but if a process's dynamic linker needs to patch a data page (relocations), it silently forks off a private copy without affecting anyone else. The deep unifying insight is that the kernel already maintains a page cache: physical page frames holding recently used file blocks, indexed by (file, offset). Ordinary `read()`/`write()` calls go through this cache implicitly — data is copied from a page-cache page into a user buffer. `mmap` simply removes the copy step: it maps the page-cache pages themselves directly into the user page table, using the same page-table and TLB machinery that backs regular process memory. This is why the same physical page can be simultaneously the "kernel's disk cache for the file" and "the process's mapped memory for that file" — they are, in a shared mapping, literally the same page frame.
How it works
Setting up a mapping is cheap: it only touches virtual-memory bookkeeping (a `struct vm_area` / VMA describing `[start, end)`, protection bits, the backing file and offset, and the mapping type). No physical page is allocated and no disk I/O occurs at `mmap()` time. Everything else is driven by the page fault handler, reused unchanged from ordinary demand paging.
Step 1 — mmap() call. The kernel validates arguments (alignment, permissions, whether the file is opened with compatible read/write access), inserts a new VMA into the process's address-space descriptor, and marks the corresponding page-table entries as not-present (or simply absent, to be filled lazily). It returns the starting virtual address.
Step 2 — first access, page fault. When the process reads or writes a byte in the mapped region for the first time, the MMU finds no valid translation and raises a page fault. The fault handler looks up the VMA covering the faulting address, discovers it is a file mapping, and computes the file offset: `file_offset = vma.file_offset + (fault_addr - vma.start)`.
Step 3 — page-cache lookup. The handler checks whether the needed file block is already resident in the page cache (perhaps because another process mapped or read the same file). If present, the handler simply installs a page-table entry pointing at that existing physical frame — no disk access needed. If absent, the handler allocates a free frame, issues a disk read for that block, and blocks the faulting thread until the read completes.
Step 4 — page-table entry installation. Once the frame holds valid data, the handler sets the page-table entry: read-only for `MAP_PRIVATE` mappings (so that writes can be intercepted for copy-on-write) or matching the requested protection for `MAP_SHARED` mappings. The faulting instruction is restarted and now succeeds.
Step 5 — write faults and copy-on-write. If the mapping is private and the process writes to the page, a second fault occurs (because the entry was installed read-only). The handler allocates a fresh anonymous frame, copies the file data into it, updates the page table to point at the private copy with write permission, and decrements the reference count on the original shared file-backed frame. From now on, that virtual page is disconnected from the file — later changes to the file by other processes are invisible to this one, and this process's writes never reach disk.
Step 6 — write-back for shared mappings. For a shared, writable mapping, a write fault simply marks the page-table entry writable and marks the underlying page-cache page dirty, exactly as an in-memory `write()` would. The kernel's periodic write-back daemon (or an explicit `msync()` call, or `munmap()`/process exit) later flushes dirty pages to disk, coalescing and scheduling I/O for efficiency rather than writing synchronously on every store instruction.
Step 7 — unmapping. `munmap()` removes the VMA and the associated page-table entries. For shared mappings, dirty pages are written back (or scheduled to be); for private mappings, any copy-on-write private frames are simply freed once no longer referenced. ``` // Simplified page-fault handling for a memory-mapped region function handle_page_fault(process, fault_addr, is_write): vma = process.find_vma(fault_addr) if vma is None: raise SEGV(process) // no mapping covers this address if vma.backing == FILE: file_off = vma.file_offset + align_down(fault_addr - vma.start, PAGE_SIZE) if is_write and vma.type == MAP_PRIVATE and vma.pte_present(fault_addr): // second fault: page already loaded read-only, now being written -> COW old_frame = vma.frame_for(fault_addr) new_frame = allocate_frame() copy_page(new_frame, old_frame) vma.set_pte(fault_addr, new_frame, PROT_READ | PROT_WRITE, anonymous=True) release_ref(old_frame) return RESUME // first touch: need to populate the page from the file frame = page_cache.lookup(vma.file, file_off) if frame is None: frame = allocate_frame() disk_read(vma.file, file_off, frame) // blocking I/O, page fault handler sleeps page_cache.insert(vma.file, file_off, frame) prot = PROT_READ if vma.type == MAP_SHARED and vma.prot_has(PROT_WRITE): prot |= PROT_WRITE // MAP_PRIVATE pages start read-only even if writable, to catch first write for COW vma.set_pte(fault_addr, frame, prot, anonymous=False) increment_ref(frame) return RESUME else: // anonymous (non-file) region: ordinary demand-paged memory frame = allocate_frame() zero_fill(frame) vma.set_pte(fault_addr, frame, vma.prot, anonymous=True) return RESUME ``` ``` // Write-back daemon (conceptually periodic) function writeback_sweep(): for page in page_cache.all_pages(): if page.dirty and page.mapped_shared_writable(): disk_write(page.file, page.file_offset, page.frame) page.dirty = False ```
Complexity & correctness
Time complexity. Setting up the mapping with `mmap()` is `O(1)` in the size of the file — it only creates a VMA descriptor and marks page-table entries absent; no data movement occurs. Accessing byte `i` of the file costs `O(1)` when the containing page is already resident (a TLB hit or, at worst, a page-table walk of depth equal to the number of paging levels, typically 3–5, a small constant). On first touch it costs one page fault plus one disk I/O for that page — asymptotically identical to a `read()` syscall's per-block cost, but without the extra `O(page size)` memory-copy from kernel to user buffer that `read()` requires. Over a scan of `n` bytes with page size `p`, memory mapping does `n/p` page faults (each amortized over `p` bytes) versus `read()`'s `n/p` syscalls plus `n` bytes copied twice; both are `O(n)` overall, but `mmap` has a smaller constant factor because it removes one full copy and can avoid a syscall per block after the initial mapping is warm in the TLB.
Space complexity. A mapping consumes `O(1)` additional kernel metadata (one VMA record) regardless of file size, plus page-table entries proportional to the number of resident pages, not the whole file — this is the entire point of demand paging. A 50 GB file can be mapped and only the touched pages (potentially a few megabytes) ever occupy physical RAM.
Correctness argument. The essential invariant is: > At any time, for a `MAP_SHARED` mapping, the physical frame reachable through the process's page table for virtual page `v` is exactly the page-cache frame holding file bytes `[vma.offset + v·p, vma.offset + (v+1)·p)`, and any write that reaches that frame is either already reflected in this cache entry or is asynchronously scheduled to be written back before the file is truncated, closed with sync semantics, or the frame is reused. This invariant is maintained by construction: the fault handler (Step 3–4 above) always installs a page-table entry pointing directly at the page-cache frame for the requested file offset, never a copy, so any subsequent write by any mapper is visible to all of them for free (they share physical memory) and is trivially visible to a concurrent `read()`/`write()` on the same file descriptor, because both `read()` and `mmap` funnel through the same page cache keyed by `(file, offset)`. Coherence therefore does not need explicit synchronization between the mmap path and the read/write path — they cannot diverge because they are the same cache. For `MAP_PRIVATE`, the invariant is refined: a page starts aliased to the shared file cache frame (read-only, enforcing copy-on-write), and the first write establishes a strictly private frame disconnected from the file cache, by the standard copy-on-write correctness argument used in Demand Paging and process forking — the copy happens exactly once, is triggered exactly by the write permission fault, and after the copy no further faults are needed for subsequent writes to that page (the PTE is now writable).
Tradeoffs. The efficiency gain is not unconditional. `mmap` overhead per call (VMA setup, TLB shootdowns on unmap in multi-threaded processes) can exceed the savings for small, one-shot reads — a single `read()` of a 200-byte config file is cheaper than mapping the file. Random access into a mapping that exceeds physical RAM causes uncontrolled page-fault storms (thrashing) with less flexibility than `read()`'s explicit buffering and readahead hints. Error handling is also harder: an I/O error while servicing a page fault surfaces as a hardware exception (`SIGBUS` on POSIX) delivered asynchronously at the faulting instruction, rather than a clean negative return value from `read()`.
Worked example
Consider a process mapping a 3-page (12 KB, page size 4 KB) file `data.bin` with `mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0)`, then performing these operations:
| Step | Operation | What happens | Physical state after |
|---|---|---|---|
| 1 | `mmap()` returns `0x7000` | VMA `[0x7000, 0xA000)` created, type=PRIVATE; no PTEs installed | No frames allocated |
| 2 | Read byte at `0x7000` | Page fault (not present). File offset 0 loaded from disk into frame #10, PTE installed read-only (COW guard) | Frame #10 = file page 0, shared with page cache |
| 3 | Read byte at `0x7050` | Same page as step 2 — TLB/page-table hit, no fault | Unchanged |
| 4 | Write byte at `0x7000` | Write fault: PTE was read-only. COW triggers: allocate frame #77, copy frame #10 into #77, install PTE for `0x7000` pointing at #77, writable | Frame #10 unchanged (still page-cache original); frame #77 = private dirty copy |
| 5 | Another process opens `data.bin` with `read()` at offset 0 | Reads from page-cache frame #10 (unmodified) | Sees original bytes — this process's private write in step 4 is invisible to it |
| 6 | Read byte at `0x8000` (page 1, offset 4096) | Page fault: page not resident. Loaded from disk into frame #11, PTE read-only | Frame #11 = file page 1 |
| 7 | `munmap()` the whole region | VMA removed; frame #77 (private, dirty) is simply freed — never written to `data.bin`; frames #10, #11 remain in page cache (may be reused by others) | `data.bin` on disk is completely unchanged from before mapping |
The trace demonstrates the two key correctness properties side by side: (a) reads that hit the same shared page-cache frame are cheap after the first fault (step 3), and (b) a private write never escapes to the file or to other processes (steps 4–5), precisely because copy-on-write disconnects the modified page from the shared cache entry at the moment of the first write fault. Had the mapping been `MAP_SHARED` instead, step 4 would simply mark frame #10 dirty in place, and step 5's `read()` in the other process would observe the modified byte immediately, and `munmap` (or a later write-back sweep) would persist it to `data.bin` on disk.
Common pitfalls & traps
| Pitfall | Why it happens | Consequence | Fix |
|---|---|---|---|
| Writing past `mmap`'s size when the file is shorter than the mapped length | Mapping length must be page-aligned; the tail of the last page beyond EOF is zero-filled but not backed by the file | Accessing beyond true EOF within the page reads zeros silently; writing there is lost, not extended into the file | Track and check the real file length separately from mapping length; use `ftruncate` before mapping if you intend to extend |
| Assuming `mmap` writes are durable immediately | Shared mapping writes only dirty the page cache; write-back is asynchronous | Data loss on crash/power failure before write-back or `msync()` | Call `msync(MS_SYNC)` (or `FlushViewOfFile`+`FlushFileBuffers`) when durability is required |
| Accessing a mapped page after the underlying file was truncated by another process | The VMA still exists but the page-cache backing may no longer correspond to valid file blocks | `SIGBUS` delivered on access, often far from the code that truncated the file | Coordinate truncation with unmapping, or use file locks / advisory protocols |
| Treating a page-fault I/O error as recoverable like a `read()` error code | `read()` returns `-1`/`errno`; a fault during paging cannot return an error to arbitrary user code cleanly | Delivered as `SIGBUS`, which crashes the process unless a signal handler is installed | Install a `SIGBUS` handler (with `sigsetjmp`/`siglongjmp` in C) if partial-file mapping errors must be handled gracefully |
| Forgetting that `MAP_PRIVATE` writes are invisible to other mappers and never reach disk | Confusing private (COW) semantics with shared semantics | Bugs where "the file update" mysteriously never happens, or two processes seem unable to communicate through the mapping | Use `MAP_SHARED` explicitly whenever writes must be observed by others or persisted |
| Excessive page faults from random access on files far larger than RAM | Demand paging services one page at a time; no readahead unless the OS heuristically detects sequential access | Severe thrashing, worse latency than a well-tuned buffered `read()` with explicit prefetching | Use `madvise(MADV_SEQUENTIAL/RANDOM/WILLNEED)` to hint access pattern to the page-fault/readahead logic |
Pioneers & history
The memory-mapped file abstraction grew directly out of research into virtual memory and demand paging in the 1960s and 1970s, where designers realized that treating disk-backed segments uniformly — whether they held "program," "data," or "file content" — simplified both the kernel and its performance model.
Origins · 1 minFernando J. Corbató and how this was discovered
history 1961 — CTSS at MIT introduces early time-sharing with paged virtual memory concepts. 1965 — Multics project begins, unifying segments, files, and memory under one addressing model, directly foreshadowing memory-mapped files. 1979 — 4.2BSD-era Unix research explores mapping files into address spaces, building on Multics-style segment/file unification. 1986 — SunOS introduces a widely used `mmap()` system call, cementing the POSIX-style shared/private mapping API still used today. 1993 — Windows NT ships `MapViewOfFile` and section objects, bringing the same unification of files and memory to a mainstream commercial OS.
::: The insight that a page fault handler could serve as a general-purpose "load this data" hook — rather than only "restore this page from swap" — is what let file-backed and anonymous memory share one mechanism. Multics's segmented, paged design treated every addressable object, files included, as something the hardware's paging unit could fault in on demand; Unix's `mmap` and its BSD/SunOS descendants brought a simplified, flat-address-space version of that idea into mainstream operating systems, and it has been part of every major kernel's memory manager since.
Where this matters
Dynamic linkers (`ld.so`, Windows loader) map executable and shared-library files with `MAP_PRIVATE` for code/data segments, letting every process running the same program share identical physical pages for code while keeping per-process data private via copy-on-write. Database engines such as SQLite (in its memory-mapped I/O mode), LMDB, and various LSM-tree stores map their data files directly, letting the OS's page cache double as the database buffer pool and avoiding a second, redundant application-level cache. Version control systems and big-data frameworks (`git`'s pack-file access, memory-mapped Parquet/columnar readers) use `mmap` to randomly access huge files without paying for explicit seeks and reads on every access. Persistent object stores and some garbage collectors use `mmap`'s `PROT_NONE`/guard-page tricks to implement generational heaps and write barriers efficiently. Even inter-process communication mechanisms — shared memory segments (`shm_open` + `mmap`) — are just memory-mapped "files" backed by an in-memory tmpfs-like object rather than persistent disk storage, reusing the exact same fault-handling and page-table machinery described here.
Ask the tutor: How would you design a memory-mapped ring buffer for lock-free IPC between two processes using `MAP_SHARED`?
Does mmap load the whole file into memory immediately?
No. `mmap()` only creates a virtual memory area (VMA) describing the mapping's address range, permissions, and backing file/offset; it performs no I/O. Physical pages are populated lazily, one page at a time, only when the process actually touches an address inside the mapping and triggers a page fault. This is precisely the demand-paging discipline used for ordinary process memory, applied to file-backed pages instead of anonymous swap-backed pages. As a result, mapping a multi-gigabyte file is essentially free, and memory usage tracks the working set actually accessed, not the file's total size.
What is the difference between MAP_SHARED and MAP_PRIVATE?
`MAP_SHARED` maps the actual page-cache pages of the file; writes are visible to every other process mapping the same region, and are eventually written back to the file on disk. `MAP_PRIVATE` starts identically (pages are shared, read-only, with the page cache) but uses copy-on-write: the first write to a page allocates a private, process-local copy, so subsequent writes never reach the file or other mappers. `MAP_PRIVATE` is used for loading executable code and initialized data segments, where each process needs its own mutable view without corrupting the on-disk binary or other processes' views.
How does mmap interact with the page cache used by ordinary read()/write() calls?
They are the same cache. The kernel indexes cached file blocks by `(file, offset)` regardless of how they were accessed. A `read()` call copies bytes out of a page-cache frame into a user buffer; an `mmap`'d access maps that same frame directly into the process's page table. Because both paths reference the identical physical frame, writes through one path are immediately visible through the other — there is no separate synchronization needed, because there is fundamentally only one copy of the data in memory.
What happens if I mmap a file and then another process truncates or deletes it?
If another process truncates the file to a length shorter than your mapping, accessing the pages beyond the new end-of-file raises `SIGBUS` (a bus error), because the page-fault handler cannot find corresponding file blocks to load. If the file is deleted (unlinked) while still mapped, on POSIX systems the underlying inode and its data remain available as long as any process holds an open file descriptor or a mapping to it — deletion only removes the directory entry, not the data, so your mapping continues to work until unmapped.
Why does a page fault on a private mapping sometimes happen twice for the same page?
The first fault occurs because the page is not yet resident at all: the handler loads it from disk into a page-cache frame and installs a read-only page-table entry, even though the mapping requested read/write access. This read-only installation is intentional — it lets the kernel detect the first write to that page. The second fault, triggered only if you write to the page, is the copy-on-write fault: the handler now allocates a private frame, copies the data, and installs a writable entry pointing at the private copy. Read-only pages never incur this second fault.
Is mmap always faster than read()/write()?
No. For small files or single-pass sequential reads of modest size, the fixed overhead of setting up a VMA and handling page faults can exceed the cost of a couple of `read()` syscalls with a well-sized buffer, especially since `read()` benefits from kernel-side readahead heuristics tuned over decades. `mmap` wins clearly for large files with random or repeated access, for sharing data between processes without copying, and for reducing memory pressure from duplicate buffering. It can also complicate error handling, since I/O errors surface as signals rather than return codes.
How do memory-mapped files relate to demand paging conceptually?
A memory-mapped file is demand paging, with the backing store generalized from "swap space holding a previously evicted anonymous page" to "a specific byte range of a named file." The exact same trio of concepts — a not-present page-table entry, a page fault, and a fault handler that fetches data into a frame before resuming execution — is reused unchanged. The only new ingredients are the VMA's file/offset metadata and the shared-vs-private write semantics that determine what happens to modified pages afterward.