TCP Flow Control
Imagine a fast server streaming video to a phone. The phone's operating system has only a few kilobytes of free socket buffer space. If the server sends data as fast as the network allows, the phone's kernel buffer fills. Arriving segments are discarded, and the connection collapses into retransmissions. This is precisely the failure mode that TCP flow control exists to prevent. It is a receiver-driven mechanism that stops a fast sender from overwhelming a slow receiver. Flow control matters because TCP is a byte-stream abstraction laid over unreliable, asynchronous hardware. Senders and receivers run at different speeds, and receiving applications may read slowly. Buffer memory is also finite. Without a feedback signal telling the sender how much room remains, any mismatch between production and consumption rates causes data loss. This loss stems purely from local resource exhaustion, a problem unrelated to network congestion. Flow control solves the end-host half of TCP's reliability story. Its sibling mechanism, TCP Congestion Control, solves the network half. It reacts to shared-link contention rather than receiver capacity. The big picture is a closed feedback loop. Every acknowledgment TCP sends carries not just "data up to byte N received." It also carries a receive window (rwnd) field. This field announces how many additional bytes the receiver is currently willing to buffer. The sender maintains this value alongside its own TCP Connection Management state. It caps outstanding, unacknowledged data at that limit. As the receiving application drains its socket buffer, the advertised window grows. The sender is then permitted to push more data. As the buffer fills, the window shrinks toward zero. This throttles the sender automatically. This article gives a precise operational model of flow control. It covers the sliding-window mechanics and the persist-timer fix for the "silly window syndrome." It also includes pseudocode for both sender and receiver window bookkeeping. A worked byte-level trace is provided, too. Finally, it covers the classic pitfalls that show up in real socket-programming bugs and TCP stack implementations.
- TCP flow control prevents a fast sender from overrunning a slow receiver's buffer — a receiver-capacity problem, not a network-capacity problem.
- The receiver advertises a receive window (rwnd) in every ACK; the sender never has more than `rwnd` bytes of unacknowledged data in flight.
- The window is dynamic: it shrinks as the receiver's buffer fills and grows as the receiving application reads data out.
- A zero window halts the sender entirely; a persist timer with small probe segments prevents deadlock if the reopening ACK is lost.
- Silly window syndrome (tiny window updates, tiny segments) is avoided via Clark's algorithm on the receiver and Nagle's algorithm on the sender.
- Flow control is orthogonal to, and layered beneath, TCP Congestion Control — the sender's actual window is `min(rwnd, cwnd)`.
The receiver tells the sender, in every ACK, exactly how many bytes of buffer space remain — and the sender obeys that number as a hard ceiling on unacknowledged data.
The idea
TCP flow control is a receiver-driven, sliding-window rate-limiting scheme. Each TCP receiver maintains a finite socket receive buffer; bytes arrive from the network into this buffer and are removed only when the application calls `read()`/`recv()`. The receiver continuously computes ``` rwnd = ReceiveBufferSize − (LastByteReceived − LastByteRead) ``` and places this value in the 16-bit Window field of every outgoing TCP segment (scaled by the window-scale option when negotiated at connection setup, per TCP Connection Management). The sender tracks the receiver's most recently advertised window and enforces the invariant ``` LastByteSent − LastByteAcked ≤ rwnd ``` meaning the amount of data in flight — sent but not yet acknowledged — can never exceed what the receiver has said it can hold. This is distinct from the congestion window (cwnd), a sender-side estimate of network capacity; the two are combined by taking the minimum, `EffectiveWindow = min(rwnd, cwnd)`, so flow control and congestion control cooperate without either subsystem needing to know about the other's internal state. The mechanism is called "sliding window" because as bytes are acknowledged, the left edge of the sender's outstanding-data window slides forward, and as the receiver reports more space, the right edge slides forward too — the window slides along the byte stream rather than resetting. A receiver window of zero is a legitimate, important state: it tells the sender "stop completely," used when the receiving application has stalled (e.g., a slow disk write or a paused video decoder) and the buffer is full.
How it works
The protocol involves cooperating logic on both endpoints, plus a special recovery mechanism for the zero-window case.
Receiver side, on every incoming segment: 1. Copy payload bytes into the receive buffer (if in-sequence; TCP also buffers out-of-order segments up to buffer capacity, but flow control counts all buffered bytes, not just in-order ones). 2. Recompute `rwnd` from current buffer occupancy. 3. Send an ACK (cumulative, per TCP's acknowledgment scheme) carrying the new `rwnd`.
Sender side, maintaining a running window: 1. On receiving an ACK, update `LastByteAcked` and record the advertised `rwnd`. 2. Compute `EffectiveWindow = min(rwnd, cwnd) − (LastByteSent − LastByteAcked)`. 3. Send up to `EffectiveWindow` new bytes from the application's write buffer. 4. If `rwnd` drops to zero, stop sending data segments (except 1-byte probes) and arm the persist timer. The tricky correctness issue is the zero-window deadlock: suppose the receiver's window hits zero and later the application drains the buffer, so the receiver wants to reopen it. It sends a window-update ACK — but ACKs are not retransmitted or acknowledged themselves in TCP. If that specific update segment is lost, the sender is stuck waiting forever for a window opening that already happened and will never be resent, while the receiver waits forever for data that the sender believes it isn't allowed to send. TCP breaks this deadlock with a persist timer: after the window closes, the sender periodically (with exponential backoff, capped at some maximum, commonly 60 seconds) sends a tiny window probe segment (1 byte of new or already-sent data) purely to solicit a fresh ACK carrying the current window, regardless of whether a previous update was lost. A second efficiency issue is silly window syndrome (SWS): if the receiver advertises tiny window increments (e.g., 1 byte at a time as the application reads slowly) and/or the sender transmits tiny segments as soon as any window opens, the connection degenerates into a storm of minuscule segments, each carrying enormous relative header overhead. Two complementary fixes exist:
- Clark's algorithm (receiver side): do not advertise a window update until either a significant fraction of the buffer (typically ≥ MSS, or ≥ half the buffer) has become free, or the buffer is completely empty. Small increments are deferred and coalesced.
- Nagle's algorithm / sender-side SWS avoidance: do not send a small segment (less than one MSS) until either a full MSS worth of data is available, all outstanding data has been acknowledged, or a timeout occurs — preventing a burst of undersized packets even when a nonzero window exists.
``` PSEUDOCODE: Receiver-side flow control bookkeeping state: RecvBufSize // fixed capacity of the socket receive buffer LastByteReceived // highest sequence number written into buffer LastByteRead // highest sequence number consumed by application LastAdvertisedWindow // rwnd value most recently sent to peer function onSegmentArrival(segment): if segment.seq == expectedNext: writeToBuffer(segment.payload) LastByteReceived += len(segment.payload) deliverInOrderDataToApp() // may also advance LastByteRead else: bufferOutOfOrder(segment) // still consumes buffer space newWindow = RecvBufSize - (LastByteReceived - LastByteRead) newWindow = max(newWindow, 0) if shouldAdvertise(newWindow): // Clark's algorithm send ACK(ack = expectedNext, window = newWindow) LastAdvertisedWindow = newWindow else: send ACK(ack = expectedNext, window = LastAdvertisedWindow) // no update yet function shouldAdvertise(newWindow): delta = newWindow - LastAdvertisedWindow return (delta >= min(MSS, RecvBufSize / 2)) or (newWindow == RecvBufSize)
PSEUDOCODE: Sender-side flow control + persist timer state: LastByteSent, LastByteAcked rwnd = MSS // initial assumption before first ACK cwnd // congestion window, maintained separately persistTimerActive = false function onAckReceived(ack): LastByteAcked = max(LastByteAcked, ack.ackNum) rwnd = ack.window if rwnd > 0 and persistTimerActive: cancelPersistTimer() persistTimerActive = false trySend() function trySend(): effWindow = min(rwnd, cwnd) - (LastByteSent - LastByteAcked) if effWindow <= 0: if rwnd == 0 and not persistTimerActive: armPersistTimer(initialBackoff) persistTimerActive = true return bytesToSend = min(effWindow, availableAppData(), MSS) if bytesToSend > 0 and (bytesToSend == MSS or noOutstandingUnacked() or nagleTimeoutExpired()): send(nextBytes(bytesToSend)) LastByteSent += bytesToSend function onPersistTimerFire(): sendWindowProbe(1 byte) // solicits a fresh window advertisement armPersistTimer(min(2 * currentBackoff, MAX_BACKOFF)) ```
Complexity & correctness
Space. The receiver needs O(1) extra state beyond the buffer itself — a few integers (`LastByteReceived`, `LastByteRead`, `LastAdvertisedWindow`). The receive buffer proper is O(B) where B is the configured socket buffer size, a tunable system parameter, not something that grows with connection duration. The sender likewise needs O(1) bookkeeping state plus O(W) buffering for unacknowledged data, where W is the current effective window — bounded by `min(rwnd, cwnd)`, hence bounded by the receiver's declared capacity.
Time. Every operation — updating `rwnd` on packet arrival, deciding whether to advertise a window update, deciding how much to send — is O(1) per segment. There is no search or scan: the whole mechanism is arithmetic on sequence-number counters. Over a connection carrying N segments total, the amortized per-segment overhead is O(1), so total work is O(N), which is optimal since every segment must be inspected at least once.
Correctness (safety): buffer never overflows. The key invariant the mechanism maintains is: ``` Invariant I: LastByteSent − LastByteAcked ≤ rwnd_last_advertised ``` Proof sketch. Initially both sides are zero, so I holds trivially. The sender only increases `LastByteSent` inside `trySend()`, and it does so by at most `effWindow = min(rwnd, cwnd) − (LastByteSent − LastByteAcked)` bytes — by construction this cannot push `LastByteSent − LastByteAcked` above `rwnd`. The receiver only decreases the declared `rwnd` monotonically-or-correctly as bytes accumulate, and it advertises `rwnd = RecvBufSize − (LastByteReceived − LastByteRead)`, which by definition equals the receiver's actual remaining buffer capacity at the instant the ACK is generated. Because the sender never has more unacknowledged bytes in flight than the most recent `rwnd` it learned, and because that `rwnd` was an accurate snapshot of free buffer space, the receiver can always accommodate whatever the sender has "in flight" toward it — provided no newer segments arrive faster than the buffer accounting reflects, which cannot happen because the sender is causally blocked by the invariant itself. Hence buffer overflow due to sender behavior is structurally impossible, modulo network reordering delivering more out-of-order data than accounted (handled because out-of-order bytes are also counted against `LastByteReceived` in the pseudocode above). ∎
Correctness (liveness): no permanent stall. Without the persist timer, a lost window-update ACK could stall the connection forever, since ACKs are not reliably retransmitted by TCP's normal retransmission machinery (only data segments are retransmitted on timeout, because only data carries a sequence number range that maps to buffered, resendable bytes). The persist timer restores liveness: it guarantees that whenever `rwnd` has been zero for any positive duration, the sender will, within bounded time (exponential backoff capped at a maximum), transmit a probe that forces the receiver to emit a fresh ACK — and the current `rwnd` value in that ACK is always the freshest available snapshot, so the deadlock cannot persist beyond one backoff cycle after the receiver's buffer actually reopens.
Tradeoffs. Flow control trades some throughput for safety: if the receive buffer is too small relative to the bandwidth-delay product, the window will cap throughput below the link's actual capacity — a real limitation before window scaling (RFC 1323) let 16-bit window fields represent up to a gigabyte of buffer via a scale factor. Overly aggressive Clark/Nagle coalescing can add latency to interactive traffic; this is the classic tension addressed by `TCP_NODELAY` for latency-sensitive applications. Flow control also does nothing for network congestion — a receiver with an enormous, ever-open buffer provides no protection against overloading routers in the middle of the path, which is exactly the gap TCP Congestion Control fills.
Worked example
Consider a receiver with `RecvBufSize = 6000` bytes and `MSS = 1000` bytes, connected to a sender with plenty of data queued and an (for this trace) unconstrained `cwnd`, so `EffectiveWindow` is driven purely by `rwnd`. The application reads from the socket slowly and unevenly. Sequence numbers below are relative byte offsets for clarity.
| Step | Event | LastByteReceived | LastByteRead | rwnd advertised | Sender action |
|---|---|---|---|---|---|
| 0 | Connection established | 0 | 0 | 6000 | Sender may send up to 6000 bytes |
| 1 | Sender sends bytes 0–999, 1000–1999, 2000–2999 (3 segments, 3000 bytes) | — | — | — | `LastByteSent = 3000` |
| 2 | Receiver gets seg 1 (0–999), app hasn't read yet | 1000 | 0 | 5000 | ACK(ack=1000, win=5000) |
| 3 | Receiver gets seg 2 (1000–1999), app still hasn't read | 2000 | 0 | 4000 | ACK(ack=2000, win=4000) |
| 4 | Receiver gets seg 3 (2000–2999), app reads 500 bytes | 3000 | 500 | 3500 | ACK(ack=3000, win=3500); sender now has `inflight = 3000-3000=0` free to send up to 3500 more |
| 5 | Sender sends bytes 3000–3999, 4000–4999, 5000–5999 (3000 more, hits `EffectiveWindow`) | — | — | — | `LastByteSent = 6000`, `LastByteAcked = 3000`; inflight = 3000, exactly at limit ⇒ sender stalls new sends |
| 6 | Receiver gets seg 4 (3000–3999); app reads nothing more; buffer occupancy = 4000−500=3500 used, but capacity is 6000 so 6000-(4000-500)=2500 | 4000 | 500 | 2500 | ACK(ack=4000, win=2500) |
| 7 | Receiver gets seg 5, 6 (4000–5999); app still not reading; occupancy = 6000−500=5500 used | 6000 | 500 | 500 | ACK(ack=6000, win=500) |
| 8 | Sender computes `inflight = 6000−6000=0`, `EffectiveWindow=min(500, cwnd)=500`; sends bytes 6000–6499 (500 bytes, capped below MSS) | — | — | — | `LastByteSent=6500` |
| 9 | App finally reads all 6000 buffered bytes at once | 6500 | 6500 | 6000 | ACK(ack=6500, win=6000) — large jump satisfies Clark's threshold, advertised immediately |
| 10 | Sender resumes at full rate | — | — | — | inflight=0, can send up to 6000 bytes again |
Two things this trace makes concrete: first, the window genuinely throttles the sender at step 5–8 down to 500-byte bursts purely because the application, not the network, was slow to read — flow control at work exactly as intended. Second, notice the receiver did not advertise the shrinking window in tiny decrements (it moved 6000→5000→4000→3500→2500→500, not by dribbling single-byte updates) — and on reopening it waited to jump straight to 6000 rather than trickling 500-byte increments, which is Clark's algorithm suppressing silly-window-syndrome churn. Now suppose at step 7 the ACK carrying `win=500` had been the receiver's last update before the app stalled completely for an hour, and that a subsequent smaller window-update ACK (say, after a partial read) was lost in the network. Without a persist timer, the sender would sit forever believing `rwnd=500`, having already used exactly that much room at step 8, computing `EffectiveWindow=0` and never sending again — even after the receiver later drains everything. The persist timer's periodic 1-byte probes are what eventually forces a fresh, accurate ACK through and unsticks the connection.
Common pitfalls & traps
| Pitfall | Why it happens | Consequence |
|---|---|---|
| Forgetting the persist timer | Assuming ACKs are reliably delivered like data | Permanent stall if a window-reopening ACK is lost |
| Advertising rwnd in 1-byte increments | Naive implementation updates window on every read | Silly window syndrome: tiny segments, huge relative overhead |
| Confusing rwnd with cwnd | Both are "windows" limiting the sender | Misdiagnosing a congestion-drop problem as a receiver-buffer problem or vice versa |
| Ignoring window scaling | 16-bit Window field caps rwnd at 65535 without the option | Throughput ceiling far below link's bandwidth-delay product on fast/long paths |
| Counting only in-order bytes toward buffer occupancy | Forgetting that out-of-order segments also consume buffer space | Receiver advertises more room than truly exists, risking overflow or discarding needed segments |
| Treating window updates as reliably retransmitted like data | Misunderstanding TCP's ACK semantics | Building custom protocols that deadlock the same way TCP would without a persist mechanism |
Pioneers & history
TCP's sliding-window flow control emerged directly from the design of the ARPANET-era transport protocols, formalized in the original TCP specification and later refined through operational experience with silly window syndrome and small-window pathologies observed in early Berkeley Unix networking stacks.
Origins · 1 minVinton G. Cerf and how this was discovered
Cerf, together with Robert Kahn, designed the original TCP protocol in the 1970s, embedding sliding-window acknowledgment and flow control directly into the transport layer so that heterogeneous networks could interconnect without every intermediate hop needing to guarantee reliability.
Clark's 1982 analysis of small-window and small-segment pathologies in early TCP deployments produced the receiver-side algorithm — deferring window updates until a meaningful chunk of buffer frees up — that remains standard practice in every modern TCP stack.
Where this matters
Every TCP connection on the internet — web browsers fetching pages, SSH sessions, database replication links, video-conferencing control channels riding over TCP fallback paths — relies on flow control to keep a fast server from drowning a slow or memory-constrained client. Operating system socket buffer tuning (`SO_RCVBUF`, `net.core.rmem_max` on Linux) directly determines the maximum `rwnd` a host can ever advertise, which in turn caps achievable throughput on high-bandwidth-delay-product paths unless window scaling is enabled and negotiated during TCP Connection Management's three-way handshake. Mobile devices, embedded IoT endpoints, and virtualized cloud instances with tight memory budgets are especially sensitive to receive-buffer sizing, since an under-provisioned buffer silently throttles throughput long before the network path itself becomes the bottleneck. Load balancers and reverse proxies (nginx, HAProxy) that terminate TCP on behalf of backend servers must manage flow control on both the client-facing and backend-facing connections independently, since each has its own receive buffer and its own advertised window; understanding this separation is essential for diagnosing throughput anomalies that show up as "slow client" versus "slow backend" symptoms in production traffic. The same conceptual pattern — a receiver publishing consumption capacity to throttle a producer — reappears in application-layer protocols like gRPC/HTTP-2 flow control and in message queue backpressure schemes, all descendants of the idea pioneered in TCP.
Ask the tutor: How does the sender's effective window interact with the congestion window during TCP's slow-start phase?
Is TCP flow control the same thing as TCP congestion control?
No — they solve different problems using a shared mechanical pattern (a window that caps in-flight bytes) but respond to different signals. Flow control reacts to the receiver's buffer occupancy, announced explicitly via the `rwnd` field in every ACK; it protects one specific receiving host from being overwhelmed. Congestion control reacts to inferred network-path conditions — packet loss, delay increases, or explicit congestion notification — and is entirely sender-side reasoning about shared infrastructure like router queues. The sender combines both by taking `EffectiveWindow = min(rwnd, cwnd)`, so either mechanism alone can throttle transmission. A receiver with unlimited buffer space and an always-open rwnd provides zero protection against congesting a bottleneck router in the middle of the path — that gap is exactly what TCP Congestion Control closes.
What happens if the receive window is advertised as zero?
A zero window is a legitimate, deliberate signal meaning "stop sending data entirely; my buffer is full." The sender must comply, halting all data transmission but is still permitted to send 1-byte window probe segments once a persist timer fires, purely to solicit fresh ACKs. This is essential because ordinary TCP ACKs carrying window updates are not retransmitted if lost — unlike data segments, they carry no sequence range that maps to buffered application bytes, so TCP's normal timeout-and-retransmit logic does not apply to them. Without persist-timer probing, a single lost window-reopening ACK would deadlock the connection permanently: the sender waits for an update that already happened but was never resent, while the receiver waits for data the sender believes it cannot send.
What is silly window syndrome and why is it bad?
Silly window syndrome (SWS) is a degenerate operating mode where a TCP connection settles into exchanging many tiny segments — sometimes just a few bytes of payload each — instead of efficient, near-MSS-sized segments. It arises when a slow-reading receiver advertises minuscule window increments (e.g., 1 byte at a time as the application trickles reads) and/or an eager sender immediately transmits the instant any window opens, however small. The result is drastically increased relative overhead: each segment carries roughly 40 bytes of IP/TCP header regardless of payload size, so sending 1-byte payloads wastes the vast majority of bandwidth on headers. The fix is two-sided: Clark's algorithm delays receiver-side window updates until a meaningful chunk (roughly one MSS or half the buffer) is free, and Nagle's algorithm delays sender-side transmission of sub-MSS segments until enough data accumulates or outstanding data is acknowledged.
How does window scaling work, and why was it needed?
The original TCP header allocates only 16 bits to the Window field, capping any single advertised `rwnd` at 65,535 bytes. On high-bandwidth, high-latency paths (large bandwidth-delay product, such as satellite or long-haul fiber links), this ceiling becomes the binding constraint on throughput long before the physical network is saturated — a sender simply cannot have more than 65 KB in flight regardless of link speed. RFC 1323's window scale option, negotiated once during the initial TCP Connection Management handshake (it cannot be introduced mid-connection), attaches a shift factor so the effective advertised window becomes `rwnd_field << scale_factor`, supporting windows up to roughly one gigabyte. Both endpoints must include the option in their SYN segments for scaling to be used on either direction of the connection.
Can a receiver reduce its advertised window after already advertising a larger one?
TCP allows it in principle — the receiver's application could, for instance, allocate less socket buffer memory — but shrinking an already-advertised window is discouraged and considered unsafe in general, because the sender may have already transmitted data that fits within the old, larger window but now exceeds the new, smaller one. RFC 793 and subsequent guidance recommend that a receiver never reduce the right edge of the window it has already advertised (i.e., never retract previously granted permission), instead only holding the right edge steady while `LastByteRead` catches up, or growing it. Implementations that violate this can cause spurious data loss or confusing retransmission behavior, since the sender reasonably assumes previously granted send permission remains valid.
How does flow control interact with out-of-order segment arrival?
Because TCP receivers buffer out-of-order segments (arriving ahead of the expected sequence number) so they can later be reassembled without forcing the sender to retransmit already-delivered data, those out-of-order bytes occupy real buffer space even though the application cannot yet read them. A correct implementation must subtract all buffered bytes — in-order and out-of-order alike — from `RecvBufSize` when computing `rwnd`, not just the contiguous prefix delivered to the application. Implementations that miscount this (crediting buffer space back to the sender for bytes that are physically still occupying memory as out-of-order fragments) risk genuine buffer overflow, defeating the entire safety guarantee that flow control is meant to provide.
Why doesn't flow control alone prevent network congestion collapse?
Flow control only regulates the relationship between one sender and one receiver's local buffer; it has no visibility into shared routers, switches, or links that many independent connections traverse simultaneously. A receiver could advertise an enormous, always-open window (ample local RAM) while the path between sender and receiver passes through a heavily congested router serving hundreds of other flows — flow control would place no restraint on the sender in that scenario, even though flooding that router causes queueing delay and packet loss for everyone sharing it. This is exactly the failure mode that produced early-internet congestion collapse events, motivating the separate development of TCP Congestion Control mechanisms (slow start, congestion avoidance, fast retransmit) that infer and react to network-wide conditions independently of receiver buffer state.