The TLS Handshake

A TLS handshake must achieve three properties simultaneously: secrecy (an eavesdropper learns nothing about the session key), authentication (the client knows it is talking to the intended server, and optionally vice versa), and integrity/freshness (neither party can be tricked into using a stale or partially forged transcript). No single primitive delivers all three. Diffie-Hellman exchange gives two strangers a shared secret over a public channel, but by itself is vulnerable to a man-in-the-middle: an attacker can run separate DH exchanges with each party and relay traffic, since raw DH values carry no identity. Digital signatures let a party prove possession of a private key, but a signature over nothing session-specific can be replayed. Certificates issued under a PKI bind a public key to a name, but only if the relying party actually checks the chain and it is fresh. TLS 1.3's core design is to make every ephemeral DH share and every subsequent derived key cryptographically dependent on a running hash of the entire handshake transcript, and then have the server (and optionally the client) sign that transcript hash with its long-term certified key. This single signature — sent in the `CertificateVerify` message — is what glues the anonymous, forward-secret DH exchange to a verifiable identity. Because the hash covers every message exchanged so far, including the DH shares themselves, an attacker cannot substitute a different DH share without the signature failing to verify against the tampered transcript. The final ingredient is the key schedule: a sequence of HKDF (HMAC-based Key Derivation Function) extract-and-expand steps that turns the raw DH shared secret into a tree of independent keys — handshake traffic keys, application traffic keys, exporter keys, and resumption keys — each domain-separated by labels so that compromise of one key does not reveal another. `Finished` messages, computed as a MAC over the transcript keyed by a handshake traffic secret, serve as the last proof that both parties independently derived matching keys, closing the loop before any application data is sent. Together these mechanisms deliver a handshake that is only one round trip in the common case, is fully forward secret since the DH shares are ephemeral and discarded, and is deniable of long-term-key compromise for the client if client authentication is not used.

The TLS 1.3 handshake (RFC 8446) proceeds in flights. The client sends its first flight guessing which DH groups the server supports; if the guess is right, only one round trip is needed before application data flows.

Flight 1 — ClientHello. The client picks a random nonce, offers a list of symmetric cipher suites, supported groups (e.g., X25519, secp256r1), signature algorithms, and — critically — sends one or more key_share entries: ephemeral DH public values for its guessed groups.

Flight 2 — ServerHello + encrypted extensions + Certificate + CertificateVerify + Finished. The server picks a group it supports from the client's offer, generates its own ephemeral DH keypair, and computes the shared secret. From this point on, the server encrypts the rest of its flight under handshake traffic keys derived from that shared secret. The server sends its certificate chain, a signature over the transcript hash so far (`CertificateVerify`), and a `Finished` MAC.

Flight 3 — client Finished. The client verifies the certificate chain against trusted roots, verifies `CertificateVerify`, recomputes the transcript hash, checks the server's `Finished`, then sends its own `Finished` MAC (and, if requested, its own certificate and `CertificateVerify` for mutual TLS). After this, both sides derive application traffic keys and the session proceeds symmetrically encrypted. ```

# TLS 1.3 handshake, client and server roles, 1-RTT case function CLIENT_HELLO(): client_random := random(32 bytes) (x, X) := DH_KEYGEN(group) # x = private, X = public share send ClientHello(client_random, offered_groups, key_share = X, cipher_suites, sig_algs) transcript.append(ClientHello) function SERVER_RESPOND(ClientHello): server_random := random(32 bytes) group := choose_group(ClientHello.offered_groups) (y, Y) := DH_KEYGEN(group) send ServerHello(server_random, key_share = Y, chosen_group = group) transcript.append(ServerHello) shared_secret := DH(y, ClientHello.key_share.X) # = DH(x, Y) on client side

# --- key schedule: derive handshake traffic keys --- early_secret := HKDF_Extract(salt = 0, ikm = 0) handshake_secret:= HKDF_Extract(salt = Derive(early_secret), ikm = shared_secret) server_hs_traffic_secret := HKDF_Expand_Label(handshake_secret, "s hs traffic", Hash(transcript)) client_hs_traffic_secret := HKDF_Expand_Label(handshake_secret, "c hs traffic", Hash(transcript))

# remainder of server flight is encrypted under server_hs_traffic_secret send EncryptedExtensions() send Certificate(server_cert_chain) transcript.append(EncryptedExtensions, Certificate) signature := SIGN(server_private_key, Hash(transcript)) send CertificateVerify(signature) transcript.append(CertificateVerify) finished_key := HKDF_Expand_Label(server_hs_traffic_secret, "finished", "") verify_data := HMAC(finished_key, Hash(transcript)) send Finished(verify_data) transcript.append(Finished) return handshake_secret, server_hs_traffic_secret, client_hs_traffic_secret function CLIENT_FINISH(ServerHello, EncryptedExtensions, Certificate, CertificateVerify, ServerFinished): transcript.append(ServerHello) shared_secret := DH(x, ServerHello.key_share.Y)

# ... derive handshake_secret, server_hs_traffic_secret, client_hs_traffic_secret

# exactly as above, using the same transcript hash so far ... decrypt EncryptedExtensions, Certificate using server_hs_traffic_secret transcript.append(EncryptedExtensions, Certificate) assert VALIDATE_CHAIN(Certificate, trusted_roots) # PKI check assert VERIFY(Certificate.pubkey, CertificateVerify.signature, Hash(transcript)) # binds signature to transcript transcript.append(CertificateVerify) finished_key := HKDF_Expand_Label(server_hs_traffic_secret, "finished", "") assert HMAC(finished_key, Hash(transcript)) == ServerFinished.verify_data transcript.append(ServerFinished) client_finished_key := HKDF_Expand_Label(client_hs_traffic_secret, "finished", "") my_verify_data := HMAC(client_finished_key, Hash(transcript)) send Finished(my_verify_data) # encrypted under client_hs_traffic_secret

# --- derive application traffic keys from handshake_secret and full transcript --- master_secret := HKDF_Extract(salt = Derive(handshake_secret), ikm = 0) client_app_traffic_secret := HKDF_Expand_Label(master_secret, "c ap traffic", Hash(transcript)) server_app_traffic_secret := HKDF_Expand_Label(master_secret, "s ap traffic", Hash(transcript)) return client_app_traffic_secret, server_app_traffic_secret ```

Round trips and message cost. The 1-RTT handshake requires exactly one round trip when the client's guessed DH group matches one the server supports: ClientHello travels once, the entire server flight travels once, and the client's Finished (plus any application data it wants to send immediately) piggybacks on the third flight. If the client's guess is wrong, the server sends a `HelloRetryRequest` naming its preferred group, costing one extra round trip — this is the worst case, 2-RTT. Session resumption via a pre-shared key (PSK) from a previous handshake supports 0-RTT: the client can send application data in its very first flight, encrypted under a key derived from the earlier session's resumption secret, at the cost of losing replay protection for that early data.

Computational cost. Each side performs one DH key generation and one DH computation — for X25519 this is a small constant number of finite-field multiplications, effectively $O(1)$ per handshake in the security parameter (dominant cost is a small fixed number of scalar multiplications on a fixed-size curve). The server additionally performs one signature generation and the client one signature verification, each $O(1)$ in the same sense (cost scales with key size, not input length). The transcript hash is computed incrementally, so its cost is $O(n)$ in total transcript length $n$ across the whole handshake, not per message — each new message is hashed once as it is appended. Certificate chain validation costs $O(d)$ signature verifications where $d$ is chain depth, typically 2–3. Overall the handshake's asymptotic cost is dominated by a constant number of public-key operations plus linear-in-transcript-size hashing, both negligible in practice (sub-millisecond on modern hardware for the crypto; the round trip's network latency dominates wall-clock time).

Correctness argument. The central invariant is: > After the client verifies `CertificateVerify` and both `Finished` messages, it holds that (a) the DH shared secret was computed from the same $(X, Y)$ pair the server used, (b) the server possesses the private key certified by a chain rooted in a trust anchor, and (c) neither the DH shares nor any other handshake field was altered in transit. The argument runs in three steps. First, DH correctness guarantees that if neither share is tampered with, $g^{xy} = g^{yx}$ is the same value on both sides (standard Diffie-Hellman Key Exchange correctness). Second, `CertificateVerify` is a signature over $H(\text{transcript})$, where the transcript includes both DH shares; by the unforgeability of the signature scheme, an adversary who alters either share changes the transcript hash, and cannot produce a valid signature over the new hash without the server's private key — so if verification succeeds, the transcript the client saw is the transcript the server actually sent. Third, `Finished` is a MAC keyed by a secret derived (via HKDF) from the DH shared secret and the transcript-so-far; matching `Finished` values on both ends is only possible if both parties derived the same handshake traffic secret, which by the extractor/PRF security of HKDF only happens if both computed the same DH shared secret from the same transcript. Chaining these three facts: a successful handshake implies an authenticated, tamper-free, freshly-keyed session, which is precisely the security goal (this is the informal shape of the proof used in the formal analyses of TLS 1.3, e.g. via the multi-stage key exchange security model).

Tradeoffs. Full forward secrecy is bought by discarding ephemeral DH private keys immediately after use — TLS 1.3 removed static RSA key transport entirely so this cannot be skipped. 0-RTT data trades replay resistance for latency: an attacker who captures a 0-RTT ClientHello can resend it, and the server may process the attached early data twice unless the application layer enforces idempotency. Signature verification cost and certificate chain size matter at scale (a busy server terminating thousands of handshakes per second must plan CPU budget for RSA/ECDSA operations), which is one reason session resumption and hardware acceleration are standard in production deployments.

Trace a concrete handshake between a browser (client) and `example.com` (server) using X25519 for key exchange and ECDSA-P256 for the server signature.

Now suppose an active attacker, Mallory, intercepts step 2 and substitutes her own $X' = x'G$, planning to relay a modified ServerHello back with her own $Y'$ to the client, hoping to establish two independent sessions and proxy traffic (a classic MITM). At step 9, the server signs a transcript that includes $X'$ (Mallory's substituted value) and $Y$ (server's real value) — a transcript Mallory constructed. Mallory can indeed get a validly-signed transcript for her own DH share, because the server has no way to know $X'$ isn't the client's real value — but the signature is over the leaf certificate's key, which is bound to the name `example.com` by the CA. Mallory does not possess `example.com`'s private key, so she cannot produce a different valid `CertificateVerify` for a session where she also relays back to the real client under her own identity; her only path is to relay the server's genuine signed flight unmodified. If she does that but tries to swap in her own DH share toward the client, the transcript hash the client computes (which includes Mallory's $X'$, since the client itself sent $X$, not $X'$ — contradiction: the client's own transcript includes what it sent) will not match what the server signed unless Mallory relays byte-for-byte, in which case she never learns $Z$ at all. This is exactly the argument from the correctness section: tampering with a DH share is detected because the signature commits to the whole transcript, and only the certified server can produce a valid signature over any transcript, tampered or not.

The TLS handshake sits at the confluence of contributions from Diffie–Hellman-style key agreement, RSA-style public-key signatures, and the SSL lineage from Netscape's Taher Elgamal, but its most consequential recent architect is the cryptographer who led the near-total redesign that became TLS 1.3.

The path to TLS 1.3 was shaped as much by attacks as by design ambition. The BEAST and Lucky Thirteen attacks exploited CBC-mode padding and IV predictability, motivating a move to AEAD-only ciphersuites. The Logjam and FREAK attacks showed that "export-grade" weakened DH and RSA parameters, kept for historical compliance reasons, could be forced upon unsuspecting clients — a lesson embedded directly into TLS 1.3's decision to drop all non-forward-secret and export ciphersuites outright. The 2009 renegotiation vulnerability, which allowed an attacker to inject data into an authenticated session by exploiting insecure session renegotiation, drove the introduction of the transcript-binding techniques that now tie every handshake message into every derived key. Each of these episodes reinforced the same lesson: any handshake feature that is not cryptographically bound into a single verified transcript is a seam an adversary will eventually find.

The TLS handshake underlies HTTPS, meaning it protects essentially all confidential web traffic: banking sessions, login credentials, health records, and API calls between cloud services. It secures the QUIC transport used by HTTP/3, where the TLS 1.3 handshake is embedded directly into the transport-layer setup. Certificate authorities, browser trust stores, and the entire Public-Key Infrastructure and Certificates ecosystem exist specifically to make the server-authentication half of this handshake meaningful — without a trustworthy chain of certificates, the signature step proves nothing about identity. Mutual TLS (mTLS), where both client and server present certificates, is the backbone of zero-trust microservice architectures inside modern data centers. Hardware security modules and cloud key-management services exist largely to protect the long-term private keys whose signatures anchor every handshake — a concern that connects directly to the broader discipline of Key Management, including key rotation, revocation, and hardware-backed key storage that this handshake ultimately depends on.

Ask the tutor: How does the TLS key schedule's HKDF chain prevent a compromised resumption key from revealing the original handshake's traffic keys?