Email Protocols: SMTP, IMAP, POP3

Electronic mail is built on a store-and-forward architecture: no direct connection is required between sender and recipient at the moment of sending. Instead, a message is handed to a Mail Transfer Agent (MTA) — a server whose job is to accept mail and relay it toward its destination, one hop at a time, buffering it on disk if the next hop is unreachable. The protocol that MTAs speak to each other, and that a mail client speaks to its own outgoing server, is the Simple Mail Transfer Protocol (SMTP), standardized in RFC 5321 (originally RFC 821, 1982). SMTP's entire job is pushing a message closer to its destination; it says nothing about how a human eventually reads that message. Reading mail is the job of a different pair of protocols. The Post Office Protocol (POP3), RFC 1939, treats the mail server as a temporary mailbox: a client connects, downloads the queued messages, and — traditionally — deletes them from the server. The Internet Message Access Protocol (IMAP), RFC 3501 (and its modern revision RFC 9051), treats the server as the authoritative long-term store: messages live in folders on the server, client software merely displays and manipulates them, and multiple devices see a consistently synchronized view. This asymmetry — SMTP always pushes, POP3/IMAP always pull — is why an email path uses at least two, often three, distinct protocol conversations to go from one person's keyboard to another's screen. Threaded through both halves is the message format, defined by RFC 5322 (successor to RFC 822): a mail message is a sequence of `Header: value` lines, a single blank line, and then an arbitrary body. Critically, this same format is used whether the message is in transit or at rest — SMTP relays the exact same header block that IMAP later hands to your mail client, with new `Received:` header lines prepended at every hop. Finally, routing decisions — which server should I even talk to for `alice@example.com`? — are resolved through DNS MX (Mail eXchange) records, which map a domain name to a prioritized list of mail server hostnames, decoupling a domain's public identity from the specific machines that happen to run its mail infrastructure.

Step 1 — Composition and submission. A user's Mail User Agent (MUA — Outlook, Gmail's web client, Thunderbird) constructs a message with RFC 5322 headers (`From`, `To`, `Subject`, `Date`) and hands it to a local or provider's outgoing mail server, historically over port 25, now almost always over port 587 with authentication (mail submission, RFC 6409).

Step 2 — MX lookup. The sending MTA extracts the domain from the recipient address (`bob@example.org` → `example.org`) and queries DNS for that domain's MX records. Each MX record has a preference value; lower values are tried first.

Step 3 — SMTP relay. The sending MTA opens a TCP connection to port 25 on the highest-priority MX host (falling back to lower-priority hosts if it fails) and executes the SMTP command dialogue below.

Step 4 — Store-and-forward hops. If the destination domain's MX host is itself just a relay (common in enterprise setups with a spam-filtering gateway in front of the real mailbox server), the message is forwarded again via SMTP, each hop prepending a `Received:` header recording who received it, from whom, and when.

Step 5 — Final delivery to mailbox. The last-hop MTA delivers the message into the recipient's mailbox store (a maildir, mbox file, or proprietary store).

Step 6 — Retrieval. The recipient's MUA later connects to a different server (or the same physical machine acting in a different role) using POP3 or IMAP to fetch the message.

The SMTP command dialogue

``` S: 220 mail.example.org ESMTP ready C: EHLO client.example.com S: 250-mail.example.org greets client.example.com S: 250-SIZE 35882577 S: 250-STARTTLS S: 250 8BITMIME C: STARTTLS S: 220 Ready to start TLS [ ... TLS handshake ... all further lines encrypted ... ] C: EHLO client.example.com S: 250 mail.example.org greets client.example.com C: MAIL FROM: S: 250 OK C: RCPT TO: S: 250 OK C: DATA S: 354 Start mail input; end with . C: From: alice@example.com C: To: bob@example.org C: Subject: Meeting tomorrow C: C: Hi Bob, are we still on for 10am? C: . S: 250 OK: queued as 4F3A9 C: QUIT S: 221 Bye ``` Notice the strict request-response, line-oriented structure: every client command gets exactly one server reply beginning with a 3-digit status code (2xx success, 3xx intermediate, 4xx transient failure, 5xx permanent failure). `MAIL FROM` and `RCPT TO` establish the envelope sender/recipient — which can differ from the `From:`/`To:` header fields inside `DATA` — a distinction that matters enormously for mailing lists, forwarding, and spam analysis. The message body is terminated by a line containing only a single period.

The MX lookup algorithm

``` function DELIVER(message, recipient_address): domain ← substring_after(recipient_address, "@") mx_records ← DNS_QUERY(domain, type=MX) # e.g. [(10, "mx1.ex.org"), (20, "mx2.ex.org")] sort mx_records by preference ascending if mx_records is empty: mx_records ← [(0, domain)] # RFC 5321 fallback: try the A/AAAA record directly for (pref, host) in mx_records: try: ip ← DNS_QUERY(host, type=A_or_AAAA) connection ← TCP_CONNECT(ip, port=25) if connection succeeds: result ← SMTP_SESSION(connection, message) if result == PERMANENT_FAILURE: return BOUNCE(message, reason=result) # do not retry other MX hosts on 5xx if result == SUCCESS: return DELIVERED

# else TRANSIENT_FAILURE: fall through, try next MX except NetworkError: continue # try next MX host ENQUEUE_FOR_RETRY(message) # exponential backoff, eventually bounce after ~5 days ``` The key correctness point: a 5xx permanent failure (e.g., "no such mailbox") should stop the whole attempt and generate a bounce message back to the sender, while a 4xx transient failure or connection error should trigger retries against remaining MX hosts, and eventually a scheduled retry of the whole message over hours or days.

POP3 session

``` S: +OK POP3 server ready C: USER bob S: +OK C: PASS hunter2 S: +OK bob's maildrop has 2 messages C: LIST S: +OK 2 messages (1200 octets) S: 1 500 S: 2 700 S: . C: RETR 1 S: +OK 500 octets follow S: [ ... full RFC 5322 message ... ] S: . C: DELE 1 S: +OK message 1 deleted C: QUIT S: +OK POP3 server signing off (deletes marked messages now) ``` POP3's state machine has exactly three phases: AUTHORIZATION (USER/PASS), TRANSACTION (LIST/RETR/DELE), and UPDATE (deletions actually committed only on QUIT). This is deliberately minimal — POP3 has no concept of folders, flags, or partial fetches in its base spec.

IMAP session

``` S: OK IMAP4rev1 Service Ready C: a1 LOGIN bob hunter2 S: a1 OK LOGIN completed C: a2 SELECT INBOX S: 172 EXISTS S: 1 RECENT S: OK [UIDVALIDITY 3857529045] S: a2 OK [READ-WRITE] SELECT completed C: a3 FETCH 172 (BODY[HEADER]) S: 172 FETCH (BODY[HEADER] {342} S: [ ... header block ... ] S: ) S: a3 OK FETCH completed C: a4 STORE 172 +FLAGS (\Seen) S: 172 FETCH (FLAGS (\Seen)) S: a4 OK STORE completed C: a5 LOGOUT S: * BYE IMAP4rev1 server terminating S: a5 OK LOGOUT completed ``` Every IMAP command is prefixed by a client-chosen tag (`a1`, `a2`, …), and the matching response repeats that tag — this allows pipelining multiple outstanding commands and unambiguously matching replies, unlike POP3's strictly synchronous one-command-one-reply model. IMAP exposes mailbox state (message counts, UID validity, flags like `\Seen`, `\Deleted`, `\Flagged`) so that several clients can observe a mutually consistent view of the same mailbox.

Message-relay complexity. For a message with $n$ intermediate MTAs on its path (typically $n \in \{1,2,3\}$: submission → possible spam gateway → final mailbox server), the total work is $O(n)$ SMTP sessions, each $O(1)$ round trips for the envelope commands (`EHLO`, `MAIL FROM`, `RCPT TO`, `DATA`) plus $O(m)$ time to transfer a message of $m$ bytes. Each hop appends one `Received:` header, so header size grows $O(n)$ — in practice bounded by a small constant, and RFC 5321 recommends a maximum hop count (typically 100) specifically to detect routing loops and prevent unbounded growth.

MX resolution correctness. The MX algorithm's correctness rests on an invariant: at every retry, the sender is contacting a server that has declared itself, via DNS, willing to accept mail for this domain, in priority order. Because MX preference values impose a total (well, at least partial with ties broken arbitrarily) order, the algorithm is guaranteed to attempt the "best" available host first, and — because it does not retry additional hosts after a 5xx — it avoids delivering the same message to two independent final mailboxes when the first host has authoritatively rejected it (e.g., "no such user"). The RFC 5321 fallback (try the domain's own A/AAAA record if no MX record exists) preserves backward compatibility with domains predating MX records, and is itself a correctness patch layered onto the earlier, simpler design.

Delivery is best-effort, not guaranteed. SMTP offers at-least-once, not exactly-once delivery semantics in the presence of network partitions: if a connection drops after `DATA` completes on the server side but before the client receives the final `250 OK`, the sending MTA cannot distinguish "message was accepted" from "message was lost," and RFC 5321-compliant behavior is to retry — which can occasionally cause duplicate delivery. This is a deliberate tradeoff: SMTP favors availability and eventual delivery over strict exactly-once guarantees, pushing duplicate-detection (if wanted) up to the Message-ID header and client-side deduplication.

POP3 vs IMAP state complexity. POP3's state machine is $O(1)$ in the number of concepts it must track per session — messages are numbered 1..N for the session only, and the protocol has no persistent cross-session state beyond "deleted or not." This makes POP3 servers trivially simple to implement and verify but pushes all organizational complexity (folders, read/unread tracking across devices) to the client, and to disk. IMAP's server-side state (per-mailbox UID validity counters, per-message flags, hierarchical folder trees) is $O(F \cdot M)$ in space for $F$ folders and $M$ messages per folder, but this cost buys multi-client consistency: two IMAP clients reading the same mailbox concurrently see the same flags and folder structure, because the server — not each client — is the single source of truth. This is the classic centralized-state-vs-thin-protocol tradeoff seen throughout distributed systems.

Security is bolted on, not built in. Both SMTP and POP3/IMAP were designed as plaintext protocols with optional authentication; nothing in the base specification prevents a network eavesdropper from reading `USER`/`PASS` credentials or message contents. The STARTTLS mechanism (upgrade a plaintext connection to TLS mid-session) and the implicit-TLS ports (465 for SMTPS, 993 for IMAPS, 995 for POP3S) are later additions that wrap the existing command grammar inside an encrypted channel without changing the protocol semantics — a pattern examined further under TLS and Secure Channels. Because STARTTLS negotiation itself happens before encryption starts, a network attacker performing a "STARTTLS stripping" attack can, if the client is not strict, suppress the upgrade and force a plaintext session — a real, documented class of vulnerability, not a theoretical one.

Trace the message `alice@cs.example.edu` → `bob@corp.example.org`, where `example.org` has two MX records (a primary gateway and a backup) and Bob reads mail with IMAP on two devices.

Step 1 — DNS MX lookup performed by `cs.example.edu`'s outgoing MTA:

Step 2 — SMTP relay to `mail1.example.org` (preference 10, tried first):

Suppose `mail1.example.org` is a spam-filtering gateway, not the final mailbox server. It performs a second SMTP hop internally to `mailstore.corp.internal`, prepending: ``` Received: from mail1.example.org (mail1.example.org [203.0.113.5]) by mailstore.corp.internal with ESMTP id B3C1; Tue, 11 Mar 2025 14:02:11 +0000 Received: from cs.example.edu (cs.example.edu [198.51.100.9]) by mail1.example.org with ESMTPS id A19F2; Tue, 11 Mar 2025 14:02:07 +0000 From: alice@cs.example.edu To: bob@corp.example.org Subject: Meeting tomorrow Date: Tue, 11 Mar 2025 14:01:55 +0000 Hi Bob, are we still on for 10am? ``` Reading `Received:` headers bottom to top reconstructs the actual path: originated at `cs.example.edu`, relayed by `mail1.example.org` at 14:02:07, finally delivered into `mailstore.corp.internal` at 14:02:11. This chain is what forensic email-tracing and anti-spoofing tools rely on, since the `From:` header is client-supplied and unverified, but each `Received:` line was stamped by a server that actually observed the connection.

Step 3 — Bob reads mail via IMAP on his laptop and phone.

Because IMAP state lives on the server, both devices converge to the same view — Bob never sees the message unread on his phone after reading it on his laptop. Had Bob instead used POP3 with default settings, the laptop's `RETR` + `DELE` + `QUIT` sequence would have removed the message from the server entirely, and the phone would never see it — a common real-world source of "where did my email go?" confusion.

Email's architecture was shaped by researchers solving a very concrete 1970s–1980s problem: how do you reliably move a text message across a network of loosely connected, independently administered machines, none of which is guaranteed to be reachable at any given instant? The store-and-forward answer they converged on — proven first on ARPANET file-transfer mechanisms, then formalized as SMTP — still underlies every mail server today.

Every major consumer webmail provider (Gmail, Outlook.com, Yahoo Mail) runs SMTP servers for outbound relay and offers IMAP for retrieval — Gmail deprecated meaningful POP3 emphasis specifically because IMAP's multi-device consistency matches how people actually use phones, tablets, and laptops together. Enterprise mail gateways (spam and malware filtering appliances) sit as intermediate SMTP hops exactly as sketched above, inspecting and sometimes rewriting messages before final delivery. Mailing-list software, transactional email services (password resets, receipts), and marketing platforms all speak SMTP as their outbound interface, often through submission APIs layered atop the same RFC 5321 command set. Anti-spoofing infrastructure — SPF, DKIM, and DMARC — exists precisely because the `MAIL FROM` envelope and `From:` header are independently forgeable, a direct consequence of SMTP's original trust-everyone design. Mobile push-mail protocols (Exchange ActiveSync, and IMAP IDLE extensions) build on IMAP's server-state model to notify devices of new mail without constant polling. And the historical absence of built-in encryption in SMTP, POP3, and IMAP is the direct motivating case study for why TLS and Secure Channels had to be retrofitted onto so many pre-Web Internet protocols rather than designed in from the start.

Ask the tutor: How do SPF, DKIM, and DMARC actually verify a sender's identity on top of plain SMTP?