Modular Arithmetic and Primes for Cryptography

Imagine two people, Alice and Bob, who have never met and share no secret. Alice wants to send Bob a message that no eavesdropper can read. Yet Bob has published nothing except a few numbers on a webpage. This is the founding puzzle of public-key cryptography. Its entire machinery rests on a surprisingly small toolbox: clocks that wrap around, remainders, and prime numbers. Modular arithmetic turns the infinite integers into a finite, well-behaved universe. In this universe, addition, multiplication, and even division-by-inverse make sense. Primes supply the raw material that makes some computations easy in one direction. In the other direction, they make computations brutally hard. This asymmetry is not a curiosity — it is the load-bearing wall of modern cryptography. Multiplying two 1024-bit primes takes a computer a fraction of a second. Recovering those primes from their product is believed to take much longer. With known classical algorithms, it would take longer than the age of the universe. Systems like RSA, Diffie–Hellman key exchange, and elliptic-curve cryptography all live inside modular arithmetic. All of them depend on number-theoretic facts to function. These facts include Fermat's Little Theorem, Euler's Theorem, and the extended Euclidean algorithm. They let us actually compute keys, encrypt, decrypt, and verify signatures. This article builds that toolbox from scratch. It connects every piece to its cryptographic job. We cover congruences and modular arithmetic as an algebraic structure. We also cover the Euclidean and extended Euclidean algorithms. These compute greatest common divisors and modular inverses. We explore the structure of primes and primality testing. Finally, we examine two centerpiece theorems: Fermat's Little Theorem and Euler's Theorem. These let us compute modular exponents and inverses efficiently. By the end, you will be able to compute modular inverses by hand and in code. You will be able to explain why Euler's Theorem generalizes Fermat's Little Theorem. You will also see precisely where these ideas plug in. They connect to The RSA Cryptosystem and The Discrete Logarithm Problem. These are the two central pillars built on top of this arithmetic.

Key takeaways
  • Modular arithmetic replaces the integers with a finite "clock" of size nn, where numbers wrap around after reaching nn.
  • The Euclidean algorithm computes gcd⁡(a,b)\gcd(a,b) in O(log⁡(min⁡(a,b)))O(\log(\min(a,b))) steps; its extended version also finds modular inverses.
  • An inverse of aa modulo nn exists exactly when gcd⁡(a,n)=1\gcd(a,n)=1; this single fact underlies encryption/decryption key pairs.
  • Fermat's Little Theorem (ap−1≡1(modp)a^{p-1}\equiv 1 \pmod p for prime pp) and its generalization, Euler's Theorem (aϕ(n)≡1(modn)a^{\phi(n)}\equiv 1 \pmod n), power fast exponentiation and RSA correctness.
  • Fast modular exponentiation via repeated squaring computes ab  na^b \bmod n in O(log⁡b)O(\log b) multiplications instead of O(b)O(b).
  • Primality testing (trial division, Fermat test, Miller–Rabin) is what lets us actually manufacture the large primes cryptography needs.
The core insight

Modular arithmetic gives a finite algebraic playground where multiplication is easy to do but, when built from large primes, hard to undo — and that gap is the engine of public-key cryptography.

The idea

Two integers aa and bb are congruent modulo nn, written a≡b(modn)a \equiv b \pmod n, if nn divides a−ba - b. Equivalently, aa and bb leave the same remainder when divided by nn. This relation partitions the integers into nn residue classes, usually represented by {0,1,…,n−1}\{0, 1, \dots, n-1\}, and the set of these classes with addition and multiplication defined "mod nn" is denoted Zn\mathbb{Z}_n. Congruence behaves like equality for the purposes of addition, subtraction, and multiplication: if a≡b(modn)a \equiv b \pmod n and c≡d(modn)c \equiv d \pmod n, then a+c≡b+da+c \equiv b+d, and ac≡bd(modn)ac \equiv bd \pmod n. This is why we can reduce intermediate results modulo nn at any point in a computation without changing the final answer — the key property that makes modular arithmetic tractable on computers, where numbers never grow unboundedly large. Division is subtler. In ordinary arithmetic, dividing by aa means multiplying by 1/a1/a. In Zn\mathbb{Z}_n, we instead seek a modular inverse: an integer a−1a^{-1} such that a⋅a−1≡1(modn)a \cdot a^{-1} \equiv 1 \pmod n. Such an inverse exists if and only if gcd⁡(a,n)=1\gcd(a, n) = 1, i.e., aa and nn are coprime. This is not a technicality — it is the crux of why cryptographic moduli are built from primes or products of primes: they guarantee that almost every element has an inverse, or that we can precisely control which elements do. Two theorems anchor everything that follows. Fermat's Little Theorem says that if pp is prime and gcd⁡(a,p)=1\gcd(a,p)=1, then ap−1≡1(modp)a^{p-1} \equiv 1 \pmod p. Euler's Theorem generalizes this to any modulus nn: if gcd⁡(a,n)=1\gcd(a,n)=1, then aϕ(n)≡1(modn)a^{\phi(n)} \equiv 1 \pmod n, where ϕ(n)\phi(n) is Euler's totient function, counting integers in {1,…,n}\{1,\dots,n\} coprime to nn. These theorems tell us the order of the multiplicative structure of Zn\mathbb{Z}_n and let us collapse enormous exponents down to something manageable — precisely the trick RSA uses to guarantee that encrypting then decrypting returns the original message. Together with the extended Euclidean algorithm for computing inverses, and primality tests for manufacturing large primes, these ideas form the complete algebraic substrate of The RSA Cryptosystem and set the stage for the multiplicative-group viewpoint needed in The Discrete Logarithm Problem.

How it works

The Euclidean algorithm (GCD)

The Euclidean algorithm computes gcd⁡(a,b)\gcd(a,b) using the identity gcd⁡(a,b)=gcd⁡(b,a  b)\gcd(a,b) = \gcd(b, a \bmod b), repeatedly reducing the problem until the second argument hits zero. ``` function GCD(a, b): while b != 0: (a, b) = (b, a mod b) return a ```

The extended Euclidean algorithm (modular inverse)

The extended version tracks coefficients so that at every step we maintain integers x,yx, y with ax+by=gcd⁡(a,b)a x + b y = \gcd(a,b) (Bézout's identity). When gcd⁡(a,n)=1\gcd(a,n) = 1, the coefficient of aa gives exactly the modular inverse of a(modn)a \pmod n. ``` function ExtendedGCD(a, b): // returns (g, x, y) such that ax + by = g = gcd(a, b) (old_r, r) = (a, b) (old_s, s) = (1, 0) (old_t, t) = (0, 1) while r != 0: q = old_r div r (old_r, r) = (r, old_r - q r) (old_s, s) = (s, old_s - q s) (old_t, t) = (t, old_t - q * t) return (old_r, old_s, old_t) // g, x, y function ModInverse(a, n): (g, x, _) = ExtendedGCD(a, n) if g != 1: error "no inverse exists" return x mod n ```

Fast modular exponentiation

Cryptographic exponents are hundreds of digits long, so computing aba^b directly is impossible — the result would have more digits than atoms in the observable universe. Repeated squaring exploits the binary representation of the exponent, reducing bb multiplications to O(log⁡b)O(\log b). ``` function ModExp(a, b, n): result = 1 a = a mod n while b > 0: if b is odd: result = (result a) mod n a = (a a) mod n b = b div 2 return result ```

Primality testing

To build a cryptosystem you must manufacture large primes, not just reason about them abstractly. The workhorse is a probabilistic test built on Fermat's theorem. ``` function FermatTest(p, k): // probabilistic: k independent witnesses if p < 2: return false for i = 1 to k: a = random integer in [2, p-2] if ModExp(a, p-1, p) != 1: return "composite" return "probably prime" ``` The Fermat test has rare false positives (Carmichael numbers), so real systems use the stronger Miller–Rabin test, which additionally checks nontrivial square roots of unity along the way, driving the error probability down to at most 4−k4^{-k} for kk rounds.

Figure
From integers to a cryptographic modulus: the pipeline of ideas
Congruences a ≡ b (mod n) Euclidean Alg. gcd(a,n) = 1 ? Extended Euclid finds a⁻¹ mod n Primes p, q Miller–Rabin test Fermat / Euler Thm. a^φ(n) ≡ 1 (mod n) Modular Exponent. repeated squaring O(log b) RSA / Diffie–Hellman / ECC encrypt, decrypt, sign, exchange keys

Complexity & correctness

Euclidean algorithm — complexity. Each step replaces (a,b)(a,b) with (b,a  b)(b, a \bmod b), and a classical fact (related to Fibonacci numbers) shows that a  b<a/2a \bmod b < a/2 within two steps in the worst case. More precisely, the number of division steps is O(log⁡(min⁡(a,b)))O(\log(\min(a,b))); the worst case is achieved by consecutive Fibonacci numbers, giving the Euclidean algorithm its famous logarithmic bound (this is Lamé's theorem). Each step performs one integer division, so for kk-bit numbers, the total bit complexity is O(k2)O(k^2) using schoolbook division, or better with fast multiplication. Correctness invariant: at every iteration, gcd⁡(a,b)\gcd(a,b) is unchanged, because any common divisor of aa and bb also divides a  b=a−qba \bmod b = a - qb, and conversely. The loop terminates because bb strictly decreases and is a non-negative integer, so it must eventually hit 00, at which point aa is the answer.

Extended Euclidean algorithm. Same O(log⁡(min⁡(a,b)))O(\log(\min(a,b))) iteration count; it additionally maintains the invariant a⋅si+n⋅ti=ria \cdot s_i + n \cdot t_i = r_i at every step, which is preserved by construction of the update rule (it mirrors the same linear combination the quotient step performs on the remainders). When rir_i reaches gcd⁡(a,n)\gcd(a,n), the corresponding sis_i satisfies a⋅si≡gcd⁡(a,n)(modn)a \cdot s_i \equiv \gcd(a,n) \pmod n; if gcd⁡(a,n)=1\gcd(a,n)=1 this is the inverse.

Existence of inverses. By Bézout's identity, gcd⁡(a,n)=1\gcd(a,n)=1 implies integers x,yx,y exist with ax+ny=1ax + ny = 1, so ax≡1(modn)ax \equiv 1 \pmod n. Conversely, if d=gcd⁡(a,n)>1d = \gcd(a,n) > 1, then d∣axd \mid ax and d∣nd \mid n for any xx, so ax  nax \bmod n can never equal 11. This "iff" is exact, not a heuristic — it is why RSA key generation insists the public exponent ee be coprime to ϕ(n)\phi(n).

Modular exponentiation. Repeated squaring uses O(log⁡b)O(\log b) multiplications, each of numbers with at most log⁡n\log n bits, giving overall O((log⁡b)(log⁡n)2)O((\log b)(\log n)^2) bit operations with schoolbook multiplication. Naive repeated multiplication would take O(b)O(b) multiplications — exponential in the bit-length of bb — so this is the difference between a computation finishing instantly and one that never finishes. Correctness: writing bb in binary as b=∑bi2ib = \sum b_i 2^i, we have ab=∏ia2i⋅bia^b = \prod_i a^{2^i \cdot b_i}. The loop maintains the invariant that `a` holds a02i  na_0^{2^i} \bmod n at the start of iteration ii (where a0a_0 is the original base) and `result` holds the product of all such terms for bits already processed — squaring `a` each round advances the exponent's power of two, and multiplying into `result` exactly when the bit is 11 implements the product formula.

Fermat's Little Theorem — proof sketch. For prime pp and aa not divisible by pp, the map x↦ax  px \mapsto ax \bmod p permutes the nonzero residues {1,…,p−1}\{1, \dots, p-1\} (it is injective because pp is prime and has no inverse issues, hence bijective on a finite set). Multiplying all of them together, ∏x=1p−1(ax  p)=∏x=1p−1x\prod_{x=1}^{p-1} (ax \bmod p) = \prod_{x=1}^{p-1} x, and factoring aa out of each of the p−1p-1 terms gives ap−1∏x≡∏x(modp)a^{p-1} \prod x \equiv \prod x \pmod p. Since ∏x\prod x is coprime to pp, it cancels, leaving ap−1≡1(modp)a^{p-1} \equiv 1 \pmod p.

Euler's Theorem is the same argument run over the group of units modulo nn (residues coprime to nn, of which there are ϕ(n)\phi(n)) instead of all nonzero residues modulo a prime; it reduces to Fermat's theorem exactly when n=pn = p is prime, since then ϕ(p)=p−1\phi(p) = p - 1.

Primality testing — honest tradeoffs. Trial division is O(p)O(\sqrt{p}) and exact but useless for 1024-bit primes (p\sqrt p is astronomically large). The Fermat test is fast (O(log⁡p)O(\log p) modular exponentiations) but can be fooled by Carmichael numbers, which satisfy Fermat's congruence for every base despite being composite. Miller–Rabin fixes this by also checking for nontrivial square roots of 11 modulo pp (a prime modulus has only ±1\pm 1 as square roots of 11); it errs with probability at most 4−k4^{-k} over kk random rounds, and no known efficient deterministic test beats this in practice for cryptographic sizes — hence real systems accept a tunably tiny, non-zero error probability.

Worked example

Let's actually build a tiny (insecure, illustrative) RSA-style key pair and trace every tool above on real numbers.

Step 1 — pick primes. Let p=61p = 61, q=53q = 53. (Trial division instantly confirms both are prime; in practice Miller–Rabin would certify 300-digit primes this size in milliseconds.)

Step 2 — modulus and totient.

n=pq=61×53=3233n = pq = 61 \times 53 = 3233
ϕ(n)=(p−1)(q−1)=60×52=3120\phi(n) = (p-1)(q-1) = 60 \times 52 = 3120

Step 3 — choose public exponent ee, coprime to ϕ(n)\phi(n). Try e=17e = 17. Run the Euclidean algorithm on gcd⁡(17,3120)\gcd(17, 3120):

aabba  ba \bmod b
3120173120 mod 17 = 3
17317 mod 3 = 2
323 mod 2 = 1
212 mod 1 = 0

bb reaches 00 with a=1a = 1, so gcd⁡(17,3120)=1\gcd(17, 3120) = 1 — a valid exponent.

Step 4 — compute the private exponent d=e−1  ϕ(n)d = e^{-1} \bmod \phi(n) via extended Euclid. Tracking Bézout coefficients through the same division chain (3120 = 183·17 + 9, 17 = 1·9 + 8, 9 = 1·8 + 1, 8 = 8·1 + 0) and back-substituting:

1=9−1⋅8=9−1⋅(17−1⋅9)=2⋅9−1⋅17=2(3120−183⋅17)−1⋅17=2⋅3120−367⋅171 = 9 - 1\cdot 8 = 9 - 1\cdot(17-1\cdot9) = 2\cdot 9 - 1\cdot 17 = 2(3120 - 183\cdot17) - 1\cdot 17 = 2\cdot 3120 - 367\cdot 17

So −367⋅17≡1(mod3120)-367 \cdot 17 \equiv 1 \pmod{3120}, giving d≡−367≡2753(mod3120)d \equiv -367 \equiv 2753 \pmod{3120}. Check: 17×2753=46801=15×3120+117 \times 2753 = 46801 = 15 \times 3120 + 1. ✓ Confirmed: d=2753d = 2753.

Step 5 — encrypt a message m=65m = 65 (e.g. the ASCII code for 'A') using c=me  nc = m^e \bmod n, via repeated squaring with e=17=100012e=17=10001_2:

bit (MSB→LSB)`result` update`a = a² mod n`
1result = 65a = 65² mod 3233 = 2790
0(no update)a = 2790² mod 3233 = 2299
0(no update)a = 2299² mod 3233 = 2714
0(no update)a = 2714² mod 3233 = 2276
1result = 65·2276 mod 3233 = 2790—

So c=6517  3233=2790c = 65^{17} \bmod 3233 = 2790.

Step 6 — decrypt using m=cd  n=27902753  3233m = c^d \bmod n = 2790^{2753} \bmod 3233. Repeated squaring (2753 has 12 bits) runs in O(log⁡d)≈12O(\log d) \approx 12 squarings and a handful of multiplications, and returns m=65m = 65 — recovering the original message exactly, as guaranteed by Euler's Theorem since ed≡1(modϕ(n))ed \equiv 1 \pmod{\phi(n)} forces med≡m(modn)m^{ed} \equiv m \pmod n. This tiny trace is RSA in miniature: everything from here to The RSA Cryptosystem is scaling p,qp,q up to hundreds of digits and hardening the primality test.

Common pitfalls & traps

PitfallWhy it happensFix
Forgetting to reduce mod nn at every multiplicationIntermediate values explode to huge integers, killing performanceReduce after every multiply, as in `ModExp`
Using `a mod b` in a language where `%` returns negative resultsC/C++/Java `%` can return negative remainders for negative operandsAdd `n` and take `% n` again to force a non-negative residue
Confusing gcd⁡(a,n)=1\gcd(a,n)=1 with a<na < nCoprimality, not size, determines invertibilityAlways compute the actual gcd before assuming an inverse exists
Picking a public exponent ee that shares a factor with ϕ(n)\phi(n)Breaks the RSA key-generation precondition silentlyExplicitly check gcd⁡(e,ϕ(n))=1\gcd(e,\phi(n))=1 before proceeding
Trusting a single Fermat-test roundCarmichael numbers pass Fermat's test for every baseUse Miller–Rabin with many independent random witnesses
Off-by-one in the totient for prime powersϕ(pk)=pk−pk−1\phi(p^k) = p^k - p^{k-1}, not pk−1p^k - 1Use the correct formula per prime-power factor

Pioneers & history

Origins · 1 minPierre de Fermat and how this was discovered
PD
Pierre de Fermat
1607–1665 · French mathematician and magistrate who stated the theorem bearing his name in a 1640 letter, without a rigorous published proof, decades before number theory became a formal discipline

LE
Leonhard Euler
1707–1783 · Swiss mathematician who generalized Fermat's Little Theorem to arbitrary moduli, introducing the totient function that now bears his name and remains central to RSA key generation

History
300 BCE
— Euclid's Elements describes the algorithm for computing greatest common divisors.
1640
— Fermat states his "Little Theorem" in a letter, foundational to later primality tests.
1760
s — Euler proves and generalizes Fermat's theorem, defining the totient function ϕ(n)\phi(n).
1976
— Diffie and Hellman propose public-key cryptography, built directly on modular exponentiation.
1977
— Rivest, Shamir, and Adleman publish RSA, operationalizing Euler's Theorem as an encryption scheme.

Where this matters

Every TLS handshake that secures an HTTPS connection performs modular exponentiation, whether through RSA key transport or Diffie–Hellman key exchange, both living entirely inside the arithmetic developed here. Digital signature schemes — RSA-PSS, DSA, and elliptic-curve variants — use modular inverses to compute signatures and verify them, and the generation of the underlying keys depends on manufacturing large primes with tests like Miller–Rabin. Cryptographic libraries such as OpenSSL, libsodium, and hardware security modules implement fast modular exponentiation as a performance-critical primitive, often in constant time to defend against timing side-channels that could otherwise leak secret exponents bit by bit. Beyond classical public-key systems, the same modular structures reappear in the group-theoretic setting of The Discrete Logarithm Problem, which underlies Diffie–Hellman and elliptic-curve cryptography, and understanding Modular Exponentiation here is a direct prerequisite to reasoning about The RSA Cryptosystem's correctness and security proofs.

Ask the tutor: Can you walk through why choosing ee coprime to ϕ(n)\phi(n) is both necessary and sufficient for RSA to always correctly decrypt every possible message?

What exactly is a residue class, and why do we care?

A residue class modulo nn is the set of all integers that leave the same remainder when divided by nn — for example, modulo 5, the class of 2 contains …,−8,−3,2,7,12,…\dots, -8, -3, 2, 7, 12, \dots. We care because arithmetic on residue classes is well-defined: adding or multiplying any representatives gives a result in a predictable class, regardless of which representative you picked. This lets computers work with a fixed-size set of nn values (00 through n−1n-1) instead of unbounded integers, which is essential for building fast, fixed-precision cryptographic arithmetic that never overflows in an unpredictable way.

Why does RSA require the public exponent ee to be coprime to ϕ(n)\phi(n)?

RSA's decryption correctness relies on ed≡1(modϕ(n))ed \equiv 1 \pmod{\phi(n)} so that med≡m(modn)m^{ed} \equiv m \pmod n via Euler's Theorem. This congruence can only be satisfied if ee has a multiplicative inverse modulo ϕ(n)\phi(n), and by the Bézout/gcd argument, an inverse exists exactly when gcd⁡(e,ϕ(n))=1\gcd(e, \phi(n)) = 1. If ee shared a common factor with ϕ(n)\phi(n), no integer dd would solve ed≡1(modϕ(n))ed \equiv 1 \pmod{\phi(n)}, and the entire encrypt-then-decrypt round trip would break for some or all messages.

Is the extended Euclidean algorithm the only way to compute a modular inverse?

No — an alternative is Euler's Theorem itself: since aϕ(n)≡1(modn)a^{\phi(n)} \equiv 1 \pmod n when gcd⁡(a,n)=1\gcd(a,n)=1, we have a−1≡aϕ(n)−1(modn)a^{-1} \equiv a^{\phi(n)-1} \pmod n, computable via fast modular exponentiation. However, this requires knowing ϕ(n)\phi(n), which is easy if you know the prime factorization of nn but hard otherwise — indeed, computing ϕ(n)\phi(n) for RSA's public modulus is exactly as hard as factoring nn. The extended Euclidean algorithm is preferred because it needs no factorization at all, only aa and nn.

What happens if the modulus nn in cryptography is not prime but a product of two primes?

That is precisely RSA's construction: n=pqn = pq for two large primes. The multiplicative structure modulo nn is not as simple as modulo a prime — not every nonzero residue is invertible, only those coprime to nn (i.e., not multiples of pp or qq). Euler's Theorem, rather than the simpler Fermat's Little Theorem, is the correct tool here because it applies to any modulus, and ϕ(n)=(p−1)(q−1)\phi(n) = (p-1)(q-1) can be computed only if the factorization of nn is known — which is the secret that makes RSA's private key private.

Why can't we just use trial division to find large primes for cryptography?

Trial division to confirm a number pp is prime takes roughly O(p)O(\sqrt{p}) divisions in the worst case. For a 1024-bit prime, p\sqrt{p} is itself a number with about 512 bits — far too large to enumerate divisors up to, even at billions of operations per second, within the lifetime of the universe. Probabilistic tests like Miller–Rabin instead run in time polynomial in the number of bits of pp (specifically O(klog⁡3p)O(k \log^3 p) for kk rounds with schoolbook arithmetic), making them the only practical option at cryptographic scale.

How does this relate to the Discrete Logarithm Problem mentioned at the end?

Modular exponentiation is easy to compute forward: given aa, bb, and nn, finding ab  na^b \bmod n takes only O(log⁡b)O(\log b) steps via repeated squaring. The Discrete Logarithm Problem asks the reverse question — given aa, nn, and c=ab  nc = a^b \bmod n, find bb — and no efficient classical algorithm is known for well-chosen groups. This asymmetry between easy exponentiation and hard "delogging" is a second, independent source of cryptographic hardness alongside integer factorization, and it powers Diffie–Hellman key exchange and elliptic-curve cryptography.

Does any of this arithmetic break if nn itself is prime versus composite?

Fermat's Little Theorem only applies when the modulus is prime, while Euler's Theorem applies to any modulus, reducing to Fermat's theorem as the special case n=pn = p. Practically, this matters because Diffie–Hellman typically works modulo a large prime (so the simpler Fermat's theorem and a cyclic group structure apply directly), while RSA deliberately uses a composite modulus n=pqn = pq so that factoring becomes the hard problem protecting the private key, requiring the more general Euler's Theorem to reason about correctness.

Where this leads