Superposition in Neural Networks

Superposition in neural networks refers to the phenomenon whereby a network encodes more features than it has dimensions, representing multiple distinct concepts in overlapping, non-orthogonal directions within a single layer's activation space. This matters enormously for modern AI interpretability because it implies that individual neurons are not the fundamental unit of meaning — features are — and that the geometry of representation is far richer and more entangled than classical linear probing suggests. The remarkable fact is that a network can represent exponentially many sparse features in a space of fixed dimension with only polynomial interference, a result with deep connections to compressed sensing and random coding theory.

The core insight

A neural network with $n$ neurons in a layer need not be limited to representing $n$ independent concepts: if the true underlying features of the data are sufficiently sparse and the network tolerates a small amount of interference, it can store $m \gg n$ features as nearly-orthogonal directions in $\mathbb{R}^n$ , exploiting the counterintuitive geometry of high-dimensional spaces where exponentially many almost-orthogonal vectors exist. The "aha moment" is realising that monosemanticity (one neuron, one concept) is not the default — it is the special case that occurs only when features are dense and the network has sufficient capacity, while superposition is the generic solution to representational compression under sparsity. This fundamentally breaks the interpretability assumption that neurons are the right level of analysis, demanding instead that we search for directions in activation space.

Common trapCommon Misconception

The most common error is conflating superposition with polysemanticity and treating them as interchangeable explanations for why neurons respond to multiple unrelated stimuli. They are distinct: polysemanticity is the observed phenomenon (a neuron activates for cats AND curves AND certain textures), while superposition is one mechanistic explanation (multiple sparse features are stored in overlapping directions, and this neuron happens to have non-negligible projections onto several of them). The correct mental model is that superposition is a representational strategy at the level of the weight geometry, whereas polysemanticity is the symptom visible at the neuron level; a network could in principle be polysemantic without superposition (e.g., genuine multi-purpose computation) and interpretability methods that only examine individual neurons will conflate these cases, leading to incorrect conclusions about what concepts the network has actually learned.

How It Works

The Toy Model of Superposition

Following Elhage et al. (2022), consider a one-hidden-layer bottleneck autoencoder that must reconstruct $m$ features from a $d$ -dimensional hidden representation with $d < m$ . Let $\mathbf{x} \in \mathbb{R}^m$ be a sparse input where feature $i$ is active independently with probability $p_i \ll 1$ . The model learns an embedding matrix $W \in \mathbb{R}^{d \times m}$ and reconstructs via:

\quad \hat{\mathbf{x}} = \text{ReLU}\!\left(W^\top W\, \mathbf{x} + \mathbf{b}\right)

Explanation: The network projects the input into a low-dimensional space via $W$ and then projects back with $W^\top$ , applying a nonlinearity to suppress cross-feature interference. Here $W^\top W \in \mathbb{R}^{m \times m}$ is the effective feature interaction matrix. Denote the columns of $W$ as $\{\mathbf{w}_i\}_{i=1}^m$ , each a direction in $\mathbb{R}^d$ . The expected loss decomposes into a self-term and an interference term. Defining $f_i = \|\mathbf{w}_i\|^2$ (feature strength) and $S_{ij} = (\mathbf{w}_i^\top \mathbf{w}_j)^2 / (\|\mathbf{w}_i\|^2 \|\mathbf{w}_j\|^2)$ (cosine-squared similarity), the total loss is:

\quad \mathcal{L} = \sum_{i=1}^{m} \lambda_i \!\left[(1 - f_i)^2 p_i + \sum_{j \neq i} f_j^2\, S_{ij}\, p_j\right]

Explanation: Each feature pays a reconstruction cost from being underrepresented (first term) and an interference cost from all other co-encoded features bleeding into its direction (second term), weighted by feature importance $\lambda_i$ and sparsity $p_i$ . The key insight from this loss is that sparsity $p_i$ discounts the interference term: if only a small fraction of features are simultaneously active, the expected cross-contamination is low even when $S_{ij}$ is non-negligible. The network therefore faces an optimization over geometry — finding a packing of $m$ vectors into $\mathbb{R}^d$ that minimises weighted interference.

Capacity and the Johnson–Lindenstrauss Bound

The theoretical capacity of superposition is quantified via nearly-orthogonal vector packings. For a tolerance $\epsilon > 0$ , a set of $m$ unit vectors $\{\mathbf{w}_i\} \subset \mathbb{R}^d$ satisfying $|\mathbf{w}_i^\top \mathbf{w}_j| \leq \epsilon$ for all $i \neq j$ constitutes an $\epsilon$ -nearly-orthogonal family. By a probabilistic argument (random unit vectors in $\mathbb{R}^d$ ), one can pack:

\quad m \leq \exp\!\left(\frac{\epsilon^2 d}{2}\right)

Explanation: The number of nearly-orthogonal directions grows exponentially in the ambient dimension $d$ for any fixed interference tolerance $\epsilon$ , formalising why high-dimensional spaces can store vastly more features than they have coordinate axes. This is the geometric miracle underlying superposition: a 512-dimensional hidden layer with interference tolerance $\epsilon = 0.1$ can in principle encode on the order of $e^{2.56} \approx 13$ features per dimension, giving thousands of representable features. The catch is that simultaneous activation of two superposed features produces interference proportional to $\epsilon$ , which is acceptable only when activations are sparse.

The Phase Transition

Elhage et al. (2022) identified a sharp phase transition in the learned geometry as a function of the sparsity-importance product $\lambda_i (1 - p_i)$ . Let $S = 1 - p$ (sparsity) for a uniform model. There exists a critical threshold $S^*$ such that:

\quad f_i^* = \begin{cases} 1 & S > S^*(d, m) \\ 0 & S < S^*(d, m) \end{cases}

Explanation: Below the critical sparsity, the network simply ignores lower-priority features entirely (pure dimensionality reduction); above it, it pays the interference cost to superpose features, with the transition becoming sharper as $m/d$ increases. When features enter superposition, their representing directions snap to highly structured configurations — antipodal pairs, regular polygons projected into lower dimensions, or uniform polytope arrangements — minimising mutual interference subject to the geometric constraints. This polyhedral geometry of superposition has been confirmed empirically in toy models and partially in large language models via sparse autoencoders.

Interference and the ReLU Benefit

The ReLU nonlinearity plays a crucial mechanistic role that pure linear superposition cannot exploit. For two superposed features $i, j$ with directions satisfying $\mathbf{w}_i^\top \mathbf{w}_j = \epsilon$ , when only feature $i$ is active ( $x_i = 1, x_j = 0$ ), the pre-activation for feature $j$ 's reconstruction is $\epsilon x_i$ . The ReLU sets this to zero if the bias $b_j$ is tuned such that $\epsilon - |b_j| < 0$ :

\quad \text{ReLU}(\epsilon\, x_i + b_j) = 0 \quad \text{iff} \quad b_j < -\epsilon

Explanation: By setting a negative bias slightly below the interference threshold, the network uses the ReLU to actively cancel cross-feature contamination, meaning the nonlinearity is not merely for expressive power but serves as a denoising gate for superposed representations. This is why superposition is qualitatively different in networks with nonlinearities: the network learns both a geometric packing (the $W$ matrix) and a denoising schedule (the bias vector) jointly, achieving effective separation that pure linear projections cannot.

researcher · Chris Olah | b. 1992 | Mechanistic Interpretability and Feature Superposition

Chris Olah developed the mechanistic interpretability research programme at OpenAI and later Anthropic, introducing the rigorous use of circuits analysis to understand how neural networks implement algorithms in terms of weights and activations. His 2022 collaboration "Toy Models of Superposition" (with Elhage, Nanda, and colleagues at Anthropic) provided the first mathematically precise framework for understanding how and why neural networks encode more features than dimensions, connecting the phenomenon to sparsity, capacity, and polyhedral geometry. Unusually for a field dominated by PhD-credentialed researchers, Olah has no formal graduate degree, having built his reputation entirely through self-directed research and extraordinarily influential blog posts and papers on neural network interpretability.

researcher · Neel Nanda | b. 1998 | Empirical Verification and Sparse Autoencoders for Superposition

Neel Nanda, first at Anthropic and later DeepMind, became one of the primary empirical investigators of superposition in large-scale transformer models, developing and popularising sparse autoencoders (SAEs) as a practical tool for decomposing polysemantic neuron activations into monosemantic feature directions. His work on "Progress measures for grokking via mechanistic interpretability" (2023) and subsequent SAE scaling studies demonstrated that superposition is not merely a toy-model curiosity but is systematically present and structurally organised in models like GPT-2 and Claude. Nanda has been notably transparent about his research process, maintaining public codebases and detailed research notes that have made the sparse autoencoder methodology accessible across the interpretability community.

Figure

Superposition Geometry: Feature Packing in Low-Dimensional Space

Figure

Sparse Autoencoder Pipeline for Superposition Recovery

Intuition

The Mental Model

Imagine a small theatre with only 10 spotlights, but the director needs to represent 50 different characters across many scenes. The naive approach assigns one spotlight per character — but that only covers 10. The superposition strategy instead assigns each character a unique blend of spotlight colours and positions, exploiting the fact that at any given moment in the play only 2 or 3 characters are on stage simultaneously. When character 37 appears, a specific combination of spotlights fires; when character 22 appears, a different combination fires. As long as characters don't crowd the stage all at once (sparsity), the audience can reliably identify who is present even though no single spotlight "belongs" to one person. A neuron in superposition is like one of those spotlights: it responds to many characters (polysemanticity), not because it is confused, but because the network has cleverly divided representational responsibility across blended directions. The ReLU is the stage manager who dims residual bleed-through from off-stage characters to zero.

Historical Development

Origins · 1 minhow this was discovered

History

1943

McCulloch and Pitts formalise the neuron as a threshold logic unit, implicitly assuming one-neuron-one-concept — the assumption superposition would eventually undermine.

1986

Hinton, Rumelhart, and Williams introduce backpropagation, enabling distributed representations but also the polysemantic neurons that would puzzle researchers for decades.

2017

Olah et al. (distill.pub "Feature Visualisation") systematically show that individual CNN neurons respond to multiple distinct concepts, naming polysemanticity as a core interpretability problem.

2020

Goh et al. (Multimodal Neurons, OpenAI) find neurons in CLIP that respond to the same concept across text and images, prompting questions about whether this reflects genuine multi-purpose computation or superposition.

2022

Elhage, Nanda, Olah et al. ("Toy Models of Superposition", Anthropic) provide the first mathematical framework for superposition, defining the phase transition, polyhedral geometry, and the role of sparsity and ReLU in enabling the phenomenon.

2023

Cunningham, Conmy, Nanda et al. ("Sparse Autoencoders Find Highly Interpretable Features in Language Models") demonstrate that SAEs applied to GPT-2 recover monosemantic feature directions from superposed activations at scale, empirically validating the framework.

2023

Bricken, Templeton et al. (Anthropic) train SAEs on a one-layer transformer and find that recovered features exhibit clean geometric structure (antipodal pairs, pentagons) matching the theoretical predictions of Elhage et al.

2024

Templeton, Conerly et al. (Anthropic, "Scaling and evaluating sparse autoencoders") extend SAE methodology to Claude 3 Sonnet, finding millions of interpretable features and establishing scaling laws for superposition decomposition in frontier models.

2024

Lindsey, Gurnee et al. introduce cross-layer superposition and demonstrate that features can span multiple layers in residual stream models, challenging single-layer analyses and prompting circuit-level superposition studies.

Why It Matters

Real-World Impact

GPT-2 and Language Model Mechanistic InterpretabilitySparse autoencoders applied to GPT-2's MLP and residual stream layers (Cunningham et al., 2023) recovered thousands of monosemantic feature directions — including features for specific syntactic roles, named entities, and sentiment — that were completely invisible at the neuron level due to superposition, changing how researchers locate and study internal concepts in LLMs.

Claude (Anthropic)Anthropic's 2024 SAE scaling work on Claude 3 Sonnet identified millions of sparse features in superposition across the residual stream, including features for emotionally loaded concepts, stereotypes, and safety-relevant triggers, enabling the first direct observation of how safety-critical representations are encoded and potentially manipulated.

AI Safety and Deceptive Alignment DetectionThe superposition framework provides a theoretical basis for why deceptive alignment — a model appearing aligned during training but harbouring misaligned internal goals — may be difficult to detect: the relevant goal representations could be stored in superposition, activating only in rare deployment contexts and invisible to activation-level monitoring trained on training-distribution data.

Adversarial Robustness ResearchSuperposition partially explains why adversarial examples exist — small perturbations in input space can project onto many superposed feature directions simultaneously, triggering unexpected feature combinations that the network treats as valid inputs but humans do not recognise.

Sparse Autoencoders as Interpretability InfrastructureThe practical tool that superposition theory motivated — the sparse autoencoder — is now a standard component of the interpretability research stack at Anthropic, DeepMind, and EleutherAI, used to decompose activations for circuit tracing, feature steering, and automated concept labelling at scale.

Model Editing and Concept ErasureUnderstanding superposition informs targeted weight editing: modifying a neuron to remove an undesired concept may inadvertently erase superposed concepts sharing that direction, so superposition-aware editing (operating on identified feature directions rather than neurons) achieves more surgical control, as demonstrated by Hernandez et al. (2023) in the context of factual association editing in GPT models.

Feature Steering and Activation EngineeringSuperposition theory enables activation steering by identifying the specific direction in activation space corresponding to a concept (rather than assuming it aligns with a single neuron basis vector), yielding more precise and less collateral-damage-prone behavioural interventions, as applied in Rimsky et al. (2023) and Turner et al. (2023) for refusal and sycophancy steering.

Superposition in Neural Networks

How It Works

The Toy Model of Superposition

Capacity and the Johnson–Lindenstrauss Bound

The Phase Transition

Interference and the ReLU Benefit

Intuition

Historical Development

Why It Matters

Prerequisites

What This Leads To

See Also

Before this