Every AI concept, explained from first principles.

Articles

• AI in Science: AlphaFold — history
• Alan Turing and the Origins of AI — history
• AlexNet and the Deep Learning Breakthrough — history
• AlphaGo and Game-Playing AI — history
• Geoffrey Hinton: The Godfather of Deep Learning — history
• John McCarthy and Symbolic AI — history
• Marvin Minsky and the Perceptron Controversy — history
• Prompt Engineering — agents
• The Alignment Problem — alignment
• The Dartmouth Conference 1956 — history
• The First AI Winter — history
• The GPT Moment — history
• The ImageNet Revolution — history
• The Perceptron and Early Neural Networks — history
• The Rise of Expert Systems — history
• The Second AI Winter — history
• The Turing Test — history
• What is a Large Language Model? — llms
• Yann LeCun and Convolutional Networks — history
• Yoshua Bengio and Deep Representation Learning — history
• Activation Functions — neural-networks
• AlexNet — computer-vision
• Anomaly Detection — classical-ml
• Bag of Words and TF-IDF — nlp
• Benchmark Evaluation of LLMs — llms
• Chain-of-Thought Prompting — llms
• Chain-of-Thought Reasoning — agents
• Claude Shannon and Information Theory — history
• Convolutional Filters and Feature Maps — computer-vision
• Convolutional Neural Networks — computer-vision
• DALL-E and Text-to-Image Generation — multimodal
• Data Augmentation — neural-networks
• DBSCAN Clustering — classical-ml
• Decision Trees — classical-ml
• Dropout Regularization — neural-networks
• Early Stopping — neural-networks
• FastText — nlp
• Full Fine-Tuning vs Parameter-Efficient Fine-Tuning — efficiency
• GGUF and llama.cpp — efficiency
• GPT-2: Language Models as Unsupervised Learners — llms
• GPT-3 and In-Context Learning — llms
• ImageNet and Benchmark Culture — computer-vision
• Instruction Tuning — llms
• K-Means Clustering — classical-ml
• K-Nearest Neighbors — classical-ml
• LangChain and LangGraph — agents
• Language Model Pretraining — llms
• Linear Regression — classical-ml
• Logistic Regression — classical-ml
• Masked Language Modeling — transformers
• MMLU and Standard AI Benchmarks — alignment
• Model Cards and Responsible Disclosure — alignment
• Model Context Protocol (MCP) — agents
• Model Quantization — efficiency
• Multi-Layer Perceptrons — neural-networks
• Naive Bayes Classifier — classical-ml
• Named Entity Recognition — nlp
• Open Source LLMs: Llama, Mistral, Phi — llms
• Overfitting and Generalization — theory
• Pooling Layers — computer-vision
• Principal Component Analysis — classical-ml
• Random Forests — classical-ml
• Recommendation Systems — classical-ml
• ReLU and Its Variants — neural-networks
• Retrieval-Augmented Generation (RAG) — agents
• Reward Hacking and Goodhart's Law — alignment
• Ridge and Lasso Regression — classical-ml
• Self-Attention — transformers
• Sentiment Analysis — nlp
• Text-to-Speech with Neural Models — multimodal
• The Attention Mechanism — transformers
• The Bitter Lesson — history
• The History of Reinforcement Learning — history
• The Perceptron — neural-networks
• The Vanishing Gradient Problem — neural-networks
• Tokenization: BPE and SentencePiece — llms
• Tool Use and Function Calling — agents
• t-SNE — classical-ml
• VGGNet — computer-vision
• Weight Decay and L2 Regularization — neural-networks
• Whisper: Robust Speech Recognition — multimodal
• Word Embeddings — nlp
• Zero-Shot and Few-Shot Prompting — agents
• Agent Evaluation and Benchmarks — agents
• Agentic Loops and Scaffolding — agents
• AI Evaluations and Red-Teaming — alignment
• AI for Scientific Discovery — frontier
• AI Risk Frameworks — alignment
• Anthropic's Approach to AI Safety — alignment
• Automatic Differentiation — neural-networks
• Backpropagation — neural-networks
• Bahdanau Attention — transformers
• Batch Normalization — neural-networks
• BERT: Bidirectional Encoder Representations — transformers
• Bias-Variance Tradeoff — theory
• Byte-Pair Encoding — llms
• Chinchilla and Optimal Compute — llms
• Chunking Strategies for RAG — agents
• CLIP: Contrastive Language-Image Pre-Training — multimodal
• Constitutional AI — llms
• Cross-Attention — transformers
• Dense Retrieval and Bi-Encoders — agents
• EfficientNet and Compound Scaling — computer-vision
• Eigenvalues and Eigenvectors — foundations
• ELMo: Contextual Word Embeddings — nlp
• Emergent Capabilities in Large Models — llms
• Encoder-Decoder Architectures — neural-networks
• Episodic and Semantic Memory for Agents — agents
• Gated Recurrent Units (GRU) — neural-networks
• Gaussian Mixture Models — classical-ml
• GELU Activation Function — neural-networks
• Generative Adversarial Networks — multimodal
• GloVe: Global Vectors for Word Representation — nlp
• GPT-4V and Vision-Language Models — multimodal
• GPT: Autoregressive Language Models — transformers
• Gradient Boosting — classical-ml
• InceptionNet and GoogLeNet — computer-vision
• Inference Serving Systems — efficiency
• Inner Alignment vs Outer Alignment — alignment
• INT4 and GPTQ Quantization — efficiency
• INT8 Quantization — efficiency
• Key-Value Cache — transformers
• Knowledge Distillation — efficiency
• KV Cache Optimization — efficiency
• Layer Normalization — neural-networks
• Long Short-Term Memory (LSTM) — neural-networks
• Low-Rank Adaptation (LoRA) — efficiency
• Mode Collapse in GANs — multimodal
• Model Pruning — efficiency
• Monte Carlo Methods — foundations
• Multi-Agent Systems — agents
• Multi-Head Attention — transformers
• Multimodal Embeddings — multimodal
• No Free Lunch Theorem — theory
• Object Detection: Faster R-CNN — computer-vision
• Object Detection: YOLO — computer-vision
• ONNX and TensorRT — efficiency
• OpenAI o1 and o3: Chain-of-Thought at Scale — frontier
• Positional Encoding — transformers
• QLoRA: Efficient Fine-Tuning — efficiency
• ReAct: Reasoning and Acting — agents
• Reasoning Models and Test-Time Compute — frontier
• Recurrent Neural Networks — neural-networks
• Reranking in RAG Pipelines — agents
• Residual Connections and ResNets — neural-networks
• ResNet: Deep Residual Learning — computer-vision
• Reward Modeling — llms
• RLHF: Reinforcement Learning from Human Feedback — llms
• RMS Normalization — neural-networks
• Scaled Dot-Product Attention — transformers
• Scaling Laws for Neural Language Models — llms
• Semantic Segmentation — computer-vision
• Sequence-to-Sequence Models — nlp
• Singular Value Decomposition — foundations
• Stable Diffusion Architecture — multimodal
• Support Vector Machines — classical-ml
• T5: Text-to-Text Transfer Transformer — transformers
• The Bias-Variance Decomposition — theory
• The Decoder-Only Transformer — transformers
• The EM Algorithm — classical-ml
• The Encoder-Only Transformer — transformers
• The Kernel Trick — classical-ml
• The Transformer Architecture — transformers
• UMAP — classical-ml
• Variational Autoencoders — multimodal
• Video Generation Models — multimodal
• Vision Transformer (ViT) — computer-vision
• Weight Initialization — neural-networks
• Word2Vec — nlp
• XGBoost — classical-ml
• Activation Patching and Causal Tracing — alignment
• ALiBi Position Encoding — transformers
• Bayesian Optimization — classical-ml
• Circuits in Neural Networks — alignment
• Classifier-Free Guidance — multimodal
• Continuous Learning and Catastrophic Forgetting — frontier
• Contrastive Learning in Vision — computer-vision
• DDIM: Denoising Diffusion Implicit Models — multimodal
• Debate as an Alignment Technique — alignment
• Deceptive Alignment — alignment
• Denoising Diffusion Probabilistic Models — multimodal
• DINO: Self-Supervised Vision Features — computer-vision
• Direct Preference Optimization (DPO) — llms
• Double Descent Phenomenon — theory
• FlashAttention — transformers
• Gaussian Processes — classical-ml
• IA³: Fewer Parameters Than LoRA — efficiency
• In-Context Learning Theory — llms
• Long Context and Memory-Augmented LLMs — frontier
• Mechanistic Interpretability — alignment
• Mixture of Experts at Scale — frontier
• Mixture of Experts for LLMs — llms
• Neuromorphic Computing — frontier
• Neurosymbolic AI — frontier
• PAC Learning Framework — theory
• Paged Attention and vLLM — efficiency
• Pipeline Parallelism — efficiency
• Proximal Policy Optimization in RLHF — llms
• Rademacher Complexity — theory
• Robotics Foundation Models: RT-2 and Pi-0 — frontier
• Rotary Position Embedding (RoPE) — transformers
• Scalable Oversight — alignment
• Score Matching and Score-Based Generative Models — multimodal
• Sliding Window Attention — transformers
• Speculative Decoding — llms
• Superposition in Neural Networks — alignment
• Tensor Parallelism — efficiency
• Universal Approximation Theorem — neural-networks
• VC Dimension — theory
• What Comes After Transformers? — frontier
• World Models for Planning — frontier
• Benign Overfitting — theory
• Dreamer: Learning World Models — frontier
• Generalization Bounds and PAC-Bayes — theory
• Information Bottleneck Theory — theory
• Multimodal Reasoning at Frontier — frontier
• Neural Tangent Kernel — neural-networks
• Neural Tangent Kernel Theory — theory
• Open Problems in AI Alignment — frontier
• Sparse Autoencoders for Interpretability — alignment