Scientific Computing for Physicists

Chapters

1. Arrays in memory: layout, cost and the data structures of physical problems
   • A diffusion grid that does not fit
   • Memory layout and strides
   • Views and copies
   • The memory hierarchy and bandwidth
   • Arithmetic intensity of the diffusion stencil
   • Temporaries, in-place updates and index contraction
   • Sparse storage of a grid operator
   • The cell list and the layout of records
   • What arithmetic intensity settles and leaves open
   • Problems
2. The numerical toolkit closed: backward error and the audit of a library routine
   • One million tenths, three sums
   • Forward and backward error
   • Compensated summation
   • Safeguarded root finding for the finite well
   • Newton's method for systems
   • Audited quadrature and its error estimate
   • Infinite and oscillatory integrands
   • Two-dimensional table interpolation
   • Choosing a numerical method
   • Problems
3. Letting the computer do advanced algebra: operators, series and differentiated programs
   • Sixteen orderings of one operator
   • Assumptions and domains of symbols
   • Symbolic linear algebra for normal modes
   • Non-commuting algebra of ladder operators
   • Symbolic perturbation series
   • Symbolic ODEs for a driven, damped oscillator
   • Expression swell and common subexpressions
   • Code generation for gradients and Jacobians
   • Dual numbers for differentiating a program
   • Comparing derivative methods
   • Problems
4. Minimisation with guarantees, with constraints and with randomness
   • Fifty minimisations of thirteen atoms
   • Descent and line search
   • Hessian conditioning and slow valleys
   • Newton's method for minimisation and BFGS
   • Levenberg-Marquardt for difficult fits
   • Convexity and local minima
   • Constrained minimisation on a sphere and a chain
   • Energy landscapes, simulated annealing and basin hopping
   • Convexity and the limits of global search
   • Problems
5. Spectra with error bars: the fast transform, convolution and spectral estimation
   • A tall bin in a noise spectrum
   • The FFT algorithm by even-odd splitting
   • Circular convolution and zero padding
   • PSD normalisation and physical units
   • Periodogram variance of noise
   • Welch averaging of segments
   • Window trade-offs
   • The spectrogram of a changing frequency
   • Filtering and detrending a record
   • The spectral derivative
   • Spectrograms, Welch averaging and open questions
   • Problems
6. A repository others can build on: collaboration, history and workflows
   • A figure that changed in six hundred commits
   • Branches, merges and merge conflicts
   • Bisection search through history
   • Code review of a change
   • Environment specification with lockfiles and containers
   • The workflow graph of a study
   • Notebook hidden state and stale figures
   • Archiving and citation of a release
   • Problems
7. Is the equation solved right? Testing, debugging and code verification
   • From three tests to a test suite
   • Writing a unit test
   • A tolerance from error analysis
   • Property tests for unknown answers
   • Regression and CI, and what coverage misses
   • The debugging method and numerical failure modes
   • The order-of-accuracy test
   • Verification with manufactured solutions
   • What an order-of-accuracy test establishes
   • Problems
8. Are they the right equations? Solution verification, benchmarks and validation
   • Passing every test, missing by 42 arcseconds
   • Verification versus validation
   • Solution verification and the grid convergence index
   • Benchmark comparison and what agreement proves
   • The validation comparison and model-form error
   • The validation domain
   • Performance benchmarking
   • What a validation comparison still needs
   • Problems
9. The uncertainty of a simulated number: inputs, sensitivity and correlated output
   • The Ising model as a source of data
   • Aleatory and epistemic uncertainty
   • Input propagation, linear and Monte Carlo
   • Sensitivity indices
   • The autocorrelation time and statistical inefficiency
   • Block averaging and the simulation bootstrap
   • The uncertainty budget of a simulated number
   • What an uncertainty budget settles
   • Problems
10. Many processors: speed-up, decomposition and the cost of communication
    • Performance benchmarking on eight cores
    • Amdahl's law and weak scaling
    • Shared and distributed memory
    • Compiled loops and race conditions
    • Domain decomposition of a grid
    • Communication cost of a message
    • Load balance and reproducible reductions
    • Reporting a scaling study
    • What the scaling study establishes
    • Problems
11. Graphics processors: kernels, memory and the roofline
    • The explicit diffusion scheme on a GPU
    • The SIMT model of a GPU
    • GPU memory hierarchy and coalescing
    • A compiled kernel for the stencil
    • Host-device transfer
    • The roofline of two machines
    • Mixed precision and stagnation
    • GPU libraries and when a GPU pays
    • Problems
12. Learning from physical data: regression, classification and principal components
    • Supervised learning of an Ising chain
    • Generalisation error and the held-out test
    • The bias-variance trade-off
    • Ridge regularisation and cross-validation
    • Logistic regression of Ising phases
    • Stochastic gradient descent
    • Principal component analysis
    • Phase identification and correlated leakage
    • Limits of phase identification
    • Problems
13. Neural networks on physical data: backpropagation, convolution and symmetry
    • The feature logistic regression lacked
    • Neurons, layers and universal approximation
    • Backpropagation as reverse-mode differentiation
    • Initialisation and optimisers for deep networks
    • Memorisation and overfitting controls
    • The convolutional layer
    • Symmetry in learning
    • Learning a transition with error bars
    • Prediction uncertainty and what remains open
    • Problems