PHYSICS · Multi-part · Fermi University Physics · Mathematical Methods for Physics · Computational Physics · Linear algebra and vector spaces
Scientific Computing for Physicists, Ch Learning from physical data: regression, classification and principal components
Problem
Three short derivations, each checked with numbers.
((a)) A resonant sensor's resonance frequency is estimated by fitting a ring-down curve; the fitting procedure is repeated on K=4 independent noisy records, giving resonance-frequency estimates f^(1)=500.3, f^(2)=499.7, f^(3)=500.5, f^(4)=499.9Hz. An independent static calibration gives the true frequency f0=500.0Hz, and a new, independent frequency measurement y=f0+ε carries noise of variance σ2=0.05Hz2, known from that calibration. With ε∼N(0,σ2) independent of the fitting procedure, derive E[(y−f^)2]=(f0−E[f^])2+Var(f^)+σ2, then use the four repeats to estimate the bias squared, the variance and their sum, treating the sample mean and the mean squared deviation as the empirical expectation and variance.
((b)) A ring of N=20 Ising spins with periodic boundary conditions has true energy E=−J∑jsjsj+1 with J=0.6 (in units of kBT), fitted by ridge regression using the general quadratic form E=∑j,kJjksjsk, so that sjsj+1 and sj+1sj enter as two separate, numerically identical feature columns whose fitted coefficients add to give the physical nearest-neighbour term. Show, by minimising the ridge penalty θa2+θb2 over splits of a fixed sum θa+θb=c, that any strictly positive ridge penalty λ>0 shares whatever sum c the fit returns equally, θa=θb=c/2. Then, taking the penalty small and the data plentiful enough that ridge's shrinkage of c itself is negligible, so that c≈−J, give the approximate value each of the two learned coefficients Jj,j+1 and Jj+1,j takes.
((c)) A linear classifier assigns a magnetic domain the label y=1 (ordered) or y=0 (disordered) with probability p=σ(wx+b), σ(z)=1/(1+e−z), treating each labelled configuration as an independent Bernoulli trial. Derive the cross-entropy loss L=−∑i[yilnpi+(1−yi)ln(1−pi)] from the Bernoulli likelihood and its gradient ∂L/∂w=∑i(pi−yi)xi, ∂L/∂b=∑i(pi−yi). Evaluate both gradient components at w=0.4, b=−0.6 for the three labelled points (xi,yi)=(0.5,0),(1.5,1),(2.5,1).