Tikhonov regularisation can pull the solution towards a prior guess instead of…

PHYSICS · Multi-part · Fermi University Physics · Mathematical Methods for Physics · Numerical and computational mathematics · Numerical and computational mathematics

Error, Chance and the Machine, Ch Linear systems, least squares and the singular value decomposition

Problem

Tikhonov regularisation can pull the solution towards a prior guess x0\mathbf x_0 instead of towards zero, by minimising J(x)=∥Ax−b∥2+λ2∥x−x0∥2,J(\mathbf x)=\|A\mathbf x-\mathbf b\|^2+\lambda^2\|\mathbf x-\mathbf x_0\|^2, for a chosen regularisation parameter λ>0\lambda>0, where AA is an m×nm\times n matrix (m≥nm\ge n) of full column rank with singular value decomposition A=UΣVTA=U\Sigma V^{\rm T}: UU has orthonormal columns u1,…,un\mathbf u_1,\dots,\mathbf u_n, VV is n×nn\times n orthogonal with columns v1,…,vn\mathbf v_1,\dots,\mathbf v_n, and Σ=diag(σ1,…,σn)\Sigma=\mathrm{diag}(\sigma_1,\dots,\sigma_n) with σ1≥⋯≥σn>0\sigma_1\ge\dots\ge\sigma_n>0. ((a)) Show that the minimiser xλ\mathbf x_\lambda of JJ satisfies (ATA+λ21)x=ATb+λ2x0(A^{\rm T}A+\lambda^2\mathsf 1)\mathbf x=A^{\rm T}\mathbf b+\lambda^2\mathbf x_0, and that its components along the vi\mathbf v_i are viTxλ=fi uiTbσi+(1−fi) viTx0,fi=σi2σi2+λ2.\mathbf v_i^{\rm T}\mathbf x_\lambda=f_i\,\frac{\mathbf u_i^{\rm T}\mathbf b}{\sigma_i}+(1-f_i)\,\mathbf v_i^{\rm T}\mathbf x_0,\qquad f_i=\frac{\sigma_i^2}{\sigma_i^2+\lambda^2}. ((b)) State the limits of viTxλ\mathbf v_i^{\rm T}\mathbf x_\lambda for σi≫λ\sigma_i\gg\lambda and for σi≪λ\sigma_i\ll\lambda, say what the regularisation now does to a component the data cannot determine, and check that x0=0\mathbf x_0=\mathbf 0 returns the chapter's filtered sum. ((c)) An inverse problem retains three singular components with σ1=1.00\sigma_1=1.00, σ2=0.0500\sigma_2=0.0500, σ3=1.00×10−4\sigma_3=1.00\times10^{-4}, measured data coefficients uiTb=0.500\mathbf u_i^{\rm T}\mathbf b=0.500, 0.01000.0100, 3.00×10−63.00\times10^{-6} and prior components viTx0=0.400\mathbf v_i^{\rm T}\mathbf x_0=0.400, 0.3000.300, 0.1500.150. For λ=2.00×10−3\lambda=2.00\times10^{-3}, compute f1,f2,f3f_1,f_2,f_3 and the three components viTxλ\mathbf v_i^{\rm T}\mathbf x_\lambda to three significant figures.

Browse the problem library