A collaborator's iterative eigensolver for a discretised Laplacian on a mesh of…

PHYSICS · Subjective · Fermi University Physics · Computational Physics · High-performance and data-driven computational physics · High-performance and data-driven computational physics

Scientific Computing for Physicists, Ch Graphics processors: kernels, memory and the roofline

Problem

A collaborator's iterative eigensolver for a discretised Laplacian on a mesh of

N=2\times10^{6}

unknowns repeats a sparse matrix-vector product (SpMV)

2000

times per run. The matrix has

\mathrm{nnz}=5N=1\times10^{7}

nonzeros (a five-point stencil in sparse form), stored in compressed-row form with one

8

-byte binary64 value and one

4

-byte integer column index per nonzero. Idealise the SpMV kernel's memory traffic as

20\,\mathrm{B}

moved and

2

floating-point operations (one multiply, one add) performed per stored nonzero, the same on the CPU and on the GPU. Between calls the matrix and the current vector stay resident on whichever device runs the eigensolver; a one-off transfer at the start moves the matrix (

12\,\mathrm{B}

per nonzero) and an initial vector (

N

binary64 values) to the device, and a one-off transfer at the end returns the final vector (

N

binary64 values), over a bus with peak bandwidth

32\,\mathrm{GB\,s^{-1}}

. The CPU node has measured triad bandwidth

\beta_{\rm mem}=180\,\mathrm{GB\,s^{-1}}

and measured matrix-product peak

P_{\rm peak}=1.8\,\mathrm{TFLOP\,s^{-1}}

(ridge point

10\,\mathrm{flop\,B^{-1}}

). The GPU is an A100 with

\beta_{\rm dev}=1555\,\mathrm{GB\,s^{-1}}

and

P_{\rm peak}=9.7\,\mathrm{TFLOP\,s^{-1}}

(ridge point

6.2\,\mathrm{flop\,B^{-1}}

). Decide whether porting this eigensolver's SpMV kernel to the GPU is likely to pay off. State the decision as a short phrase and justify it quantitatively using the kernel's arithmetic intensity, both rooflines and the one-off transfer cost.

Browse the problem library