PHYSICS · Multi-part · Fermi University Physics · Computational Physics · High-performance and data-driven computational physics · High-performance and data-driven computational physics
Scientific Computing for Physicists, Ch Graphics processors: kernels, memory and the roofline
Problem
Predict the answers below before running anything, then run the fused diffusion kernel on any GPU available and compare.
((a)) A kernel for a grid of 5000 rows and 3000 columns is launched in blocks of 32×8 threads, 32 along the column index and 8 along the row index, with the chapter's global-index formulas. Predict the number of blocks launched and the number of launched threads whose (i,j) falls outside the grid and so do no work.
((b)) Without benchmarking any specific card, predict whether the fused stencil (intensity I=7/16flopB−1) is memory-bound or compute-bound on any current data-centre GPU, given that binary64 peak-to-bandwidth ratios for such cards lie between about 5 and 12flopB−1. Using the lowest of these ridge points, I∗=5flopB−1 (the case most favourable to being compute-bound), predict the largest percentage of peak the stencil could possibly reach on any such card.
((c)) The centre value of a spreading Gaussian in an early binary64 diffusion run follows un≈1/(1+n/2000), which stays between 0.5 and 1 for the first 2000 steps. It is then stored in float16, whose spacing for values in [0.5,1) is ulp=2−11, so a decrease smaller than half a ulp, 2−12≈2.44×10−4, is not recorded. Predict the step number n at which the stored float16 centre value stops changing, using the continuous rate of change of un as an approximation to the per-step decrement.