The complete gradient of the Lennard-Jones energy of two atomic clusters is computed…
PHYSICS · Multi-part · Fermi University Physics · Computational Physics · High-performance and data-driven computational physics · High-performance and data-driven computational physics
Scientific Computing for Physicists, Ch Neural networks on physical data: backpropagation, convolution and symmetry
Problem
The complete gradient of the Lennard-Jones energy of two atomic clusters is computed two ways by the same hand-written scalar classes: forward-mode dual numbers (one pass per input coordinate) and a reverse-mode tape (one forward sweep plus one backward sweep). Each is timed against the same scalar energy evaluation run without any derivative tracking. Every time is the median of seven repeats after a warm-up run, on the chapter's Apple M4 laptop. Timings: for 13 atoms (39 inputs), plain evaluation 31.3μs, forward-mode gradient 8.89ms, reverse-mode gradient 1.60ms. For 38 atoms (114 inputs), plain evaluation 279μs, forward-mode gradient 230ms, reverse-mode gradient 22.3ms.
((a)) Compute the ratio of forward-mode time to reverse-mode time at each size.
((b)) Express the forward-mode cost at each size as a number of plain evaluations per input coordinate, and the reverse-mode cost as a number of plain evaluations.
((c)) Compute how many times larger the 38-atom ratio is than the 13-atom ratio, compare it with the growth in the number of inputs, and use (b) to explain the difference.
((d)) The reverse-mode time itself grew about fourteenfold between the two clusters. Explain why this does not contradict the statement that reverse mode's cost carries no factor for the number of inputs.