Logistic regression is trained to classify recorded neural spike-train patterns as…
PHYSICS · Multi-part · Fermi University Physics · Computational Physics · High-performance and data-driven computational physics · High-performance and data-driven computational physics
Scientific Computing for Physicists, Ch Learning from physical data: regression, classification and principal components
Problem
Logistic regression is trained to classify N=5×104 recorded neural spike-train patterns as occurring before or after a stimulus. Full-batch gradient descent reaches a held-out test accuracy of 0.95 after 500 iterations, each iteration evaluating the gradient using every one of the N patterns. Stochastic gradient descent with minibatches of B=100 patterns reaches the same test accuracy of 0.95 after 8 complete passes through the data set, where one pass consists of N/B minibatch steps and touches every pattern exactly once.
((a)) State the total number of individual pattern-gradient evaluations used by each optimiser to reach test accuracy 0.95.
((b)) Compute the factor by which stochastic gradient descent is cheaper than full-batch gradient descent, measured in pattern-gradient evaluations.
((c)) Explain in one sentence why comparing evaluations, rather than iterations, is the fair comparison here.