PHYSICS · Multi-part · Fermi University Physics · Mathematical Methods for Physics · Numerical and computational mathematics · Numerical and computational mathematics
Computational Physics, Ch Numbers a machine can hold, and the two errors that follow
Problem
A current sensor reports a reading of 0.1A.
((a)) Write 0.1 in binary as a repeating fraction, giving the first twelve bits after the point and marking the repeating block.
((b)) Explain, in terms of the number of stored significand bits, why binary64 cannot store 0.1 exactly.
((c)) The exact stored value is quoted as 0.1000000000000000055511151231257827…. State the sign and the order of magnitude of the error this storage step alone introduces, relative to the true value 0.1.