A calibrated dial on an instrument reads a fraction of full scale (a dimensionless ratio).
PHYSICS · Multi-part · Fermi University Physics · Mathematical Methods for Physics · Numerical and computational mathematics · Numerical and computational mathematics
Computational Physics, Ch Numbers a machine can hold, and the two errors that follow
Problem
A calibrated dial on an instrument reads a fraction 0.625 of full scale (a dimensionless ratio).
((a)) Write 0.625 as a binary fraction, showing the place values used, and state how many bits after the binary point are needed.
((b)) State whether binary64 stores this value exactly, giving the reason in terms of the number of stored significand bits.