teach-ict.com logo

THE education site for computer science and ICT

2. Going beyond the limit

Some calculations might result in a number too large or too small to fit into the memory allotment for a single floating point number. If this happens, the CPU will flag that an error occurred. Well-written programs will take note of this error warning and do something about it. Badly-written programs will simply crash.

If the range goes beyond the most positive or most negative largest number, this is called an 'overflow'. Trying to divide a number by zero is certain to cause an overflow.

If the number is too tiny to be represented, this is called an 'underflow'. For example dividing a number by a very large number may cause an underflow.

Overflow and Underflow

overflow underflow

Truncation

Some fractions, when converted to decimal, have infinite digits. For example $\frac 1 3$ as a decimal is 0.333 recurring.

But floating point can only represent so many decimal digits. The solution is truncation, which is simply discarding all values after a certain point. This lets the number fit into the available storage space, but can cause issues with calculations that rely on precision that is no longer there.

Rounding

If you have a number such as 3.456 then if you have to round this to three significant digits, the number becomes 3.46. This introduces a small error called the 'rounding error'. Certain calculations may result in rounding errors accumulating to be large enough to be significant. If this is the case, the programmer has to take this into account as they code the calculations.

Note that the CPU will not flag any error when truncating or rounding. All a programmer can do is carefully assess whether his calculations will lead to significant rounding and truncation errors.

Challenge see if you can find out one extra fact on this topic that we haven't already told you

Click on this link: Rounding errors