John Gustafson

An Energy-Efficient and Massively Parallel Approach to Valid Numerics

John L. Gustafson
Ceranovo Inc., USA 

Computer hardware manufacturers have shown little interest in improving the validity of their numerics, and have accepted the hazards of floating point arithmetic. However, they have a very strong and growing interest in energy efficiency as they compete for the battery life of mobile devices as well as the amount of capability they can achieve in a large data center with strict megawatt-level power budgets. They are also concerned that multicore parallelism is growing much faster than algorithms can exploit. It may be possible to persuade manufacturers to embrace valid numerics not because of validity concerns but because having valid numerics can solve energy/power and parallelism concerns. 

A new universal number format, the “unum”, allows valid (provably bounded) arithmetic with about half the bits of conventional IEEE floating point on average; the bit reduction saves energy and power by reducing the bandwidth and storage demands on the processor. It also relieves the programmer from being an expert in numerical analysis, by automatically tracking the exact or ULP-wide inexact state of each value and by promoting and demoting dynamic range and fraction precision automatically. Unums pass the difficult validity tests published by Kahan, Rump, and Bailey. When used to solve physics problems, such as nonlinear ordinary differential equations, they also expose a new source of massive parallelism in what were thought to be highly serial time-dependent problems. Furthermore, because unum arithmetic obeys associative and distributive laws, parallelization of algorithms does not produce changes in the answer from rounding errors that unsophisticated programmers mistake for logic errors; this further facilitates the use of parallel architectures.

The new format and the new algorithms that go with it have the potential to completely disrupt the way computers are designed and used for technical computing.