Take the 2-minute tour ×

Converting numbers from double-precision floating-point format to single-precision floating-point format results in loss of precision. What's the algorithm used to achieve this conversion?

Are numbers greater than 3.4028234e+38 or lesser than -3.4028234e+38 simply reduced to the respective limits? I feel that the conversion process is a bit more involved than this but I couldn't find documentation for it.

share|improve this question
add comment

1 Answer

up vote 4 down vote accepted

The most common floating-point formats are the binary floating-point formats specified in the IEEE 754 standard. I will answer your question for these formats. There are also decimal floating-point formats in the new (2008) version of the standard, and there are formats other than the IEEE 754 standard, but the 754 binary formats are by far the most common. Some information about rounding, and links to the standard, are in this Wikipedia page.

Converting double precision to single precision is treated the same as rounding the result of any operation. (E.g., an addition, multiplication, or square root has an exact mathematical value, and that value is rounded according to the rules to produce the result returned from the operation. For purposes of conversion, the input value is the exact mathematical value, and it is rounded.)

Generally, the computing environment has some default rounding mode. (Various programming languages may provide ways to change the default rounding mode or to specify it particularly with each operation.) The default rounding mode is commonly round-to-nearest. Others are round-toward-zero, round-toward-positive-infinity (upward), and round-toward-negative-infinity (downward).

In round-to-nearest mode, the representable number nearest the exact value is returned. If there is a tie, then the number with the even low bit (in its fraction or significand) is returned. For this purpose, infinity effectively acts as if it were the next value in the pattern of finite numbers. In single-precision, the greatest finite numbers are 0x1.fffff8p127, 0x1.fffffap127, 0x1.fffffcp127, and 0x1.fffffep127. (There are 24 bits in the single-precision significand, so a step in that bit is a step of 2 in that last hexadecimal digit.) For rounding purposes, infinity acts as if it were at 0x2p128. So, if the exact result is closer to 0x1.fffffep127 (thus, less than 0x1.ffffffp127), it is rounded to 0x1.fffffep127. If it is greater than or equal to 0x1.ffffffp127, it is rounded to infinity. The situation for negative infinity is symmetric.

In round-toward-infinity mode, the nearest representable value that is greater than or equal to the exact value is returned. So any value above 0x1.fffffep127 rounds to infinity. Round-toward-negative-infinity returns the nearest representable value that is less than or equal to the exact vaue. Round-toward-zero returns the nearest representable value in the direction toward zero.

The IEEE 754 standard only specifies the result; it does not specify the algorithm. The method used to achieve the rounding is up to each implementation.

share|improve this answer
A note on hexadecimal format: 0x<fraction>p<exponent> stands for the result of interpreting <fraction> as a hexadecimal numeral (including the point to mark the unit position) and multiplying it by two raised to the power of <exponent>. This makes it easy to see the representation of the floating-point value, and to see things like the fact that 0x1.fffffep127 is the greatest representable finite value, because all 24 bits in its significand are on and it has the maximum exponent, 127. –  Eric Postpischil Aug 2 '12 at 12:06
Note on the note: the <exponent> is a decimal numeral. –  Eric Postpischil Aug 2 '12 at 13:11
add comment

Your Answer


