<p>Let's just take an example: I'm transposing the floating point problem using a model in base 10 with only 2 significant digits to make it simple, operation result being rounded to nearest.</p>

<p>Say we must sum the 3 numbers <code>9.9 + 8.4 + 1.4</code><br>
The exact result is <code>19.7</code> but we have only two digits so it hould be rounded to <code>20.</code></p>

<p>If we first sum <code>9.9 + 8.4</code> we get <code>18.3</code> which is then rounded to <code>18.</code><br>
We then sum <code>18. + 1.4</code> we get <code>19.4</code> rounded to <code>19.</code>.</p>

<p>If we first sum the last two terms <code>8.4 + 1.4</code> we get <code>9.8</code>, no rounding required yet.<br>
Then <code>9.9 + 9.8</code> we get <code>19.7</code> rounded to <code>20.</code>, a different result.</p>

<p><code>(9.9 + 8.4) + 1.4</code> differs from <code>9.9 + (8.4 + 1.4)</code>, the sum operation is not associative and this is due to intermediate rounding. We could exhibit similar examples with other rounding modes too...</p>

<p>The problem is exactly the same in base 2 with 53 bits significand: intermediate rounding will be causing the non associativity whatever the base or significand length.</p>

<p>To eliminate the problem, you could either sort the numbers so that the order is allways the same, or eliminate the intermediate rounding and keep only the final one, for example with a super accumulator like this <a href="https://arxiv.org/pdf/1505.05571.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1505.05571.pdf</a><br>
...Or just accept to live with an approximate result (up to you to analyze average or worse error and decide if acceptable...).</p>
Result of the sum of random-ordered IEEE 754 double precision floats
<p>Here is a pseudocode of my problem.</p>

<p>I have an array of <strong>IEEE 754</strong> double precision positive numbers.</p>

<p>The array can come in a random order but numbers are always the same, just scrambled in their positions. Also these numbers <strong>can vary in a very wide range</strong> in the valid IEEE range of the <code>double</code> representation.</p>

<p>Once I have the list, I initialize a variable:</p>

<pre><code>double sum_result = 0.0;
</code></pre>

<p>And I accumulate the sum on <code>sum_result</code>, in a loop over the whole array. At each step I do:</p>

<pre><code>sum_result += my_double_array[i]
</code></pre>

<p>Is it guaranteed that, whatever the order of the initial array of <code>double</code>, if the numbers are the same, the printed out sum result will be always the same?</p>