title: Fast Exp calculation: possible to improve accuracy without losing too much performance?
body: <p>I am trying out the fast Exp(x) function that previously was described in <a href="https://stackoverflow.com/a/412988/650012">this</a> answer to an SO question on improving calculation speed in C#:</p>
  
  <pre><code>public static double Exp(double x)
  {
    var tmp = (long)(1512775 * x + 1072632447);
    return BitConverter.Int64BitsToDouble(tmp &lt;&lt; 32);
  }
  </code></pre>
  
  <p>The expression is using some IEEE floating point "tricks" and is primarily intended for use in neural sets. The function is approximately 5 times faster than the regular <code>Math.Exp(x)</code> function.  </p>
  
  <p>Unfortunately, the numeric accuracy is only -4% -- +2% relative to the regular <code>Math.Exp(x)</code> function, ideally I would like to have accuracy within at least the sub-percent range. </p>
  
  <p>I have plotted the quotient between the approximate and the regular Exp functions, and as can be seen in the graph the relative difference appears to be repeated with practically constant frequency.</p>
  
  <p><img src="https://i.stack.imgur.com/gUgSI.png" alt="Quotient between fast and regular exp function"></p>
  
  <p>Is it possible to take advantage of this regularity to improve the accuracy of the "fast exp" function further without substantially reducing the calculation speed, or would the computational overhead of an accuracy improvement outweigh the computational gain of the original expression?</p>
  
  <p>(As a side note, I have also tried <a href="https://stackoverflow.com/a/412102/650012">one of the alternative</a> approaches proposed in the same SO question, but this approach does not seem to be computationally efficient in C#, at least not for the general case.)</p>
  
  <p><strong>UPDATE MAY 14</strong></p>
  
  <p>Upon request from @Adriano, I have now performed a very simple benchmark. I have performed 10 million computations using each of the alternative <em>exp</em> functions for floating point values in the range [-100, 100]. Since the range of values I am interested in spans from -20 to 0 I have also explicitly listed the function value at x = -5. Here are the results:</p>
  
  <pre><code>      Math.Exp: 62.525 ms, exp(-5) = 0.00673794699908547
  Empty function: 13.769 ms
       ExpNeural: 14.867 ms, exp(-5) = 0.00675211846828461
      ExpSeries8: 15.121 ms, exp(-5) = 0.00641270968867667
     ExpSeries16: 32.046 ms, exp(-5) = 0.00673666189488182
            exp1: 15.062 ms, exp(-5) = -12.3333325982094
            exp2: 15.090 ms, exp(-5) = 13.708332516253
            exp3: 16.251 ms, exp(-5) = -12.3333325982094
            exp4: 17.924 ms, exp(-5) = 728.368055056781
            exp5: 20.972 ms, exp(-5) = -6.13293614238501
            exp6: 24.212 ms, exp(-5) = 3.55518353166184
            exp7: 29.092 ms, exp(-5) = -1.8271053775984
        exp7 +/-: 38.482 ms, exp(-5) = 0.00695945286970704
  </code></pre>
  
  <p><em>ExpNeural</em> is equivalent to the <em>Exp</em> function specified in the beginning of this text. <em>ExpSeries8</em> is the <a href="https://stackoverflow.com/a/412102/650012">formulation</a> that I originally claimed was not very efficient on .NET; when implementing it exactly like Neil it was actually very fast. <em>ExpSeries16</em> is the analogous formula but with 16 multiplications instead of 8. <em>exp1</em> through <em>exp7</em> are the different functions from Adriano's answer below. The final variant of <em>exp7</em> is a variant where the sign of <em>x</em> is checked; if negative the function returns <code>1/exp(-x)</code> instead.</p>
  
  <p>Unfortunately, neither of the <em>expN</em> functions listed by <a href="https://stackoverflow.com/a/10552567/650012">Adriano</a> are sufficient in the broader negative value range I am considering. The series expansion approach by <a href="https://stackoverflow.com/a/412102/650012">Neil Coffey</a> seems to be more suitable in "my" value range, although it is too rapidly diverging with larger negative <em>x</em>, especially when using "only" 8 multiplications.</p>