title: matlab: exp(X) gives inf even though exp(max(X)) does not
body: <p><em>[Edit: The whole thing has a very simple solution: the matrix used the single datatype instead of the default double]</em></p>
  
  <p>I have just noticed a somewhat peculiar (I think) behaviour in matlab and wonder what's causing it. I have a 10000x500 matrix M with values ranging from </p>
  
  <pre><code>min(min(M)) = -226.9723 to 
  max(max(M)) =  92.8173
  </code></pre>
  
  <p>and</p>
  
  <pre><code>exp(-227) =  2.6011e-99
  exp(93) = 2.4512e+40
  </code></pre>
  
  <p>but if I exp the entire matrix, this matrix has inf values:</p>
  
  <pre><code>ii = isinf(exp(M));
  sum(sum(ii))
  ans =
       2
  </code></pre>
  
  <p>How does Matlab store the values in the matrix so that operations on individual elements  can give a different result than when doing the same operation on the matrix itself?</p>
  
  <p>I.e. </p>
  
  <pre><code>expM = exp(M);
  exp(M(1)) == expM(1) ; %can be false, which I find surprising
  </code></pre>
  
  <p>I know I have to change the algorithm anyway as the high exponents will give inexact results even if I can avoid inf values. It happens in a formula for a artificial neural network calculation like: </p>
  
  <pre><code> sum(log(1+exp(ones(numcases,1)*b_h + data*w_vh)),2);
  </code></pre>
  
  <p>so my plan is to split this up into two cases, first where the exponent is small I do the calculation as above, for high values I approximate </p>
  
  <pre><code>log(1+exp(ones(numcases,1)*b_h + data*w_vh)
  </code></pre>
  
  <p>with </p>
  
  <pre><code>ones(numcases,1)*b_h + data*w_vh
  </code></pre>
  
  <p>Does that sound reasonable? My reasoning of course is that </p>
  
  <pre><code>log(1+exp(x)) ≈ log(exp(x)) ≈ x, for large x
  </code></pre>
  
  <p>btw: is there a better way to get the maximum element of a matrix other than doing max twice as in max(max(M))?</p>