title: Using logarithms to normalize a vector to avoid overflow
body: <p><a href="https://stackoverflow.com/questions/2293762/problem-with-arithmetic-using-logarithms-to-avoid-numerical-underflow-take-2">Problem with arithmetic using logarithms to avoid numerical underflow (take 2)</a></p>
  
  <p>Having seen the above and having seen softmax normalization I was trying to normalize a vector while avoiding overflow - </p>
  
  <p>that is if I have an array
  <code>x[1], x[2] x[3], x[4], ... , x[n]</code></p>
  
  <p>the normalized form for me has the sum of squares of elements as 1.0
  and is obtained by dividing each element by
  <code>sqrt(x[1]*x[1]+x[2]*x[2]+...+x[n]*x[n])</code></p>
  
  <p>now the sum of squares can overflow even if the square root is small enough to fit into a floating point variable, so I imagined one could do something like 
  <code>s=(2*log(fabs(x[1]))+2*log(fabs(x[2]))+...+2*log(fabs(x[n])))/2</code></p>
  
  <p>and calculating the elements as </p>
  
  <p><code>exp(log(fabs(x[1]))-s), ..., exp(log(fabs(x[n]))-s</code></p>
  
  <p><strong>BUT</strong></p>
  
  <p>The above is incorrect as log(A+B) is not log(A)+log(B) - now is there a way to do vector normalization that avoids overflow better? </p>