title: Steepest descent to find the solution to a linear system with a Hilbert matrix
body: <p>I am using the method of <strong>steepest descent</strong> to figure out the solution to a linear system with a 5x5 Hilbert matrix. I believe the code is fine in the regard that it gives me the right answer. </p>
  
  <p>My problem is that:</p>
  
  <ol>
  <li><p>I think it is taking too many iterations to get to the right answer. I believe I may have missed something in the algorithm but I'm not sure what at this point. </p></li>
  <li><p>I am not sure if this is the most effective way to implement the algorithm and additionally, it is a little confusing on which "tol" to pick.</p></li>
  </ol>
  
  <p>Any insight on these would be appreciated (especially 1.). Thanks!</p>
  
  <pre><code>% Method of Steepest Descent with tol 10^-6
  h = hilb(5);                            %Hilbert 5x5 matrix
  b = [1;1;1;1;1];                        %solution matrix
  solution = zeros(d,1);                  %Initialization 
  residual = h*solution - b;
  tol = 10^(-6)
  count = 0; 
  
  while residual'*residual &gt; tol;
      roe = (residual'*residual)/(residual'*h*residual);
      solution = solution - roe*residual;
      residual = h*solution - b;
      count = count + 1;
  end
  
  count 
  solution 
  
  
  %Method of Steepest Descent with tol 10^-12
  solution = zeros(d,1);
  residual = h*solution - b;
  tol = 10^(-12)
  count = 0; 
  
  while residual'*residual &gt; tol;
      roe = (residual'*residual)/(residual'*h*residual);
      solution = solution - roe*residual;
      residual = residual - roe*h*residual;
      count = count + 1;
  end
  
  count
  solution
  
  %another_solution = invhilb(5)*b           %Check for solution
  </code></pre>