title: How to improve Levenberg-Marquardt's method for polynomial curve fitting?
body: <p>Some weeks ago I started coding the Levenberg-Marquardt algorithm from scratch in Matlab. I'm interested in the polynomial fitting of the data but I haven't been able to achieve the level of accuracy I would like. I'm using a fifth order polynomial after I tried other polynomials and it seemed to be the best option. The algorithm always converges to the same function minimization no matter what improvements I try to implement. So far, I have unsuccessfully added the following features:</p>
  <ul>
  <li>Geodesic acceleration term as a second order correction</li>
  <li>Delayed gratification for updating the damping parameter</li>
  <li>Gain factor to get closer to the Gauss-Newton direction or
  the steepest descent direction depending on the iteration.</li>
  <li>Central differences and forward differences for the finite difference method</li>
  </ul>
  <p>I don't have experience in nonlinear least squares, so I don't know if there is a way to minimize the residual even more or if there isn't more room for improvement with this method. I attach below an image of the behavior of the polynomial for the last iterations. If I run the code for more iterations, the curve ends up not changing from iteration to iteration. As it is observed, there is a good fit from time = 0 to time = 12. But I'm not able to fix the behavior of the function from time = 12 to time = 20. Any help will be very appreciated.</p>
  <p><img src="https://i.stack.imgur.com/OncIf.png" alt="data points and various fitted curves" /></p>