I can only see it breaking in 2 circumstances: 1. the new steps are smaller and you hit the maximum number of iterations (at this point I think its probably safe to say that the start guess is rubbish) 2. the new steps are bigger so it misses a narrow minima. However, I suspect that would use the sqrt(epsilon) which would be a smaller step -> so would still work as long as point 1 does not happen.