We show that gradient descent on an unregularized logistic regression
 problem, for almost all separable datasets, converges to the same direction as the max-margin solution. The result generalizes also to other monotone decreasing loss functions with an infimum at infinity, and we also discuss a multi-class generalizations to the cross entropy loss. Furthermore,
 we show this convergence is very slow, and only logarithmic in the
 convergence of the loss itself. This can help explain the benefit
 of continuing to optimize the logistic or cross-entropy loss even
 after the training error is zero and the training loss is extremely
 small, and, as we show, even if the validation loss increases. Our
 methodology can also aid in understanding implicit regularization
 in more complex models and with other optimization methods.
The normalized solution of gradient descent on logistic regression (or a similarly decaying loss) slowly converges to the L2 max margin solution on separable data. The paper offers a formal proof that gradient descent on the logistic loss converges very slowly to the hard SVM solution in the case where the data are linearly separable.  This paper focuses on characterising the behaviour of log loss minimisation on linearly separable data, and shows that log-loss, minimised with gradient descent, leads to convergence to the max-margin solution.