title: AttributeError: 'MLPClassifier' object has no attribute 'best_loss_'
body: <p>I was experimenting with the warm_start parameter in scikit-learn's MLPClassifier. However, I received an error while running the following code.</p>
  
  <pre><code>    clf = MLPClassifier(solver='adam',
                          hidden_layer_sizes=(128, 128),
                          activation='relu',
                          max_iter = 3,
                          verbose=True,
                          tol= 1e-100,
                          n_iter_no_change=10,
                          early_stopping=True,
                          warm_start=True)
  
      clf.fit(df_feat, df_label)
      clf.fit(df_feat, df_label)
  </code></pre>
  
  <p>Below is my output and error message. The first clf.fit() was able to run to completion, but the second clf.fit() produced an error message after 1 iteration.</p>
  
  <pre><code>Iteration 1, loss = 1.84596208
  Validation score: 0.610841
  Iteration 2, loss = 1.04435735
  Validation score: 0.731758
  Iteration 3, loss = 0.84135025
  Validation score: 0.760945
  C:\Users\LHongRu1\Anaconda3\envs\py36\lib\site-packages\sklearn\neural_network\_multilayer_perceptron.py:571: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (3) reached and th
  e optimization hasn't converged yet.
    % self.max_iter, ConvergenceWarning)
  Iteration 4, loss = 0.77392135
  Traceback (most recent call last):
    File "train_3(to_experiment).py", line 70, in &lt;module&gt;
      main()
    File "train_3(to_experiment).py", line 62, in main
      clf.fit(df_feat, df_label)
    File "C:\Users\LHongRu1\Anaconda3\envs\py36\lib\site-packages\sklearn\neural_network\_multilayer_perceptron.py", line 995, in fit
      hasattr(self, "classes_")))
    File "C:\Users\LHongRu1\Anaconda3\envs\py36\lib\site-packages\sklearn\neural_network\_multilayer_perceptron.py", line 370, in _fit
      intercept_grads, layer_units, incremental)
    File "C:\Users\LHongRu1\Anaconda3\envs\py36\lib\site-packages\sklearn\neural_network\_multilayer_perceptron.py", line 540, in _fit_stochastic
      self._update_no_improvement_count(early_stopping, X_val, y_val)
    File "C:\Users\LHongRu1\Anaconda3\envs\py36\lib\site-packages\sklearn\neural_network\_multilayer_perceptron.py", line 604, in _update_no_improvement_count
      if self.loss_curve_[-1] &gt; self.best_loss_ - self.tol:
  AttributeError: 'MLPClassifier' object has no attribute 'best_loss_'
  </code></pre>
  
  <p>If I change to <code>warm_start = False</code>, both clf.fit() are able to run to completion.</p>