title: MLP learning rate optimization with GridSearchCV
body: <p>I'm trying to tune the hyperparameters of MLP classifier using GridSearchCV but facing the following issue:</p>
  <pre><code>/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py:536: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan.
  
  
  Details: 
  
  ValueError: learning rate 0.01 is not supported. 
  
    FitFailedWarning)
  
  /usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py:536: FitFailedWarning: Estimator fit failed. The score on this train-test partition for these parameters will be set to nan.
  
  Details: 
  
  ValueError: learning rate 0.02 is not supported
  
  ........
  </code></pre>
  <p>Code:</p>
  <pre><code>clf = MLPClassifier()
  params= {
     'hidden_layer_sizes': hidden_layers_generator(X,np.arange(1,17,1)),
     'solver': ['sgd'],
     'momentum': np.arange(0.1,1.1,0.1),
     'learning_rate': np.arange(0.01,1.01,0.01),
     'max_iter': np.arange(100,2100,100)}
  grid = GridSearchCV(clf, params, cv=10, scoring='accuracy')
  grid.fit(X, y)
  grid_mean_scores = grid.cv_results_['mean_test_score']
  pd.DataFrame(grid.cv_results_)[['mean_test_score', 'std_test_score', 'params']]
  </code></pre>
  <p>The code of hidden_layers_generator is as follows:</p>
  <pre><code>from itertools import combinations_with_replacement
  def hidden_layers_generator(df,hidden_layers):
    hd_sizes = []
    for l in range(1, len(hidden_layers)):
      comb = combinations_with_replacement(np.arange(1,len(df.columns),10), l)
      hd_sizes.append(list(comb))
    return hd_sizes
  </code></pre>
  <p>Here's a small snippet of X and y dataframes:</p>
  <pre><code>X.head()
      sl      sw      pl      pw
  0   5.1     3.5     1.4     0.2
  1   4.9     3.0     1.4     0.2
  2   4.7     3.2     1.3     0.2
  3   4.6     3.1     1.5     0.2
  4   5.0     3.6     1.4     0.2
  </code></pre>
  <pre><code>y.head()
  0    0
  1    1
  2    1
  3    0
  4    0
  </code></pre>