title: ConvergenceWarning: lbfgs failed to converge (status=1): STOP: TOTAL NO. of ITERATIONS REACHED LIMIT
body: <p>I have a dataset consisting of both numeric and categorical data and I want to predict adverse outcomes for patients based on their medical characteristics. I defined a prediction pipeline for my dataset like so:</p>
  <pre class="lang-py prettyprint-override"><code>X = dataset.drop(columns=['target'])
  y = dataset['target']
  
  # define categorical and numeric transformers
  numeric_transformer = Pipeline(steps=[
      ('knnImputer', KNNImputer(n_neighbors=2, weights=&quot;uniform&quot;)),
      ('scaler', StandardScaler())])
  
  categorical_transformer = Pipeline(steps=[
      ('imputer', SimpleImputer(strategy='constant', fill_value='missing')),
      ('onehot', OneHotEncoder(handle_unknown='ignore'))])
  
  #  dispatch object columns to the categorical_transformer and remaining columns to numerical_transformer
  preprocessor = ColumnTransformer(transformers=[
      ('num', numeric_transformer, selector(dtype_exclude=&quot;object&quot;)),
      ('cat', categorical_transformer, selector(dtype_include=&quot;object&quot;))
  ])
  
  # Append classifier to preprocessing pipeline.
  # Now we have a full prediction pipeline.
  clf = Pipeline(steps=[('preprocessor', preprocessor),
                        ('classifier', LogisticRegression())])
  
  X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
  
  clf.fit(X_train, y_train)
  print(&quot;model score: %.3f&quot; % clf.score(X_test, y_test))
  </code></pre>
  <p>However, when running this code, I get the following warning message:</p>
  <pre><code>ConvergenceWarning: lbfgs failed to converge (status=1):
  STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.
  Increase the number of iterations (max_iter) or scale the data as shown in:
      https://scikit-learn.org/stable/modules/preprocessing.html
  Please also refer to the documentation for alternative solver options:
      https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
    extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)
  
      model score: 0.988
  </code></pre>
  <p>Can someone explain to me what this warning means? I am new to machine learning so am a little lost as to what I can do to improve the prediction model. As you can see from the numeric_transformer, I scaled the data through standardisation. I am also confused as to how the model score is quite high and whether this is a good or bad thing.</p>