title: Why is cross_validate returning NaN scores?
body: <p>The data is from:
  <a href="https://www.kaggle.com/andieminogue/newspaper-churn" rel="nofollow noreferrer">https://www.kaggle.com/andieminogue/newspaper-churn</a></p>
  <p>The code is shown below df_og is the loaded dataframe, some of the columns have been removed and NaN rows removed, other than that, the data hasn't been transformed:</p>
  <pre><code>df_og2 = df_og.drop(['Address','State','City','SubscriptionID','Zip Code'], axis = 1)
  df_og2 = df_og2.dropna()
  df_og2.reset_index(drop=True, inplace=True)
  
  from sklearn.pipeline import Pipeline
  from sklearn.preprocessing import LabelEncoder, OneHotEncoder, OrdinalEncoder
  from sklearn.compose import ColumnTransformer
  from sklearn.base import BaseEstimator, TransformerMixin
  from category_encoders import ordinal
  from scipy.sparse import csr_matrix
  from sklearn.preprocessing import StandardScaler
  from imblearn.under_sampling import RandomUnderSampler
  from sklearn.model_selection import cross_validate
  from sklearn.model_selection import RepeatedStratifiedKFold
  from sklearn.metrics import classification_report
  from imblearn.pipeline import Pipeline
  from sklearn.inspection import permutation_importance
  from imblearn.under_sampling import RandomUnderSampler
  
  X = df_og2.drop(['Subscriber'], axis=1)
  y = df_og2[['Subscriber']]
  X, y = RandomUnderSampler(random_state=2).fit_resample(X,y)
  
  numeric = df_og2.select_dtypes(include=['int64']).columns.tolist()
  multi_label_cat = ['HH Income', 'Age range', 'weekly fee']
  binary_label_cat = ['Home Ownership', 'dummy for Children']
  onehot_cat = ['County','Deliveryperiod','Ethnicity','Language','Nielsen Prizm', 'Source Channel']
  
  def getScores(model):
      
      HH = df_og2['HH Income'].unique().tolist()
      AR = df_og2['Age range'].unique().tolist()
      WF = df_og2['weekly fee'].unique().tolist()
      HH.sort()
      HH.insert(0,HH.pop())
      AR.sort()
      WF.sort()
      WF.append(WF.pop(5))
      HH_map = {i:j for i,j in zip(HH,[i for i in range(len(HH))])}
      AR_map = {i:j for i,j in zip(AR,[i for i in range(len(AR))])}
      WF_map = {i:j for i,j in zip(WF,[i for i in range(len(WF))])}
      col_map = [{'col': 'HH Income', 'mapping': HH_map}, {'col': 'Age range', 'mapping': AR_map}, 
                                                          {'col': 'weekly fee', 'mapping': WF_map}]
      encode_pipeline = ColumnTransformer([
              ('multi_label_cat', ordinal.OrdinalEncoder(mapping=col_map), multi_label_cat),
              ('binary_label_cat', OrdinalEncoder(), binary_label_cat),
              ('onehot_cat', OneHotEncoder(), onehot_cat),
      ], remainder = 'passthrough', sparse_threshold = 0)
      
      full_pipeline = Pipeline([
              ('encode',encode_pipeline),
              ('scale', StandardScaler()),
              ('model', model)
      ])
  
      cv = RepeatedStratifiedKFold(n_splits=10, n_repeats=2)
      scoring = {'acc': 'accuracy',
                 'f1': 'f1',
                 'prec': 'precision',
                 'rec': 'recall'}
      scores = cross_validate(full_pipeline, X, y, scoring=scoring , cv=cv, n_jobs=-1)
      print(scores)
  
  from sklearn.linear_model import LogisticRegression
  getScores(LogisticRegression())
  </code></pre>
  <p>Sometimes changing the number of folds and repeats to large numbers yields some values (in the same procedure) in the score with most of them being NaN. However after trying to fix it, I can't reproduce that phenomenon anymore.</p>