title: CatBoost precision imbalanced classes
body: <p>I use a CatBoostClassifier and my classes are highly imbalanced. I applied a scale_pos_weight parameter to account for that. While training with an evaluation dataset (test) CatBoost shows a high precision on test. However, when I make predictions on test using a predict method, I only get a low precision score (calculated using the sklearn.metrics).</p>
  
  <p>I think this might be related to class weights that I applied. However, I don't quite understand how a precision score is affected by this.</p>
  
  <pre><code>params = frozendict({
      'task_type': 'CPU',
      'loss_function': 'Logloss',
      'eval_metric': 'F1', 
      'custom_metric': ['F1', 'Precision', 'Recall'],
      'iterations': 100,
      'random_seed': 20190128,
      'scale_pos_weight': 56.88657244809081,
      'learning_rate': 0.5412829495147387, 
      'depth': 7, 
      'l2_leaf_reg': 9.526905230698302
  })
  
  from catboost import CatBoostClassifier
  model = cb.CatBoostClassifier(**params)
  model.fit(
      X_train, y_train,
      cat_features=np.where(X_train.dtypes == np.object)[0],
      eval_set=(X_test, y_test),
      verbose=False,
      plot=True
  )
  
  model.get_best_score()
  {'learn': {'Recall': 0.9243007537531925,
    'Logloss': 0.15892360013680026,
    'F1': 0.9416723809244181,
    'Precision': 0.9640191600545249},
   'validation_0': {'Recall': 0.914252301192093,
    'Logloss': 0.1714387314107052,
    'F1': 0.9357892623978286,
    'Precision': 0.9642642597943112}}
  
  y_test_pred = model.predict(data=X_test)
  
  from sklearn.metrics import balanced_accuracy_score, recall_score, precision_score, f1_score
  print('Balanced accuracy: {:.2f}'.format(balanced_accuracy_score(y_test, y_test_pred)))
  print('Precision: {:.2f}'.format(precision_score(y_test, y_test_pred)))
  print('Recall: {:.2f}'.format(recall_score(y_test, y_test_pred)))
  print('F1: {:.2f}'.format(f1_score(y_test, y_test_pred)))
  
  Balanced accuracy: 0.94
  Precision: 0.29
  Recall: 0.91
  F1: 0.44
  </code></pre>
  
  <p>I expected to get the same precision as CatBoost show while training, however, it's not so. What am I doing wrong?</p>