title: Python XgBoost on unbalanced class
body: <p>I'm currently working on a machine learning project which aim to predict a binary class (negative : 0, positive : 1). The dataset is unbalanced. The proportion of positive value are 0.1%.</p>
  
  <p>I'm running an xgboost model with the gini as my metric of performance.
  The problem is that during the boosting iterations it need a lot of runs to improve the score</p>
  
  <p>Exemple :</p>
  
  <pre><code>[Fold 1/2]
  [0] train-gini:-0.048192    validation-gini:-0.042979
  Multiple eval metrics have been passed: 'validation-gini' will be used for early stopping.
  
  Will train until validation-gini hasn't improved in 200 rounds.
  [10]    train-gini:-0.048192    validation-gini:-0.042979
  [20]    train-gini:-0.048192    validation-gini:-0.042979
  [30]    train-gini:-0.048192    validation-gini:-0.042979
  [40]    train-gini:-0.048192    validation-gini:-0.042979
  [50]    train-gini:-0.048192    validation-gini:-0.042979
  [60]    train-gini:-0.048192    validation-gini:-0.042979
  [70]    train-gini:-0.048192    validation-gini:-0.042979
  [80]    train-gini:-0.048192    validation-gini:-0.042979
  [90]    train-gini:0.197521 validation-gini:0.114222
  [100]   train-gini:0.247692 validation-gini:0.150601
  [110]   train-gini:0.2742   validation-gini:0.169023
  [120]   train-gini:0.278983 validation-gini:0.168095
  [130]   train-gini:0.316636 validation-gini:0.19118
  [140]   train-gini:0.347296 validation-gini:0.191045
  [150]   train-gini:0.368581 validation-gini:0.20094
  [160]   train-gini:0.374773 validation-gini:0.20906
  [170]   train-gini:0.398815 validation-gini:0.215193
  [180]   train-gini:0.426088 validation-gini:0.220467
  [190]   train-gini:0.439271 validation-gini:0.22249
  [200]   train-gini:0.455897 validation-gini:0.226621
  [210]   train-gini:0.469989 validation-gini:0.229512
  [220]   train-gini:0.485784 validation-gini:0.233432
  [230]   train-gini:0.496734 validation-gini:0.23747
  [240]   train-gini:0.503718 validation-gini:0.241804
  [250]   train-gini:0.51102  validation-gini:0.241841
  [260]   train-gini:0.523444 validation-gini:0.244312
  [270]   train-gini:0.530968 validation-gini:0.245467
  [280]   train-gini:0.538703 validation-gini:0.247433
  [290]   train-gini:0.546911 validation-gini:0.244196
  [300]   train-gini:0.553623 validation-gini:0.244161
  [310]   train-gini:0.561385 validation-gini:0.245099
  [320]   train-gini:0.571532 validation-gini:0.244787
  [330]   train-gini:0.578088 validation-gini:0.246146
  [340]   train-gini:0.585054 validation-gini:0.245624
  [350]   train-gini:0.591924 validation-gini:0.245463
  [360]   train-gini:0.596331 validation-gini:0.247517
  [370]   train-gini:0.600661 validation-gini:0.249465
  [380]   train-gini:0.606264 validation-gini:0.249034
  [390]   train-gini:0.611768 validation-gini:0.249182
  [400]   train-gini:0.617176 validation-gini:0.248239
  [410]   train-gini:0.621629 validation-gini:0.249248
  [420]   train-gini:0.626766 validation-gini:0.24975
  [430]   train-gini:0.631587 validation-gini:0.247824
  [440]   train-gini:0.636737 validation-gini:0.246586
  [450]   train-gini:0.641735 validation-gini:0.246552
  [460]   train-gini:0.649765 validation-gini:0.246332
  [470]   train-gini:0.654319 validation-gini:0.243546
  [480]   train-gini:0.659301 validation-gini:0.241965
  [490]   train-gini:0.665632 validation-gini:0.242562
  [500]   train-gini:0.669333 validation-gini:0.241306
  [510]   train-gini:0.673625 validation-gini:0.240314
  [520]   train-gini:0.678935 validation-gini:0.239846
  [530]   train-gini:0.683851 validation-gini:0.240029
  [540]   train-gini:0.685694 validation-gini:0.240691
  [550]   train-gini:0.689285 validation-gini:0.239974
  [560]   train-gini:0.691698 validation-gini:0.239079
  [570]   train-gini:0.694017 validation-gini:0.239407
  Stopping. Best iteration:
  [373]   train-gini:0.60227  validation-gini:0.24996
  </code></pre>
  
  <p>We can see that at round 80 the score for the train and the validation are finally improves. This situation will repeat even if I change the seed  of my split (but the n° of the round at which the score increase will changes).</p>
  
  <p>Does anybody have encounter this kind of problem ?</p>
  
  <p>Cheers,
  Astrus</p>