title: xgboost: what data to use in the watchlist
body: <p>When running xgboost to optimize its hyperparameters, is it right that I use my dev (validation) set in the watchlist AND to draw predictions from to check its performance? I also had a question about following this same process when using hyperopt to search hyperparameters. </p>
  
  <p>I am wondering if this is right because it is impressed that we never use the test set for anything other than to make a final prediction at the end. Thanks for your time.</p>
  
  <pre><code># prepare xgboost data
  xg_train = xgb.DMatrix(X_train, label=y_train)
  xg_test = xgb.DMatrix(X_test, label=y_test)
  xg_dev = xgb.DMatrix(X_dev, label=y_dev)
  
  # validations set to watch performance
  watchlist = [(xg_train,'train'), (xg_dev,'eval')]
  
  # validation results
  evals_results = {}
  
  # boost
  bst = xgb.train(params=param,
                  dtrain=xg_train,
                  num_boost_round=5000,
                  evals=watchlist,
                  evals_result=evals_results,
                  early_stopping_rounds=100)
  
  # generate scores
  predicted_y = bst.predict(xg_dev, ntree_limit=bst.best_iteration)
  current_score = f1_score(y_dev, np.round(predicted_y), average='weighted')
  </code></pre>