title: Choosing optimal training weights in grid search
body: <p>Can optimal <code>sample_weight</code> values be selected among candidates the same way we select hyperparameters in a grid search?</p>
  
  <p>It's possible that some training data should be weighted differently (e.g. based on group membership, where some training data is trusted more), and the choice of several possible weights would ideally be found as part of a grid search. Different <code>sample_weight</code> values would be tested to see how the model performs against a gold standard holdout test set where the results are maximally trusted.</p>
  
  <p>Training weights can be utilized in <a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV" rel="nofollow noreferrer">GridSearchCV</a> or <a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.RandomizedSearchCV.html" rel="nofollow noreferrer">RandomizedSearchCV</a> using the <code>scoring</code> parameter with the <code>make_scorer</code> function (<a href="https://stackoverflow.com/questions/49581104/sklearn-gridsearchcv-not-using-sample-weight-in-score-function">example</a>). However, it's not clear how the best <code>sample_weight</code> vector among several candidates could be selected as part of a sklearn pipeline. Is this possible?</p>