title: How to create/customize your own scorer function in scikit-learn?
body: <p>I am using <a href="http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html" rel="noreferrer">Support Vector Regression</a> as an estimator in <a href="http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.GridSearchCV.html" rel="noreferrer">GridSearchCV</a>. But I want to change the error function: instead of using the default (R-squared: coefficient of determination), I would like to define my own custom error function.</p>
  
  <p>I tried to make one with <code>make_scorer</code>, but it didn't work.</p>
  
  <p>I read the documentation and found that it's possible to create <a href="http://scikit-learn.org/dev/developers/contributing.html#apis-of-scikit-learn-objects" rel="noreferrer">custom estimators</a>, but I don't need to remake the entire estimator - only the error/scoring function.</p>
  
  <p>I think I can do it by defining a callable as a scorer, like it says in the <a href="http://scikit-learn.org/stable/modules/model_evaluation.html#implementing-your-own-scoring-object" rel="noreferrer">docs</a>.</p>
  
  <p>But I don't know how to use an estimator: in my case SVR. Would I have to switch to a classifier (such as SVC)? And how would I use it?</p>
  
  <p>My custom error function is as follows:</p>
  
  <pre><code>def my_custom_loss_func(X_train_scaled, Y_train_scaled):
      error, M = 0, 0
      for i in range(0, len(Y_train_scaled)):
          z = (Y_train_scaled[i] - M)
          if X_train_scaled[i] &gt; M and Y_train_scaled[i] &gt; M and (X_train_scaled[i] - Y_train_scaled[i]) &gt; 0:
              error_i = (abs(Y_train_scaled[i] - X_train_scaled[i]))**(2*np.exp(z))
          if X_train_scaled[i] &gt; M and Y_train_scaled[i] &gt; M and (X_train_scaled[i] - Y_train_scaled[i]) &lt; 0:
              error_i = -(abs((Y_train_scaled[i] - X_train_scaled[i]))**(2*np.exp(z)))
          if X_train_scaled[i] &gt; M and Y_train_scaled[i] &lt; M:
              error_i = -(abs(Y_train_scaled[i] - X_train_scaled[i]))**(2*np.exp(-z))
      error += error_i
      return error
  </code></pre>
  
  <p>The variable <code>M</code> isn't null/zero. I've just set it to zero for simplicity.</p>
  
  <p>Would anyone be able to show an example application of this custom scoring function? Thanks for your help!</p>