title: Understanding cost-sensitive evaluation in Weka (cost matrix)
body: <p>I am using Weka 3.7.1</p>
  
  <p>I am attempting to analyze sport predictions for baseball using weka. I would like to use a cost matrix because the cost of different outcomes is not the same at a sportsbook where I gamble on the game. My data set is simple: it is a set of predictions with a nominal class {WIN,LOSS}. For this question, the attributes are not a concern.</p>
  
  <p>In the WEKA Explorer, after loading my arff file I can setup a cost matrix from </p>
  
  <blockquote>
    <p>Classify->More Options...->Cost-sensitive evaluation->Set...->There is
    a 2x2 grid that appears in the weka cost-sensitive evaluation after I
    set the classes == 2</p>
  </blockquote>
  
  <p>Here are the values I would like to enter in to the cost matrix:</p>
  
  <ul>
  <li>Correctly classified as loss, cost is 0 (I did not wager)</li>
  <li>Incorrectly classified as loss, cost is 0 (I did not wager)</li>
  <li>Correctly classified as win, cost is -.909 (I won .909 dollars)</li>
  <li>Incorrectly classified as win, cost is 1.0 (I lost a dollar)</li>
  </ul>
  
  <p>Observe that to stay true with it being a 'cost matrix' that I set my profit to a negative value (which is the opposite of cost, it is a profit); and that I set the loss to a positive number (because it cost me when I lost the wager).</p>
  
  <p>After some reflection I decided to use the following grid, and I have not a clue if I did this correctly, please let me know if I did this correctly:</p>
  
  <pre><code> - a   b       &lt;---- "classified as"
   - 0   1.0     a=LOSS
   - 0   -.909   b=WIN
  </code></pre>
  
  <p>And here is my probably faulty logic: (col, row)</p>
  
  <pre><code>(0,0) of grid=0: classified as LOSS, and was LOSS
  (0,1) of grid=0: classified as LOSS, but was WIN
  (1,0) of grid=1.0; classified as WIN, but was LOSS
  (1,1) of grid=.909; classified as WIN, was WIN
  </code></pre>
  
  <p>and of course (0,0) and (0,1) represent the classifier predicting a LOSS and in these cases I do not wager, and therefore there is no cost.</p>
  
  <p>on the other hand (1,0) and (1,1) represent the classifier predicting a WIN and in these cases I place a wager, and therefore there is a cost associated.</p>
  
  <p>One other item that is of great confusion: after I setup the cost matrix and execute a classifier, the output report contains the following:</p>
  
  <p>Evaluation cost matrix:</p>
  
  <pre><code> 0    1   
   0    0.91   &lt;--- notice that this is not a negative value!
  </code></pre>
  
  <p>And as you can see, in the report (1,1) is 0.91 when I had actually entered -.909. I did find another post about this topic, but it does not explain why the negative value became positive.</p>
  
  <p>Thank you in advance. Please note that these are answerable questions; however, if you want to provide some guidance I would be very happy as I am a newbie still trying to build a framework of understanding.</p>