title: Comparing categorical variables between columns in pandas.DataFrame
body: <p><strong>How do I make this comparison using the set categorical rules instead of the lexicon order rule?</strong></p>
  <p>Given the dataset:</p>
  <pre><code>df = pd.DataFrame({
      'NUMBER':[12, 26, 16, 34, 38, 1, 26, 8],
      'SHIRT_SIZE':['S', 'M', 'XL', 'L', 'S', 'M', 'L', 'XL'],
      'SHIRT_SIZE2':['M', 'S', 'L', 'XL', 'M', 'L', 'XL', 'S']
  })
  from pandas.api.types import CategoricalDtype
  c_dtype = CategoricalDtype(categories = [&quot;S&quot;,&quot;M&quot;,&quot;L&quot;,&quot;XL&quot;],ordered = True)
  df['SHIRT_SIZE'] = df['SHIRT_SIZE'].astype(c_dtype)
  df['SHIRT_SIZE2'] = df['SHIRT_SIZE2'].astype(c_dtype)
  </code></pre>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th style="text-align: right;"></th>
  <th style="text-align: right;">NUMBER</th>
  <th style="text-align: left;">SHIRT_SIZE</th>
  <th style="text-align: left;">SHIRT_SIZE2</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td style="text-align: right;">0</td>
  <td style="text-align: right;">12</td>
  <td style="text-align: left;">S</td>
  <td style="text-align: left;">M</td>
  </tr>
  <tr>
  <td style="text-align: right;">1</td>
  <td style="text-align: right;">26</td>
  <td style="text-align: left;">M</td>
  <td style="text-align: left;">S</td>
  </tr>
  <tr>
  <td style="text-align: right;">2</td>
  <td style="text-align: right;">16</td>
  <td style="text-align: left;">XL</td>
  <td style="text-align: left;">L</td>
  </tr>
  <tr>
  <td style="text-align: right;">3</td>
  <td style="text-align: right;">34</td>
  <td style="text-align: left;">L</td>
  <td style="text-align: left;">XL</td>
  </tr>
  <tr>
  <td style="text-align: right;">4</td>
  <td style="text-align: right;">38</td>
  <td style="text-align: left;">S</td>
  <td style="text-align: left;">M</td>
  </tr>
  <tr>
  <td style="text-align: right;">5</td>
  <td style="text-align: right;">1</td>
  <td style="text-align: left;">M</td>
  <td style="text-align: left;">L</td>
  </tr>
  <tr>
  <td style="text-align: right;">6</td>
  <td style="text-align: right;">26</td>
  <td style="text-align: left;">L</td>
  <td style="text-align: left;">XL</td>
  </tr>
  <tr>
  <td style="text-align: right;">7</td>
  <td style="text-align: right;">8</td>
  <td style="text-align: left;">XL</td>
  <td style="text-align: left;">S</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>The <code>dtype</code> of <code>'SHIRT_SIZE'</code> and <code>'SHIRT_SIZE2'</code> is <code>Categories (4, object): ['S' &lt; 'M' &lt; 'L' &lt; 'XL']</code>
  I would like to compare the shirt sizes between the two columns <code>'SHIRT_SIZE'</code> and <code>'SHIRT_SIZE2'</code></p>
  <p>I attempted:</p>
  <pre><code>def compare_size(row):
      if (row['SHIRT_SIZE'] &lt; row['SHIRT_SIZE2']):
          return 'SMALLER'
      elif (row['SHIRT_SIZE'] &gt; row['SHIRT_SIZE2']):
          return 'LARGER'
      else:
          return 'SAME'
  
  df['COMPARE_SIZE'] = df.apply(lambda row: compare_size(row), axis=1)
  </code></pre>
  <p>Resulting in:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th style="text-align: right;"></th>
  <th style="text-align: right;">NUMBER</th>
  <th style="text-align: left;">SHIRT_SIZE</th>
  <th style="text-align: left;">SHIRT_SIZE2</th>
  <th style="text-align: left;">COMPARE_SIZE</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td style="text-align: right;">0</td>
  <td style="text-align: right;">12</td>
  <td style="text-align: left;">S</td>
  <td style="text-align: left;">M</td>
  <td style="text-align: left;">LARGER</td>
  </tr>
  <tr>
  <td style="text-align: right;">1</td>
  <td style="text-align: right;">26</td>
  <td style="text-align: left;">M</td>
  <td style="text-align: left;">S</td>
  <td style="text-align: left;">SMALLER</td>
  </tr>
  <tr>
  <td style="text-align: right;">2</td>
  <td style="text-align: right;">16</td>
  <td style="text-align: left;">XL</td>
  <td style="text-align: left;">L</td>
  <td style="text-align: left;">LARGER</td>
  </tr>
  <tr>
  <td style="text-align: right;">3</td>
  <td style="text-align: right;">34</td>
  <td style="text-align: left;">L</td>
  <td style="text-align: left;">XL</td>
  <td style="text-align: left;">SMALLER</td>
  </tr>
  <tr>
  <td style="text-align: right;">4</td>
  <td style="text-align: right;">38</td>
  <td style="text-align: left;">S</td>
  <td style="text-align: left;">M</td>
  <td style="text-align: left;">LARGER</td>
  </tr>
  <tr>
  <td style="text-align: right;">5</td>
  <td style="text-align: right;">1</td>
  <td style="text-align: left;">M</td>
  <td style="text-align: left;">L</td>
  <td style="text-align: left;">LARGER</td>
  </tr>
  <tr>
  <td style="text-align: right;">6</td>
  <td style="text-align: right;">26</td>
  <td style="text-align: left;">L</td>
  <td style="text-align: left;">XL</td>
  <td style="text-align: left;">SMALLER</td>
  </tr>
  <tr>
  <td style="text-align: right;">7</td>
  <td style="text-align: right;">8</td>
  <td style="text-align: left;">XL</td>
  <td style="text-align: left;">S</td>
  <td style="text-align: left;">LARGER</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p><strong>Notice that there are some rows <em>e.g. row 0 where 'S' -&gt; 'M' and row 1 where 'M' -&gt; 'S'</em> do not follow the order of our categorical <code>dtype</code> rules</strong></p>
  <p>Logically, the interpretation is: &quot;SHIRT_SIZE is &lt;COMPARE_SIZE&gt; THAN SHIRT_SIZE2&quot;</p>
  <p>I am guessing that the lexicon order of the string is the underlying rule used to compare these shirt sizes rather than the categorical rule we have set where <code>Categories (4, object): ['S' &lt; 'M' &lt; 'L' &lt; 'XL']</code>.</p>
  <p>I hope to compare the shirt sizes according to the categorical order.</p>