[Source 1](https://www.geeksforgeeks.org/contingency-table-in-python/)

[Source 2](https://www.statology.org/contingency-table-python/)

Our HELLO WORD example


```python
import numpy as np
import pandas as pd
```


```python
x = np.array([[1, 0],
              [1, 0],
              [0, 1],
              [0, 1],
              [1, 1],
              [1, 1],
              [0, 0],
              [0, 1]])
b = np.array([0, 1, 0, 1, 0, 1, 0, 0])
# b is a line precursor of y. We transpose it. 
y = b.reshape(-1,1)
```


```python
# trandform arrays to dataframe
df1 = pd.DataFrame(x, columns = ['X1', 'X2'])
df2 = pd.DataFrame(y, columns = ['Y'])
df1
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>X1</th>
      <th>X2</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>5</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>6</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>7</th>
      <td>0</td>
      <td>1</td>
    </tr>
  </tbody>
</table>
</div>




```python
df2
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Y</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>0</td>
    </tr>
    <tr>
      <th>5</th>
      <td>1</td>
    </tr>
    <tr>
      <th>6</th>
      <td>0</td>
    </tr>
    <tr>
      <th>7</th>
      <td>0</td>
    </tr>
  </tbody>
</table>
</div>




```python
# merge dataframe x and y
df1['Y'] = df2
```


```python
df1
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>X1</th>
      <th>X2</th>
      <th>Y</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>0</td>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>1</td>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>5</th>
      <td>1</td>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>6</th>
      <td>0</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>7</th>
      <td>0</td>
      <td>1</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
</div>




```python
# unique line counter for dataframe
dz = df1.groupby(['X1', 'X2', 'Y']).size().reset_index(name='Count')
# Without reset index, something strange is happening
dz
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>X1</th>
      <th>X2</th>
      <th>Y</th>
      <th>Count</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>2</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>5</th>
      <td>1</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>6</th>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
    </tr>
  </tbody>
</table>
</div>




```python
# basic table
pd.crosstab(index=df1['Y'], columns=df1['X2'])
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>X2</th>
      <th>0</th>
      <th>1</th>
    </tr>
    <tr>
      <th>Y</th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>2</td>
      <td>3</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
      <td>2</td>
    </tr>
  </tbody>
</table>
</div>




```python
# Contingency Table with margins
pd.crosstab(index=df1['Y'], columns=df1['X1'], margins=True)
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>X1</th>
      <th>0</th>
      <th>1</th>
      <th>All</th>
    </tr>
    <tr>
      <th>Y</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>3</td>
      <td>2</td>
      <td>5</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
      <td>2</td>
      <td>3</td>
    </tr>
    <tr>
      <th>All</th>
      <td>4</td>
      <td>4</td>
      <td>8</td>
    </tr>
  </tbody>
</table>
</div>




```python
# Contingency Table with margins and normalize all
pd.crosstab(index=df1['Y'], columns=df1['X1'],normalize="all", margins=True)
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>X1</th>
      <th>0</th>
      <th>1</th>
      <th>All</th>
    </tr>
    <tr>
      <th>Y</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0.375</td>
      <td>0.25</td>
      <td>0.625</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0.125</td>
      <td>0.25</td>
      <td>0.375</td>
    </tr>
    <tr>
      <th>All</th>
      <td>0.500</td>
      <td>0.50</td>
      <td>1.000</td>
    </tr>
  </tbody>
</table>
</div>




```python
# Contingency Table with margins and normalize by index
pd.crosstab(index=df1['Y'], columns=df1['X1'],normalize="index", margins=True)
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>X1</th>
      <th>0</th>
      <th>1</th>
    </tr>
    <tr>
      <th>Y</th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0.600000</td>
      <td>0.400000</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0.333333</td>
      <td>0.666667</td>
    </tr>
    <tr>
      <th>All</th>
      <td>0.500000</td>
      <td>0.500000</td>
    </tr>
  </tbody>
</table>
</div>




```python
# Contingency Table with margins and normalize by column
pd.crosstab(index=df1['Y'], columns=df1['X1'],normalize="columns", margins=True)
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>X1</th>
      <th>0</th>
      <th>1</th>
      <th>All</th>
    </tr>
    <tr>
      <th>Y</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0.75</td>
      <td>0.5</td>
      <td>0.625</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0.25</td>
      <td>0.5</td>
      <td>0.375</td>
    </tr>
  </tbody>
</table>
</div>




```python
# Contingency Table with margins and all colums in dataframe
pd.crosstab(index=df1['Y'], columns=[df1['X1'], df1['X2']], margins=True)
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead tr th {
        text-align: left;
    }

    .dataframe thead tr:last-of-type th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr>
      <th>X1</th>
      <th colspan="2" halign="left">0</th>
      <th colspan="2" halign="left">1</th>
      <th>All</th>
    </tr>
    <tr>
      <th>X2</th>
      <th>0</th>
      <th>1</th>
      <th>0</th>
      <th>1</th>
      <th></th>
    </tr>
    <tr>
      <th>Y</th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1</td>
      <td>2</td>
      <td>1</td>
      <td>1</td>
      <td>5</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0</td>
      <td>1</td>
      <td>1</td>
      <td>1</td>
      <td>3</td>
    </tr>
    <tr>
      <th>All</th>
      <td>1</td>
      <td>3</td>
      <td>2</td>
      <td>2</td>
      <td>8</td>
    </tr>
  </tbody>
</table>
</div>




```python
from sympy import *
from fractions import Fraction

print(Fraction(5, 6))
```

    5/6



```python
test = pd.crosstab(index=df1['Y'], columns=df1['X1'], margins=True)
test
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>X1</th>
      <th>0</th>
      <th>1</th>
      <th>All</th>
    </tr>
    <tr>
      <th>Y</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>3</td>
      <td>2</td>
      <td>5</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
      <td>2</td>
      <td>3</td>
    </tr>
    <tr>
      <th>All</th>
      <td>4</td>
      <td>4</td>
      <td>8</td>
    </tr>
  </tbody>
</table>
</div>




```python
test.applymap(lambda x: Fraction(x, 5))
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th>X1</th>
      <th>0</th>
      <th>1</th>
      <th>All</th>
    </tr>
    <tr>
      <th>Y</th>
      <th></th>
      <th></th>
      <th></th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>3/5</td>
      <td>2/5</td>
      <td>1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1/5</td>
      <td>2/5</td>
      <td>3/5</td>
    </tr>
    <tr>
      <th>All</th>
      <td>4/5</td>
      <td>4/5</td>
      <td>8/5</td>
    </tr>
  </tbody>
</table>
</div>




```python

```


```python
def crosstab_to_fraction(test:pd.DataFrame) -> pd.DataFrame:
    # convert crosstab result to numpy
    data_top = list(test.columns) 
    input_data = test.to_numpy()
    # select subarray
    lines, rows = input_data.shape
    subarray = input_data[:(lines-1),:rows-1]
    # define fraction denominator
    denom = input_data[:,-1:]
    denom = denom[:-1]
    # save all digit as a fraction
    input_data = input_data.astype('object')

    for i in range(0, len(subarray)):
        for j in range(0, len(subarray[i])):
            input_data[i,j] = Fraction(subarray[i,j], denom[i][0])

    # convert numpy to dataframe
    df = pd.DataFrame(input_data, columns=data_top)
    return df
    
res = crosstab_to_fraction(test)
res
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>0</th>
      <th>1</th>
      <th>All</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>3/5</td>
      <td>2/5</td>
      <td>5</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1/3</td>
      <td>2/3</td>
      <td>3</td>
    </tr>
    <tr>
      <th>2</th>
      <td>4</td>
      <td>4</td>
      <td>8</td>
    </tr>
  </tbody>
</table>
</div>






```python
from itertools import product

x = np.array([[2, 0],
              [1, 1],
              [0, 2],
              [2, 0],
              [1, 2],
              [0, 0],
              [2, 0]])


def unique_counter(test:pd.DataFrame) -> pd.DataFrame:
    # unique line counter for dataframe
    dz = test.groupby(list(test.columns)).size().reset_index(name='Count')
    return dz


def expand_array(test:np.array) -> np.array:
    # get unique element
    rows, cols = test.shape
    test = np.reshape(test, (cols, rows))
    # calculate product of colums in array
    lines = product(test[0], test[1])
    exploist = np.array(list(set(lines)))
    return exploist

def set_diff2d(A, B):
    """
    from here https://cloudstack.ninja/msunij/hot-to-get-the-set-difference-of-two-2d-numpy-arrays-or-equivalent-of-np-setdiff1d-in-a-2d-array/
    """
    nrows, ncols = A.shape
    dtype={'names':['f{}'.format(i) for i in range(ncols)],
        'formats':ncols * [A.dtype]}
    C = np.setdiff1d(A.view(dtype), B.view(dtype))

    return C.view(A.dtype).reshape(-1, ncols)

print("Original data")
print(x)
print("==================================")
print("Data after expansion ")
data = expand_array(x)
print(data)
print("==================================")
print("Difference between original and expansion")
news = set_diff2d(data, x)
print(news)
print("==================================")


# calculate unique counter of ofiginal data
df1 = pd.DataFrame(x, columns = ['y', 'yHat'])
newdf = unique_counter(df1)

# calculate data to append
adddf = pd.DataFrame(news, columns = ['y', 'yHat'])
adddf['Count'] = 0

frames = [newdf, adddf]
result = pd.concat(frames)
result.reset_index(drop=True, inplace=True)

print("Original data with extension and counting")
result
```

    Original data
    [[2 0]
     [1 1]
     [0 2]
     [2 0]
     [1 2]
     [0 0]
     [2 0]]
    ==================================
    Data after expansion 
    [[0 1]
     [1 2]
     [0 0]
     [2 1]
     [2 0]
     [1 1]
     [2 2]
     [1 0]
     [0 2]]
    ==================================
    Difference between original and expansion
    [[0 1]
     [1 0]
     [2 1]
     [2 2]]
    ==================================
    Original data with extension and counting





<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>y</th>
      <th>yHat</th>
      <th>Count</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>0</td>
      <td>2</td>
      <td>1</td>
    </tr>
    <tr>
      <th>2</th>
      <td>1</td>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>1</td>
      <td>2</td>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>2</td>
      <td>0</td>
      <td>3</td>
    </tr>
    <tr>
      <th>5</th>
      <td>0</td>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>6</th>
      <td>1</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>7</th>
      <td>2</td>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8</th>
      <td>2</td>
      <td>2</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
</div>




```python
x = np.array([[2, 0],
              [1, 1],
              [0, 2],
              [2, 0],
              [1, 2],
              [0, 0],
              [2, 0]])
x.sort(axis=0)
x
```




    array([[0, 0],
           [0, 0],
           [1, 0],
           [1, 0],
           [2, 1],
           [2, 2],
           [2, 2]])




```python
x = np.array([[2, 0],
              [1, 1],
              [0, 2],
              [2, 0],
              [1, 2],
              [0, 0],
              [2, 0]])
fliprX = np.fliplr(x)
fliprX
```




    array([[0, 2],
           [1, 1],
           [2, 0],
           [0, 2],
           [2, 1],
           [0, 0],
           [0, 2]])




```python
fliprX.T
```




    array([[0, 1, 2, 0, 2, 0, 0],
           [2, 1, 0, 2, 1, 0, 2]])




```python
np.lexsort(fliprX.T)
```




    array([5, 2, 1, 4, 0, 3, 6])




```python

print(x[np.lexsort(fliprX.T)], "\nx =\n", x)
# x[np.lexsort(np.fliplr(x).T)]
# x
```

    [[0 0]
     [0 2]
     [1 1]
     [1 2]
     [2 0]
     [2 0]
     [2 0]] 
    x =
     [[2 0]
     [1 1]
     [0 2]
     [2 0]
     [1 2]
     [0 0]
     [2 0]]



```python

```


```python

```

2cc19ab3eb9a093a831bd754e679652335d8b16e
ipynb
Jupyter Notebook
Chapter 2/python/classification metrics/Contingency Table more advanced.ipynb
borisgarbuzov/schulich_data_science_1
fd05ec2bbbe35408f90ebfcf10bb4ca588e7871c
MIT
Chapter 2/python/classification metrics/Contingency Table more advanced.ipynb
borisgarbuzov/schulich_data_science_1
fd05ec2bbbe35408f90ebfcf10bb4ca588e7871c
MIT
Chapter 2/python/classification metrics/Contingency Table more advanced.ipynb
borisgarbuzov/schulich_data_science_1
fd05ec2bbbe35408f90ebfcf10bb4ca588e7871c
MIT
2020-10-25T05:26:50.000Z
2021-07-07T08:25:58.000Z
Qwen/Qwen-72B
1. YES
2. YES