# Gender classification model through naive bayes

## When to use naive bayes
    - Naive bayes is used for strings and numbers(categorically)
    - it can be used for classification
    - Can not use for regression
    - best for binary classification.
    - It can be used for multiclass classification also in the same way of binary classification.In other words, you compute the probability of each class label in the usual way, then pick the class with the largest probability.
    - The number of parameters used is independent of the size of training data
    - This model is mostly used for large datasets.
    - It gives fast predictions.
    - Naïve Bayes can also be an extremely good text classifier as it performs well, such as in the spam ham dataset
    - Naive Bayes is linear classifier which might not be suitable to classes that are not linearly separated in a dataset

## Assumptions
    - It assumes that features of a dataset are completely independent of each other. But it is generally not true that is why we also call it a ‘naive’ algorithm
    


```python
import pandas as pd
import numpy as np
import pprint
```

#### Reading dataset


```python
df = pd.read_csv('./dataset/dataset.csv')
df.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Favorite Color</th>
      <th>Favorite Music Genre</th>
      <th>Favorite Beverage</th>
      <th>Favorite Soft Drink</th>
      <th>Gender</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>Cool</td>
      <td>Rock</td>
      <td>Vodka</td>
      <td>7UP/Sprite</td>
      <td>F</td>
    </tr>
    <tr>
      <th>1</th>
      <td>Neutral</td>
      <td>Hip hop</td>
      <td>Vodka</td>
      <td>Coca Cola/Pepsi</td>
      <td>F</td>
    </tr>
    <tr>
      <th>2</th>
      <td>Warm</td>
      <td>Rock</td>
      <td>Wine</td>
      <td>Coca Cola/Pepsi</td>
      <td>F</td>
    </tr>
    <tr>
      <th>3</th>
      <td>Warm</td>
      <td>Folk/Traditional</td>
      <td>Whiskey</td>
      <td>Fanta</td>
      <td>F</td>
    </tr>
    <tr>
      <th>4</th>
      <td>Cool</td>
      <td>Rock</td>
      <td>Vodka</td>
      <td>Coca Cola/Pepsi</td>
      <td>F</td>
    </tr>
  </tbody>
</table>
</div>



#### Probabilities and logic

X = {Favorite Color, Favorite Music Genre, Favorite Beverage, Favorite Soft Drink}
  = {f_color, f_music, f_bev, f_drink}
y = {Gender}

$
\begin{align}
    p( A | B ) &= \text{probability of happaning event A when event B already happened} \\
               &= \frac{p( B | A ) p(A)}{p(B)}
\end{align}
$

As we assume in this algorithm that each features(inputs) are independet from each other, so the probability of independent event is as follow.

$
\begin{align}
    p(A, B, C, ... , Z) &= p(A) p(B) p(C) ... p(Z)
\end{align}
$

$
\begin{align}
    p(\text{ f_color, f_music, f_bev, f_drink }) = p(\text{f_color}) p(\text{f_music}) p(\text{f_bev})  p(\text{f_drink})
\end{align}
$

In our case,


$
\begin{align}
    p( y | X ) &= \text{probability of happaning event y when event X already happened} \\ \\
    p( y | X ) &= \frac{p( X | y ) p(y)}{p(X)} \\ \\
    p( y | \text{ [f_color, f_music, f_bev, f_drink] }) &= \frac{p( \text{ [f_color, f_music, f_bev, f_drink] }| y ) p(y)}{p(\text{ f_color, f_music, f_bev, f_drink })}\\ \\
    &= \frac{p(\text{f_color} | y) p(\text{f_music} | y) p(\text{f_bev} | y)p(\text{f_drink} | y)p(y)}{p(\text{f_color}) p(\text{f_music}) p(\text{f_bev})  p(\text{f_drink})}
\end{align}
$

Let me calculate for one column,

|Favourite color   |F        |M        |p( ... \|F) |p( ... \|M) |
|------------------|---------|---------|------------|------------|
|Cool.             |17.      |20.      |17/33 = 0.51|20/33 = 0.61|
|Neutral           |3.       |4.       |3/33 = 0.091|4/33 = 0.121|
|Warm.             |13.      |9.       |13/33 = 0.39|9/33 = 0.273|
|total.            |33.      |33.      |1           |1.          |

|Gender |freq|p(F), p(M)|
|-------|----|----|
|F      |33|33/66 = .5|
|M.     |33|33/66 = .5|
|total. |66|1|

for example we want to predict 

f_color = Cool

f_music = pop

f_bev = vodka

f_drink = fenta

Gender = ?

$
\begin{align}
\text{p(M | cool, pop, vodka, fenta)} &= \frac{\text{p(cool|M) p(pop|M) p(vodka|M) p(fenta|M) p(M)}}{\text{p(cool) p(pop) p(vodka) p(fenta)}} \\ \\
&=  \frac{\text{0.61 p(pop|M) p(vodka|M) p(fenta|M) 0.5}}{\text{p(cool) p(pop) p(vodka) p(fenta)}}
\end{align}
$

$
\begin{align}
\text{p(F | cool, pop, vodka, fenta)} &= \frac{\text{p(cool|F) p(pop|F) p(vodka|F) p(fenta|F) p(F)}}{\text{p(cool) p(pop) p(vodka) p(fenta)}} \\ \\
&= \frac{\text{0.51 p(pop|F) p(vodka|F) p(fenta|F) 0.5}}{\text{p(cool) p(pop) p(vodka) p(fenta)}}
\end{align}
$


```python
conditional_p = {}
for each_column in df.columns:
    conditional_p[each_column] = {}
    for each_value in df[each_column].unique():
        conditional_p[each_column][each_value] = {}
        conditional_p[each_column][each_value]['F'] = len(df.loc[(df[each_column] == each_value) & (df['Gender'] == 'F')].index)
        conditional_p[each_column][each_value]['M'] = len(df.loc[(df[each_column] == each_value) & (df['Gender'] == 'M')].index)
        conditional_p[each_column][each_value]['p( ' + each_value + ' | F)'] = conditional_p[each_column][each_value]['F'] / len(df.loc[df['Gender'] == 'F'].index)
        conditional_p[each_column][each_value]['p( ' + each_value + ' | M)'] = conditional_p[each_column][each_value]['M'] / len(df.loc[df['Gender'] == 'M'].index)

m_p = len(df.loc[df['Gender'] == 'M'].index) / len(df.index)
f_p = len(df.loc[df['Gender'] == 'F'].index) / len(df.index)
print(m_p, f_p)
pprint.pprint(conditional_p)
           
```

    0.5 0.5
    {'Favorite Beverage': {'Beer': {'F': 6,
                                    'M': 7,
                                    'p( Beer | F)': 0.18181818181818182,
                                    'p( Beer | M)': 0.21212121212121213},
                           "Doesn't drink": {'F': 5,
                                             'M': 9,
                                             "p( Doesn't drink | F)": 0.15151515151515152,
                                             "p( Doesn't drink | M)": 0.2727272727272727},
                           'Other': {'F': 7,
                                     'M': 4,
                                     'p( Other | F)': 0.21212121212121213,
                                     'p( Other | M)': 0.12121212121212122},
                           'Vodka': {'F': 4,
                                     'M': 5,
                                     'p( Vodka | F)': 0.12121212121212122,
                                     'p( Vodka | M)': 0.15151515151515152},
                           'Whiskey': {'F': 5,
                                       'M': 4,
                                       'p( Whiskey | F)': 0.15151515151515152,
                                       'p( Whiskey | M)': 0.12121212121212122},
                           'Wine': {'F': 6,
                                    'M': 4,
                                    'p( Wine | F)': 0.18181818181818182,
                                    'p( Wine | M)': 0.12121212121212122}},
     'Favorite Color': {'Cool': {'F': 17,
                                 'M': 20,
                                 'p( Cool | F)': 0.5151515151515151,
                                 'p( Cool | M)': 0.6060606060606061},
                        'Neutral': {'F': 3,
                                    'M': 4,
                                    'p( Neutral | F)': 0.09090909090909091,
                                    'p( Neutral | M)': 0.12121212121212122},
                        'Warm': {'F': 13,
                                 'M': 9,
                                 'p( Warm | F)': 0.3939393939393939,
                                 'p( Warm | M)': 0.2727272727272727}},
     'Favorite Music Genre': {'Electronic': {'F': 2,
                                             'M': 6,
                                             'p( Electronic | F)': 0.06060606060606061,
                                             'p( Electronic | M)': 0.18181818181818182},
                              'Folk/Traditional': {'F': 2,
                                                   'M': 2,
                                                   'p( Folk/Traditional | F)': 0.06060606060606061,
                                                   'p( Folk/Traditional | M)': 0.06060606060606061},
                              'Hip hop': {'F': 1,
                                          'M': 7,
                                          'p( Hip hop | F)': 0.030303030303030304,
                                          'p( Hip hop | M)': 0.21212121212121213},
                              'Jazz/Blues': {'F': 3,
                                             'M': 1,
                                             'p( Jazz/Blues | F)': 0.09090909090909091,
                                             'p( Jazz/Blues | M)': 0.030303030303030304},
                              'Pop': {'F': 13,
                                      'M': 4,
                                      'p( Pop | F)': 0.3939393939393939,
                                      'p( Pop | M)': 0.12121212121212122},
                              'R&B and soul': {'F': 2,
                                               'M': 4,
                                               'p( R&B and soul | F)': 0.06060606060606061,
                                               'p( R&B and soul | M)': 0.12121212121212122},
                              'Rock': {'F': 10,
                                       'M': 9,
                                       'p( Rock | F)': 0.30303030303030304,
                                       'p( Rock | M)': 0.2727272727272727}},
     'Favorite Soft Drink': {'7UP/Sprite': {'F': 8,
                                            'M': 5,
                                            'p( 7UP/Sprite | F)': 0.24242424242424243,
                                            'p( 7UP/Sprite | M)': 0.15151515151515152},
                             'Coca Cola/Pepsi': {'F': 17,
                                                 'M': 15,
                                                 'p( Coca Cola/Pepsi | F)': 0.5151515151515151,
                                                 'p( Coca Cola/Pepsi | M)': 0.45454545454545453},
                             'Fanta': {'F': 6,
                                       'M': 8,
                                       'p( Fanta | F)': 0.18181818181818182,
                                       'p( Fanta | M)': 0.24242424242424243},
                             'Other': {'F': 2,
                                       'M': 5,
                                       'p( Other | F)': 0.06060606060606061,
                                       'p( Other | M)': 0.15151515151515152}},
     'Gender': {'F': {'F': 33, 'M': 0, 'p( F | F)': 1.0, 'p( F | M)': 0.0},
                'M': {'F': 0, 'M': 33, 'p( M | F)': 0.0, 'p( M | M)': 1.0}}}


#### Testing

let us predict gender for my interest

color = cool

song = tradional

bev = beer

drink = 7up

Male prob = ?
Female prob = ?

$
\begin{align}
\text{p(M | Cool, Folk/Traditional, Beer, 7UP/Sprite)} &= \frac{\text{p(Cool|M) p(Folk/Traditional|M) p(Beer|M) p(7UP/Sprite|M) p(M)}}{\text{p(Cool) p(Folk/Traditional) p(Beer) p(7UP/Sprite)}} \\ \\
\end{align}
$

$
\begin{align}
\text{p(F | Cool, Folk/Traditional, Beer, 7UP/Sprite)} &= \frac{\text{p(Cool|F) p(Folk/Traditional|F) p(Beer|F) p(7UP/Sprite|F) p(F)}}{\text{p(Cool) p(Folk/Traditional) p(Beer) p(7UP/Sprite)}} \\ \\
\end{align}
$


---
$
\begin{align}
\text{p(M | Cool, Folk/Traditional, Beer, 7UP/Sprite)} + \text{p(F | Cool, Folk/Traditional, Beer, 7UP/Sprite)} = 1 \\ \\
\end{align}
$

$
\begin{align}
\frac{\text{p(Cool|M) p(Folk/Traditional|M) p(Beer|M) p(7UP/Sprite|M) p(M)}}{\text{p(Cool) p(Folk/Traditional) p(Beer) p(7UP/Sprite)}} + \frac{\text{p(Cool|F) p(Folk/Traditional|F) p(Beer|F) p(7UP/Sprite|F) p(F)}}{\text{p(Cool) p(Folk/Traditional) p(Beer) p(7UP/Sprite)}} &= 1 \\ \\
\end{align}
$

$
\begin{align}
\frac{\text{p(Cool|M) p(Folk/Traditional|M) p(Beer|M) p(7UP/Sprite|M) p(M)} + \text{p(Cool|F) p(Folk/Traditional|F) p(Beer|F) p(7UP/Sprite|F) p(F)}}{\text{p(Cool) p(Folk/Traditional) p(Beer) p(7UP/Sprite)}} &= 1 \\ \\
\end{align}
$

$
\begin{align}
\text{p(Cool|M) p(Folk/Traditional|M) p(Beer|M) p(7UP/Sprite|M) p(M)} + \text{p(Cool|F) p(Folk/Traditional|F) p(Beer|F) p(7UP/Sprite|F) p(F)} &= \text{p(Cool) p(Folk/Traditional) p(Beer) p(7UP/Sprite)} \\ \\
\end{align}
$

$
\text{p(Cool|M) p(Folk/Traditional|M) p(Beer|M) p(7UP/Sprite|M) p(M)} = \text{p(c_m) p(t_m) p(b_m) p(s_m) p(M)} \\
\text{p(Cool|F) p(Folk/Traditional|F) p(Beer|F) p(7UP/Sprite|F) p(F)} = \text{p(c_f) p(t_f) p(b_f) p(s_f) p(F)} \\
\text{p(Cool) p(Folk/Traditional) p(Beer) p(7UP/Sprite)} = \text{p(c) p(t) p(b) p(s)}
$

$
\begin{align}
\text{p(c_m) p(t_m) p(b_m) p(s_m) p(M)} + \text{p(c_f) p(t_f) p(b_f) p(s_f) p(F)} &= \text{p(c) p(t) p(b) p(s)} \\ \\
\end{align}
$



```python
f_c = 'Cool'  #favourite_color
f_s = 'Rock'  #favourite_music_genere
f_b = "Doesn't drink"  #favourite_beverage
f_d = '7UP/Sprite'  #favourite_drink

cp_f_c_m = conditional_p['Favorite Color'][f_c]['p( ' + f_c + ' | M)']           # p(Cool|M)
cp_f_s_m = conditional_p['Favorite Music Genre'][f_s]['p( ' + f_s + ' | M)']     # p(Rock|M)
cp_f_b_m = conditional_p['Favorite Beverage'][f_b]['p( ' + f_b + ' | M)']        # p(Doesn't drink|M)
cp_f_d_m = conditional_p['Favorite Soft Drink'][f_d]['p( ' + f_d + ' | M)']      # p(7UP|M)

cp_f_c_f = conditional_p['Favorite Color'][f_c]['p( ' + f_c + ' | F)']           # p(Cool|F)
cp_f_s_f = conditional_p['Favorite Music Genre'][f_s]['p( ' + f_s + ' | F)']     # p(Rock|F)
cp_f_b_f = conditional_p['Favorite Beverage'][f_b]['p( ' + f_b + ' | F)']        # p(Doesn't drink|F)
cp_f_d_f = conditional_p['Favorite Soft Drink'][f_d]['p( ' + f_d + ' | F)']      # p(7UP|F)

p_t = (cp_f_c_m * cp_f_s_m * cp_f_b_m * cp_f_d_m * m_p) + (cp_f_c_f * cp_f_s_f * cp_f_b_f * cp_f_d_f * f_p)

p_m = (cp_f_c_m * cp_f_s_m * cp_f_b_m * cp_f_d_m * m_p) / p_t
p_f = (cp_f_c_f * cp_f_s_f * cp_f_b_f * cp_f_d_f * f_p) / p_t

print('Male prob: {}, Female prob: {}'.format(p_m, p_f))
```

    Male prob: 0.5436241610738255, Female prob: 0.45637583892617456



```python

```

fbb60b963f157cb93531dc05673065bb4a289686
ipynb
Jupyter Notebook
Classification/BinaryClassification/GenderClassification_NaivBayes/GenderClassification.ipynb
dhruvinmodi/ml_without_libraries
8bbe30e2286b1d342dcea7de60d8275872591ab7
Apache-2.0
Classification/BinaryClassification/GenderClassification_NaivBayes/GenderClassification.ipynb
dhruvinmodi/ml_without_libraries
8bbe30e2286b1d342dcea7de60d8275872591ab7
Apache-2.0
Classification/BinaryClassification/GenderClassification_NaivBayes/GenderClassification.ipynb
dhruvinmodi/ml_without_libraries
8bbe30e2286b1d342dcea7de60d8275872591ab7
Apache-2.0
Qwen/Qwen-72B
1. YES
2. YES