# Naive Bayes Classifier

Naive Bayes classifier assumes that the effect of a particular feature in a class is independent of other features. 

$
\begin{align}
\ P(h|D) = \frac{P(D|h) P(h)}{P(D)}
\end{align}
$

- P(h): the probability of hypothesis h being true (regardless of the data). This is known as the prior probability of h
- P(D): the probability of the data (regardless of the hypothesis). This is known as the prior probability
- P(h|D): the probability of hypothesis h given the data D. This is known as posterior probability
- P(D|h): the probability of data d given that the hypothesis h was true. This is known as posterior probability


```python
import pickle as pkl

with open('../data/titanic_tansformed.pkl', 'rb') as f:
    df_data = pkl.load(f)
```


```python
df_data.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Survived</th>
      <th>Age</th>
      <th>SibSp</th>
      <th>Parch</th>
      <th>Fare</th>
      <th>1</th>
      <th>2</th>
      <th>3</th>
      <th>female</th>
      <th>male</th>
      <th>C</th>
      <th>Q</th>
      <th>S</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>0</td>
      <td>22.0</td>
      <td>1</td>
      <td>0</td>
      <td>7.2500</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>1</th>
      <td>1</td>
      <td>38.0</td>
      <td>1</td>
      <td>0</td>
      <td>71.2833</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>1</td>
      <td>26.0</td>
      <td>0</td>
      <td>0</td>
      <td>7.9250</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>3</th>
      <td>1</td>
      <td>35.0</td>
      <td>1</td>
      <td>0</td>
      <td>53.1000</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
    <tr>
      <th>4</th>
      <td>0</td>
      <td>35.0</td>
      <td>0</td>
      <td>0</td>
      <td>8.0500</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>1</td>
      <td>0</td>
      <td>0</td>
      <td>1</td>
    </tr>
  </tbody>
</table>
</div>




```python
df_data.shape
```




    (889, 13)




```python
data = df_data.drop("Survived",axis=1)
label = df_data["Survived"]
```


```python
from sklearn.model_selection import train_test_split  
data_train, data_test, label_train, label_test = train_test_split(data, label, test_size = 0.2, random_state = 101)
```


```python
from sklearn.naive_bayes import GaussianNB
import time

tic = time.time()
nb_cla = GaussianNB()
nb_cla.fit(data_train,label_train)
print('Time taken for training Naive Bayes', (time.time()-tic), 'secs')

predictions = nb_cla.predict(data_test)
print('Accuracy', nb_cla.score(data_test, label_test))

from sklearn.metrics import classification_report, confusion_matrix                
print(confusion_matrix(label_test, predictions))  
print(classification_report(label_test, predictions)) 
```

    Time taken for training Naive Bayes 0.0014061927795410156 secs
    Accuracy 0.8202247191011236
    [[95 12]
     [20 51]]
                 precision    recall  f1-score   support
    
              0       0.83      0.89      0.86       107
              1       0.81      0.72      0.76        71
    
    avg / total       0.82      0.82      0.82       178
    


## Multinomial Naive-Bayes
- Used when the values are discrete



```python
from sklearn.naive_bayes import MultinomialNB
import time

tic = time.time()
nb_cla = MultinomialNB()
nb_cla.fit(data_train,label_train)
print('Time taken for training Naive Bayes', (time.time()-tic), 'secs')

predictions = nb_cla.predict(data_test)
print('Accuracy', nb_cla.score(data_test, label_test))
```

    Time taken for training Naive Bayes 0.12916111946105957 secs
    Accuracy 0.7303370786516854


## Bernoulli Naive-Bayes
- Used when the values of all the features are binary


```python
from sklearn.naive_bayes import BernoulliNB
import time

tic = time.time()
nb_cla = BernoulliNB()
nb_cla.fit(data_train,label_train)
print('Time taken for training Naive Bayes', (time.time()-tic), 'secs')

predictions = nb_cla.predict(data_test)
print('Accuracy', nb_cla.score(data_test, label_test))
```

    Time taken for training Naive Bayes 0.0020270347595214844 secs
    Accuracy 0.7865168539325843



```python

```

278f3688830ec995075dbb045b2ef97cea569dbd
ipynb
Jupyter Notebook
classification/notebooks/08 - Naive Bayes.ipynb
pshn111/Machine-Learning-Package
fbbaa44daf5f0701ea77e5b62eb57ef822e40ab2
MIT
classification/notebooks/08 - Naive Bayes.ipynb
pshn111/Machine-Learning-Package
fbbaa44daf5f0701ea77e5b62eb57ef822e40ab2
MIT
classification/notebooks/08 - Naive Bayes.ipynb
pshn111/Machine-Learning-Package
fbbaa44daf5f0701ea77e5b62eb57ef822e40ab2
MIT
Qwen/Qwen-72B
1. YES
2. YES