<h1 align=center>Naive Bayes</h1>

<p><code>Naive Bayes</code> methods are a set of supervised learning algorithms
based on applying Bayes’ theorem with the “naive” assumption of
conditional independence between every pair of features given the
value of the class variable. Bayes’ theorem states the following
relationship, given class variable $y$ and dependent feature vector  $x_1$ through $x_n$</p>


$P(y \mid x_1, \dots, x_n) = \frac{P(y) P(x_1, \dots x_n \mid y)}
                                 {P(x_1, \dots, x_n)}$

Using the naive conditional independence assumption that

$P(x_i | y, x_1, \dots, x_{i-1}, x_{i+1}, \dots, x_n) = P(x_i | y)$

for all , this relationship is simplified to

$P(y \mid x_1, \dots, x_n) = \frac{P(y) \prod_{i=1}^{n} P(x_i \mid y)}
                                 {P(x_1, \dots, x_n)}$
                                 
Since $P(x_1, \dots, x_n)$ is constant given the input, we can use the following classification rule:

$ \begin{align}\begin{aligned}P(y \mid x_1, \dots, x_n) \propto P(y) \prod_{i=1}^{n} P(x_i \mid y)\\\Downarrow\\\hat{y} = \arg\max_y P(y) \prod_{i=1}^{n} P(x_i \mid y),\end{aligned}\end{align} $

and we can use Maximum A Posteriori (MAP) estimation to estimate $P(y)$ and $P(x_i \mid y)$ the former is then the relative frequency of class $y$  in the training set.


The different naive Bayes classifiers differ mainly by the assumptions they make regarding the distribution of $P(x_i \mid y)$

    In spite of their apparently over-simplified assumptions, naive Bayes classifiers have worked quite well in many real-world situations, famously document classification and spam filtering. They require a small amount of training data to estimate the necessary parameters. (For theoretical reasons why naive Bayes works well, and on which types of data it does, see the references below.)

Naive Bayes learners and classifiers can be extremely fast compared to more sophisticated methods. The decoupling of the class conditional feature distributions means that each distribution can be independently estimated as a one dimensional distribution. This in turn helps to alleviate problems stemming from the curse of dimensionality.

### Gaussian Naive Bayes
GaussianNB implements the Gaussian Naive Bayes algorithm for classification. The likelihood of the features is assumed to be Gaussian:

$P(x_i \mid y) = \frac{1}{\sqrt{2\pi\sigma^2_y}} \exp\left(-\frac{(x_i - \mu_y)^2}{2\sigma^2_y}\right)$


The parameters  $\sigma_y$
 and $\mu_y$
 are estimated using `maximum likelihood`.


```python
import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.naive_bayes import GaussianNB
```


```python
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
```


```python
Data = load_iris()
```


```python
Data.data
```




    array([[5.1, 3.5, 1.4, 0.2],
           [4.9, 3. , 1.4, 0.2],
           [4.7, 3.2, 1.3, 0.2],
           [4.6, 3.1, 1.5, 0.2],
           [5. , 3.6, 1.4, 0.2],
           [5.4, 3.9, 1.7, 0.4],
           [4.6, 3.4, 1.4, 0.3],
           [5. , 3.4, 1.5, 0.2],
           [4.4, 2.9, 1.4, 0.2],
           [4.9, 3.1, 1.5, 0.1],
           [5.4, 3.7, 1.5, 0.2],
           [4.8, 3.4, 1.6, 0.2],
           [4.8, 3. , 1.4, 0.1],
           [4.3, 3. , 1.1, 0.1],
           [5.8, 4. , 1.2, 0.2],
           [5.7, 4.4, 1.5, 0.4],
           [5.4, 3.9, 1.3, 0.4],
           [5.1, 3.5, 1.4, 0.3],
           [5.7, 3.8, 1.7, 0.3],
           [5.1, 3.8, 1.5, 0.3],
           [5.4, 3.4, 1.7, 0.2],
           [5.1, 3.7, 1.5, 0.4],
           [4.6, 3.6, 1. , 0.2],
           [5.1, 3.3, 1.7, 0.5],
           [4.8, 3.4, 1.9, 0.2],
           [5. , 3. , 1.6, 0.2],
           [5. , 3.4, 1.6, 0.4],
           [5.2, 3.5, 1.5, 0.2],
           [5.2, 3.4, 1.4, 0.2],
           [4.7, 3.2, 1.6, 0.2],
           [4.8, 3.1, 1.6, 0.2],
           [5.4, 3.4, 1.5, 0.4],
           [5.2, 4.1, 1.5, 0.1],
           [5.5, 4.2, 1.4, 0.2],
           [4.9, 3.1, 1.5, 0.2],
           [5. , 3.2, 1.2, 0.2],
           [5.5, 3.5, 1.3, 0.2],
           [4.9, 3.6, 1.4, 0.1],
           [4.4, 3. , 1.3, 0.2],
           [5.1, 3.4, 1.5, 0.2],
           [5. , 3.5, 1.3, 0.3],
           [4.5, 2.3, 1.3, 0.3],
           [4.4, 3.2, 1.3, 0.2],
           [5. , 3.5, 1.6, 0.6],
           [5.1, 3.8, 1.9, 0.4],
           [4.8, 3. , 1.4, 0.3],
           [5.1, 3.8, 1.6, 0.2],
           [4.6, 3.2, 1.4, 0.2],
           [5.3, 3.7, 1.5, 0.2],
           [5. , 3.3, 1.4, 0.2],
           [7. , 3.2, 4.7, 1.4],
           [6.4, 3.2, 4.5, 1.5],
           [6.9, 3.1, 4.9, 1.5],
           [5.5, 2.3, 4. , 1.3],
           [6.5, 2.8, 4.6, 1.5],
           [5.7, 2.8, 4.5, 1.3],
           [6.3, 3.3, 4.7, 1.6],
           [4.9, 2.4, 3.3, 1. ],
           [6.6, 2.9, 4.6, 1.3],
           [5.2, 2.7, 3.9, 1.4],
           [5. , 2. , 3.5, 1. ],
           [5.9, 3. , 4.2, 1.5],
           [6. , 2.2, 4. , 1. ],
           [6.1, 2.9, 4.7, 1.4],
           [5.6, 2.9, 3.6, 1.3],
           [6.7, 3.1, 4.4, 1.4],
           [5.6, 3. , 4.5, 1.5],
           [5.8, 2.7, 4.1, 1. ],
           [6.2, 2.2, 4.5, 1.5],
           [5.6, 2.5, 3.9, 1.1],
           [5.9, 3.2, 4.8, 1.8],
           [6.1, 2.8, 4. , 1.3],
           [6.3, 2.5, 4.9, 1.5],
           [6.1, 2.8, 4.7, 1.2],
           [6.4, 2.9, 4.3, 1.3],
           [6.6, 3. , 4.4, 1.4],
           [6.8, 2.8, 4.8, 1.4],
           [6.7, 3. , 5. , 1.7],
           [6. , 2.9, 4.5, 1.5],
           [5.7, 2.6, 3.5, 1. ],
           [5.5, 2.4, 3.8, 1.1],
           [5.5, 2.4, 3.7, 1. ],
           [5.8, 2.7, 3.9, 1.2],
           [6. , 2.7, 5.1, 1.6],
           [5.4, 3. , 4.5, 1.5],
           [6. , 3.4, 4.5, 1.6],
           [6.7, 3.1, 4.7, 1.5],
           [6.3, 2.3, 4.4, 1.3],
           [5.6, 3. , 4.1, 1.3],
           [5.5, 2.5, 4. , 1.3],
           [5.5, 2.6, 4.4, 1.2],
           [6.1, 3. , 4.6, 1.4],
           [5.8, 2.6, 4. , 1.2],
           [5. , 2.3, 3.3, 1. ],
           [5.6, 2.7, 4.2, 1.3],
           [5.7, 3. , 4.2, 1.2],
           [5.7, 2.9, 4.2, 1.3],
           [6.2, 2.9, 4.3, 1.3],
           [5.1, 2.5, 3. , 1.1],
           [5.7, 2.8, 4.1, 1.3],
           [6.3, 3.3, 6. , 2.5],
           [5.8, 2.7, 5.1, 1.9],
           [7.1, 3. , 5.9, 2.1],
           [6.3, 2.9, 5.6, 1.8],
           [6.5, 3. , 5.8, 2.2],
           [7.6, 3. , 6.6, 2.1],
           [4.9, 2.5, 4.5, 1.7],
           [7.3, 2.9, 6.3, 1.8],
           [6.7, 2.5, 5.8, 1.8],
           [7.2, 3.6, 6.1, 2.5],
           [6.5, 3.2, 5.1, 2. ],
           [6.4, 2.7, 5.3, 1.9],
           [6.8, 3. , 5.5, 2.1],
           [5.7, 2.5, 5. , 2. ],
           [5.8, 2.8, 5.1, 2.4],
           [6.4, 3.2, 5.3, 2.3],
           [6.5, 3. , 5.5, 1.8],
           [7.7, 3.8, 6.7, 2.2],
           [7.7, 2.6, 6.9, 2.3],
           [6. , 2.2, 5. , 1.5],
           [6.9, 3.2, 5.7, 2.3],
           [5.6, 2.8, 4.9, 2. ],
           [7.7, 2.8, 6.7, 2. ],
           [6.3, 2.7, 4.9, 1.8],
           [6.7, 3.3, 5.7, 2.1],
           [7.2, 3.2, 6. , 1.8],
           [6.2, 2.8, 4.8, 1.8],
           [6.1, 3. , 4.9, 1.8],
           [6.4, 2.8, 5.6, 2.1],
           [7.2, 3. , 5.8, 1.6],
           [7.4, 2.8, 6.1, 1.9],
           [7.9, 3.8, 6.4, 2. ],
           [6.4, 2.8, 5.6, 2.2],
           [6.3, 2.8, 5.1, 1.5],
           [6.1, 2.6, 5.6, 1.4],
           [7.7, 3. , 6.1, 2.3],
           [6.3, 3.4, 5.6, 2.4],
           [6.4, 3.1, 5.5, 1.8],
           [6. , 3. , 4.8, 1.8],
           [6.9, 3.1, 5.4, 2.1],
           [6.7, 3.1, 5.6, 2.4],
           [6.9, 3.1, 5.1, 2.3],
           [5.8, 2.7, 5.1, 1.9],
           [6.8, 3.2, 5.9, 2.3],
           [6.7, 3.3, 5.7, 2.5],
           [6.7, 3. , 5.2, 2.3],
           [6.3, 2.5, 5. , 1.9],
           [6.5, 3. , 5.2, 2. ],
           [6.2, 3.4, 5.4, 2.3],
           [5.9, 3. , 5.1, 1.8]])




```python
Data.target
```




    array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
           0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
           0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
           1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
           1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
           2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
           2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2])




```python
df = pd.DataFrame(Data.data,columns=Data.feature_names)
df
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>sepal length (cm)</th>
      <th>sepal width (cm)</th>
      <th>petal length (cm)</th>
      <th>petal width (cm)</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>5.1</td>
      <td>3.5</td>
      <td>1.4</td>
      <td>0.2</td>
    </tr>
    <tr>
      <th>1</th>
      <td>4.9</td>
      <td>3.0</td>
      <td>1.4</td>
      <td>0.2</td>
    </tr>
    <tr>
      <th>2</th>
      <td>4.7</td>
      <td>3.2</td>
      <td>1.3</td>
      <td>0.2</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4.6</td>
      <td>3.1</td>
      <td>1.5</td>
      <td>0.2</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5.0</td>
      <td>3.6</td>
      <td>1.4</td>
      <td>0.2</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>145</th>
      <td>6.7</td>
      <td>3.0</td>
      <td>5.2</td>
      <td>2.3</td>
    </tr>
    <tr>
      <th>146</th>
      <td>6.3</td>
      <td>2.5</td>
      <td>5.0</td>
      <td>1.9</td>
    </tr>
    <tr>
      <th>147</th>
      <td>6.5</td>
      <td>3.0</td>
      <td>5.2</td>
      <td>2.0</td>
    </tr>
    <tr>
      <th>148</th>
      <td>6.2</td>
      <td>3.4</td>
      <td>5.4</td>
      <td>2.3</td>
    </tr>
    <tr>
      <th>149</th>
      <td>5.9</td>
      <td>3.0</td>
      <td>5.1</td>
      <td>1.8</td>
    </tr>
  </tbody>
</table>
<p>150 rows × 4 columns</p>
</div>




```python
df['Target'] = Data.target
```


```python
df.head()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>sepal length (cm)</th>
      <th>sepal width (cm)</th>
      <th>petal length (cm)</th>
      <th>petal width (cm)</th>
      <th>Target</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>5.1</td>
      <td>3.5</td>
      <td>1.4</td>
      <td>0.2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>1</th>
      <td>4.9</td>
      <td>3.0</td>
      <td>1.4</td>
      <td>0.2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>4.7</td>
      <td>3.2</td>
      <td>1.3</td>
      <td>0.2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4.6</td>
      <td>3.1</td>
      <td>1.5</td>
      <td>0.2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5.0</td>
      <td>3.6</td>
      <td>1.4</td>
      <td>0.2</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
</div>




```python
df.tail()
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>sepal length (cm)</th>
      <th>sepal width (cm)</th>
      <th>petal length (cm)</th>
      <th>petal width (cm)</th>
      <th>Target</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>145</th>
      <td>6.7</td>
      <td>3.0</td>
      <td>5.2</td>
      <td>2.3</td>
      <td>2</td>
    </tr>
    <tr>
      <th>146</th>
      <td>6.3</td>
      <td>2.5</td>
      <td>5.0</td>
      <td>1.9</td>
      <td>2</td>
    </tr>
    <tr>
      <th>147</th>
      <td>6.5</td>
      <td>3.0</td>
      <td>5.2</td>
      <td>2.0</td>
      <td>2</td>
    </tr>
    <tr>
      <th>148</th>
      <td>6.2</td>
      <td>3.4</td>
      <td>5.4</td>
      <td>2.3</td>
      <td>2</td>
    </tr>
    <tr>
      <th>149</th>
      <td>5.9</td>
      <td>3.0</td>
      <td>5.1</td>
      <td>1.8</td>
      <td>2</td>
    </tr>
  </tbody>
</table>
</div>




```python
df.isna().sum()
```




    sepal length (cm)    0
    sepal width (cm)     0
    petal length (cm)    0
    petal width (cm)     0
    Target               0
    dtype: int64




```python
sns.pairplot(df)
```


```python
x = df.drop('Target',axis=1)
y = df['Target']
```


```python
sns.pairplot(x)
```


```python
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.35, random_state=0)
```


```python
X_train.shape
```




    (97, 4)




```python
X_test.shape
```




    (53, 4)




```python
y_train.shape
```




    (97,)




```python
y_test.shape
```




    (53,)




```python
gnb = GaussianNB()
```


```python
gnb.fit(X_train,y_train)
```




    GaussianNB(priors=None, var_smoothing=1e-09)




```python
New_preddiction = gnb.predict(X_test)
New_preddiction
```




    array([2, 1, 0, 2, 0, 2, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 2, 1,
           0, 0, 2, 0, 0, 1, 1, 0, 2, 1, 0, 2, 2, 1, 0, 1, 1, 1, 2, 0, 2, 0,
           0, 1, 2, 2, 1, 2, 1, 2, 1])




```python
print("Number of mislabeled points out of a total %d points : %d"
      % (X_test.shape[0], (y_test != New_preddiction).sum()))
```

    Number of mislabeled points out of a total 53 points : 2



```python
gnb_Train_score = gnb.score(X_train,y_train)*100
gnb_Train_score
```




    95.87628865979381




```python
gnb_Test_score= gnb.score(X_test,y_test)*100
gnb_Test_score
```




    96.22641509433963




```python
from sklearn.metrics import classification_report,confusion_matrix
```


```python
cm = confusion_matrix(y_test,New_preddiction)
cm
```




    array([[16,  0,  0],
           [ 0, 21,  0],
           [ 0,  2, 14]], dtype=int64)




```python
sns.heatmap(cm,annot=True)
```


```python
print(classification_report(y_test,New_preddiction))
```

                  precision    recall  f1-score   support
    
               0       1.00      1.00      1.00        16
               1       0.91      1.00      0.95        21
               2       1.00      0.88      0.93        16
    
        accuracy                           0.96        53
       macro avg       0.97      0.96      0.96        53
    weighted avg       0.97      0.96      0.96        53
    


### Bernoulli Naive Bayes
BernoulliNB implements the naive Bayes training and classification algorithms for data that is distributed according to multivariate Bernoulli distributions; i.e., there may be multiple features but each one is assumed to be a `binary-valued (Bernoulli, boolean) variable`. Therefore, this class requires samples to be represented as `binary-valued` feature vectors; if handed any other kind of data, a BernoulliNB instance may binarize its input (depending on the binarize parameter).

The decision rule for Bernoulli naive Bayes is based on

$P(x_i \mid y) = P(i \mid y) x_i + (1 - P(i \mid y)) (1 - x_i)$

which differs from multinomial NB’s rule in that it explicitly penalizes the non-occurrence of a feature $i$  that is an indicator for class $y$, where the multinomial variant would simply ignore a non-occurring feature.

In the case of text classification, word occurrence vectors (rather than word count vectors) may be used to train and use this classifier. BernoulliNB might perform better on some datasets, especially those with shorter documents. It is advisable to evaluate both models, if time permits.


```python
from sklearn.naive_bayes import BernoulliNB
```


```python
clf = BernoulliNB()
```


```python
clf.fit(X_train,y_train)
```




    BernoulliNB(alpha=1.0, binarize=0.0, class_prior=None, fit_prior=True)




```python
y_pred = clf.predict(X_test)
```


```python
pd.DataFrame({'Actual_data':y_test,
             "New_Predication":y_pred})
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Actual_data</th>
      <th>New_Predication</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>114</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>62</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>33</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>107</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>7</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>100</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>40</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>86</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>76</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>71</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>134</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>51</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>73</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>54</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>63</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>37</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>78</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>90</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>45</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>16</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>121</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>66</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>24</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>126</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>22</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>44</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>97</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>93</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>26</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>137</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>84</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>27</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>127</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>132</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>59</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>18</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>83</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>61</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>92</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>112</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>141</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>43</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>10</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>60</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>116</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>144</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>119</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>108</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>69</th>
      <td>1</td>
      <td>0</td>
    </tr>
    <tr>
      <th>135</th>
      <td>2</td>
      <td>0</td>
    </tr>
    <tr>
      <th>56</th>
      <td>1</td>
      <td>0</td>
    </tr>
  </tbody>
</table>
</div>




```python
print("Number of mislabeled points out of a total %d points : %d"
      % (X_test.shape[0], (y_test != y_pred).sum()))
```

    Number of mislabeled points out of a total 53 points : 37



```python
Clf_Train = clf.score(X_train,y_train)
```


```python
Clf_Train
```




    0.35051546391752575




```python
Clf_Test = clf.score(X_test,y_test)
Clf_Test
```




    0.3018867924528302



### Multinomial Naive Bayes

MultinomialNB implements the naive Bayes algorithm for multinomially distributed data, and is one of the two classic naive Bayes variants used in text classification (where the data are typically represented as word vector counts, although tf-idf vectors are also known to work well in practice). The distribution is parametrized by vectors $\theta_y = (\theta_{y1},\ldots,\theta_{yn})$ for each class $y$,where $n$ is the number of features (in text classification, the size of the vocabulary) and $\theta_{yi}$ is the probability $P(x_i \mid y)$ of feature $i$ appearing in a sample belonging to class $y$.
The parameters $\theta_y$ is estimated by a smoothed version of maximum likelihood, i.e. relative frequency counting:

$\hat{\theta}_{yi} = \frac{ N_{yi} + \alpha}{N_y + \alpha n}$

where

$N_{yi} = \sum_{x \in T} x_i$ s the number of times feature $i$ appears in a sample of class $y$ in the training set $T$ and $N_{y} = \sum_{i=1}^{n} N_{yi}$    the total count of all features for class $y$



The smoothing priors $\alpha \ge 0$ accounts for features not present in the learning samples and prevents zero probabilities in further computations. Setting  $\alpha = 1$ is called Laplace smoothing, while $\alpha < 1$ is called Lidstone smoothing.


```python
from sklearn.naive_bayes import MultinomialNB
```


```python
mnb = MultinomialNB()
```


```python
mnb.fit(X_train,y_train)
```




    MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)




```python
mnb.coef_
```




    array([[-0.70623447, -1.09313756, -1.92673658, -3.66067647],
           [-0.87705159, -1.63443011, -1.21999634, -2.36768292],
           [-0.95882831, -1.74376609, -1.1300177 , -2.1305659 ]])




```python
y_pred = mnb.predict(X_test)
```


```python
pd.DataFrame({"Actual":y_test,
              "predicated":y_pred})
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Actual</th>
      <th>predicated</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>114</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>62</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>33</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>107</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>7</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>100</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>40</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>86</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>76</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>71</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>134</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>51</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>73</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>54</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>63</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>37</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>78</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>90</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>45</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>16</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>121</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>66</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>24</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>126</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>22</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>44</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>97</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>93</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>26</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>137</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>84</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>27</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>127</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>132</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>59</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>18</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>83</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>61</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>92</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>112</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>141</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>43</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>10</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>60</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>116</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>144</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>119</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>108</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>69</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>135</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>56</th>
      <td>1</td>
      <td>2</td>
    </tr>
  </tbody>
</table>
</div>




```python
print("Number of mislabeled points out of a total %d points : %d"
      % (X_test.shape[0], (y_test != y_pred).sum()))
```

    Number of mislabeled points out of a total 53 points : 21



```python
mnb_Train = mnb.score(X_train,y_train)
mnb_Train
```




    0.7422680412371134




```python
mnb_Test = mnb.score(X_test,y_test)
mnb_Test
```




    0.6037735849056604




```python
cm = confusion_matrix(y_test,y_pred)
```


```python
cm
```




    array([[16,  0,  0],
           [ 0,  0, 21],
           [ 0,  0, 16]], dtype=int64)




```python
sns.heatmap(cm,annot=True)
```


```python
print(classification_report(y_test,y_pred))
```

                  precision    recall  f1-score   support
    
               0       1.00      1.00      1.00        16
               1       0.00      0.00      0.00        21
               2       0.43      1.00      0.60        16
    
        accuracy                           0.60        53
       macro avg       0.48      0.67      0.53        53
    weighted avg       0.43      0.60      0.48        53
    


    c:\users\reddy\appdata\local\programs\python\python36\lib\site-packages\sklearn\metrics\_classification.py:1272: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
      _warn_prf(average, modifier, msg_start, len(result))


###  Complement Naive Bayes
ComplementNB implements the complement naive Bayes (CNB) algorithm. CNB is an adaptation of the standard multinomial naive Bayes (MNB) algorithm that is particularly suited for `imbalanced data sets`. Specifically, CNB uses statistics from the complement of each class to compute the model’s weights. The inventors of CNB show empirically that the parameter estimates for CNB are more stable than those for MNB. Further, CNB regularly outperforms MNB (often by a considerable margin) on text classification tasks. The procedure for calculating the weights is as follows:

$ \begin{align}\begin{aligned}\hat{\theta}_{ci} = \frac{\alpha_i + \sum_{j:y_j \neq c} d_{ij}}
                         {\alpha + \sum_{j:y_j \neq c} \sum_{k} d_{kj}}\\w_{ci} = \log \hat{\theta}_{ci}\\w_{ci} = \frac{w_{ci}}{\sum_{j} |w_{cj}|}\end{aligned}\end{align} $
                         
                         
where the summations are over all documents $j$ not in class $c$, $d_{ij}$ is either the count or `tf-idf` value of term $i$ in document $j$ $\alpha_i$ is a smoothing hyperparameter like that found in MNB, and $\alpha = \sum_{i} \alpha_i$ The second normalization addresses the tendency for longer documents to dominate parameter estimates in MNB. The classification rule is



$\hat{c} = \arg\min_c \sum_{i} t_i w_{ci}$



```python
from sklearn.naive_bayes import ComplementNB
```


```python
CNB = ComplementNB()
CNB
```




    ComplementNB(alpha=1.0, class_prior=None, fit_prior=True, norm=False)




```python
CNB.fit(X_train,y_train)
```




    ComplementNB(alpha=1.0, class_prior=None, fit_prior=True, norm=False)




```python
New_predi = CNB.predict(X_test)
```


```python
pd.DataFrame({'Actual_Data':y_test,
              "New_Data":New_predi})
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Actual_Data</th>
      <th>New_Data</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>114</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>62</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>33</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>107</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>7</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>100</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>40</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>86</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>76</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>71</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>134</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>51</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>73</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>54</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>63</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>37</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>78</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>90</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>45</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>16</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>121</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>66</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>24</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>126</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>22</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>44</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>97</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>93</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>26</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>137</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>84</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>27</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>127</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>132</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>59</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>18</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>83</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>61</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>92</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>112</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>141</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>43</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>10</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>60</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>116</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>144</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>119</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>108</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>69</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>135</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>56</th>
      <td>1</td>
      <td>2</td>
    </tr>
  </tbody>
</table>
</div>




```python
print("Number of mislabeled points out of a total %d points : %d"
      % (X_test.shape[0], (y_test != New_predi).sum()))
```

    Number of mislabeled points out of a total 53 points : 21



```python
CNB_Train_score =CNB.score(X_train,y_train)
CNB_Train_score
```




    0.7010309278350515




```python
CNB_Test_score= CNB.score(X_test,y_test)
CNB_Test_score
```




    0.6037735849056604




```python
cm = confusion_matrix(y_test,New_predi)
cm
```




    array([[16,  0,  0],
           [ 0,  0, 21],
           [ 0,  0, 16]], dtype=int64)




```python
sns.heatmap(cm,annot=True)
```

### Categorical Naive Bayes

CategoricalNB implements the categorical naive Bayes algorithm for `categorically distributed data`. It assumes that each feature, which is described by the index $i$, has its own categorical distribution.

For each feature $i$ in the training set $X$, CategoricalNB estimates a categorical distribution for each feature i of X conditioned on the class y. The index set of the samples is defined as $J = \{ 1, \dots, m \}$ , with $m$ as the number of samples.


The probability of category $t$ in feature $i$  given class $c$ is estimated as:

$P(x_i = t \mid y = c \: ;\, \alpha) = \frac{ N_{tic} + \alpha}{N_{c} +
                                       \alpha n_i}$
 
where $N_{tic} = |\{j \in J \mid x_{ij} = t, y_j = c\}|$ is the number of times category $t$ appears in the samples $x_i$  which belong to class $c$, $N_{c} = |\{ j \in J\mid y_j = c\}$ is the number of samples with class c,$\alpha$  is a smoothing parameter and $n_i$ is the number of available categories of feature $i$.

CategoricalNB assumes that the sample matrix $X$ is encoded (for instance with the help of OrdinalEncoder) such that all categories for each feature $i$ are represented with numbers $0, ..., n_i - 1$ where $n_i$ is the number of available categories of feature $i$.


```python
from sklearn.naive_bayes import CategoricalNB
```


```python
CatNB = CategoricalNB()
```


```python
CatNB.fit(X_train,y_train)
```




    CategoricalNB(alpha=1.0, class_prior=None, fit_prior=True)




```python
CatNB_train_score = CatNB.score(X_train,y_train)
CatNB_train_score
```




    0.9381443298969072




```python
CatNB_Test_Scoer = CatNB.score(X_test,y_test)
CatNB_Test_Scoer
```




    0.9245283018867925




```python
New_pred = CatNB.predict(X_test)
pd.DataFrame({'Actual_Data':y_test,
             "New_predict":New_pred})
```




<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>Actual_Data</th>
      <th>New_predict</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>114</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>62</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>33</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>107</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>7</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>100</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>40</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>86</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>76</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>71</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>134</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>51</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>73</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>54</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>63</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>37</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>78</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>90</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>45</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>16</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>121</th>
      <td>2</td>
      <td>1</td>
    </tr>
    <tr>
      <th>66</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>24</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>8</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>126</th>
      <td>2</td>
      <td>1</td>
    </tr>
    <tr>
      <th>22</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>44</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>97</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>93</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>26</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>137</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>84</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>27</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>127</th>
      <td>2</td>
      <td>1</td>
    </tr>
    <tr>
      <th>132</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>59</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>18</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>83</th>
      <td>1</td>
      <td>2</td>
    </tr>
    <tr>
      <th>61</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>92</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>112</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>2</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>141</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>43</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>10</th>
      <td>0</td>
      <td>0</td>
    </tr>
    <tr>
      <th>60</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>116</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>144</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>119</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>108</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>69</th>
      <td>1</td>
      <td>1</td>
    </tr>
    <tr>
      <th>135</th>
      <td>2</td>
      <td>2</td>
    </tr>
    <tr>
      <th>56</th>
      <td>1</td>
      <td>1</td>
    </tr>
  </tbody>
</table>
</div>




```python
print("Number of mislabeled points out of a total %d points : %d"
      % (X_test.shape[0], (y_test != New_pred).sum()))
```

    Number of mislabeled points out of a total 53 points : 4



```python
cm = confusion_matrix(y_test,New_pred)
```


```python
sns.heatmap(cm,annot=True)
```


```python
print(classification_report(y_test,New_pred))
```

                  precision    recall  f1-score   support
    
               0       1.00      1.00      1.00        16
               1       0.87      0.95      0.91        21
               2       0.93      0.81      0.87        16
    
        accuracy                           0.92        53
       macro avg       0.93      0.92      0.93        53
    weighted avg       0.93      0.92      0.92        53
    



```python
from sklearn.metrics import accuracy_score
```


```python
accuracy_score(y_test,New_pred)
```




    0.9245283018867925




```python
di = {'Multinomial_Train_score':mnb_Train,
             'Multinomial_Test_score':mnb_Test,
             'Gaussian_Train_score':gnb_Train_score,
             'Gaussian_Test_score':gnb_Test_score,
             'Bernoulli_Train_score':Clf_Train,
             'Bernoulli_Test_Score':Clf_Test,
             'CategoricalNB_Train_Score':CatNB_train_score,
             'CategoricalNB_Test_score':CatNB_Test_Scoer,
             'ComplementNB_Train_score':CNB_Train_score,
             'ComplementNB_Test_score':CNB_Test_score
             }
```


```python
di
```




    {'Multinomial_Train_score': 0.7422680412371134,
     'Multinomial_Test_score': 0.6037735849056604,
     'Gaussian_Train_score': 95.87628865979381,
     'Gaussian_Test_score': 96.22641509433963,
     'Bernoulli_Train_score': 0.35051546391752575,
     'Bernoulli_Test_Score': 0.3018867924528302,
     'CategoricalNB_Train_Score': 0.9381443298969072,
     'CategoricalNB_Test_score': 0.9245283018867925,
     'ComplementNB_Train_score': 0.7010309278350515,
     'ComplementNB_Test_score': 0.6037735849056604}




```python
index=['Multinomial','Gaussian','Bernoulli','CategoricalNB','ComplementNB']
```

|Model|Data type|Difference|
|------|--------|----------|
|Multinomial|Discrete eg=count||
|Gaussian|Continuous|
|Bernoulli|Binary(T/F,Y/N,0/1)|

0f4033fec875d16779d9d875ad2e694858a05c27
ipynb
Jupyter Notebook
Classification/Naive Bayes/ML0101EN-Naive Bayes.ipynb
vivekrathi14/Machine-Learning-with-Scikit-Learn-Python-3.x
16828cf26e4956166ec5c4cee7755a527ad27322
MIT
2019-05-01T01:14:05.000Z
2022-03-13T14:59:32.000Z
Supervised Learning/Classifications/Naive Bayes/ML0101EN-Naive Bayes.ipynb
shsarv/Machine-Learning
bb511ec208d508a886a540957c62a70347214f47
MIT
2020-09-26T05:31:29.000Z
2020-09-26T06:16:01.000Z
Supervised Learning/Classifications/Naive Bayes/ML0101EN-Naive Bayes.ipynb
shsarv/Machine-Learning
bb511ec208d508a886a540957c62a70347214f47
MIT
2019-05-01T01:20:48.000Z
2022-01-15T17:38:48.000Z
Qwen/Qwen-72B
1. YES
2. YES