Machine Learning

A practical explanation of a Naive Bayes classifier

Bruno Stecanella
by Bruno Stecanella

A practical explanation of a Naive Bayes classifier

The simplest solutions are usually the most powerful ones, and Naive Bayes is a good example of that. In spite of the great advances of the Machine Learning in the last years, it has proven to not only be simple but also fast, accurate, and reliable. It has been successfully used for many purposes, but it works particularly well with natural language processing (NLP) problems.


We’re going to be working with an algorithm called Multinomial Naive Bayes. We’ll walk through the algorithm applied to NLP with an example, so by the end, not only will you know how this method works, but also why it works. Then, we’ll lay out a few advanced techniques that can make Naive Bayes competitive with more complex Machine Learning algorithms, such as SVM and neural networks.

A Simple Example


Text Tag
“A great game” Sports
“The election was over” Not sports
“Very clean match” Sports
“A clean but forgettable game” Sports
“It was a close election” Not sports


Since Naive Bayes is a probabilistic classifier, we want to calculate the probability that the sentence “A very close game” is Sports and the probability that it’s Not Sports. Then, we take the largest one. Written mathematically, what we want is P(Sports | a\, very\, close\, game)— the probability that the tag of a sentence is Sports given that the sentence is “A very close game”.

That’s great, but how do we calculate these probabilities?

Let’s dig in!

Feature Engineering




Bayes’ Theorem



    \[ P(A | B) = \frac {P(B | A) \times P(A)} { P(B)} \]

In our case, we have P(sports | a\, very\, close\, game), so using this theorem we can reverse the conditional probability:

    \[ P(sports | a\, very\, close\, game) = \frac {P(a\, very\, close\, game | sports) \times P(sports)} { P(a\, very\, close\, game)} \]


P(a\, very\, close\, game | Sports) \times P(Sports)


P(a\, very\, close\, game | Not\, Sports) \times P(Not\, Sports)

This is better, since we could actually calculate these probabilities! Just count how many times the sentence A very close game” appears in the Sports tag, divide it by the total, and obtain P(a\, very\, close\, game | Sports).


Being Naive


We write this as:

P(a\, very\, close\, game) = P(a) \times P(very) \times P(close) \times P(game)

This assumption is very strong but super useful. It’s what makes this model work well with little data or data that may be mislabeled. The next step is just applying this to what we had before:

P(a\, very\, close\, game | Sports) = P(a | Sports) \times P(very | Sports) \times P(close | Sports) \times P(game | Sports)


Calculating Probabilities


Calculating a probability is just counting in our training data.


Then, calculating P(game | Sports) means counting how many times the word “game” appears in Sports texts (2) divided by the total number of words in sports (11). Therefore, P(game | Sports) = \frac{2 }{11}

However, we run into a problem here: “close” doesn’t appear in any Sports text! That means that P(close | Sports) = 0.  This is rather inconvenient since we are going to be multiplying it with the other probabilities, so we’ll end up with P(a | Sports) \times P(very | Sports) \times 0 \times P(game | Sports). This equals 0, since in a multiplication if one of the terms is zero, the whole calculation is nullified. Doing things this way simply doesn’t give us any information at all, so we have to find a way around.


Since the number of possible words is 14 (I counted them!), applying smoothing we get that P(game | sports) = \dfrac {2 + 1}{11 + 14}. The full results are:

Word P(word | Sports) P(word | Not Sports)
a \dfrac {2 + 1}{11 + 14} \dfrac {1 + 1}{9 + 14}
very \dfrac {1 + 1}{11 + 14} \dfrac {0 + 1}{9 + 14}
close \dfrac {0 + 1}{11 + 14} \dfrac {1 + 1}{9 + 14}


P(a | Sports) \times P(very | Sports) \times P(close | Sports) \times P(game | Sports) \times P(Sports )\\ = 2.76\times 10^{-5}\\ = 0.0000276\\\\ P(a \vert Not\, Sports) \times P(very | Not\, Sports) \times P(close | Not\, Sports) \times P(game | Not\, Sports) \times P( Not\, Sports)\\ = 0.572\times 10^{-5}\\ = 0.00000572


Advanced techniques



Creating a Naive Bayes Classifier with MonkeyLearn




  • It’s simple to use


  • No setup is required:

You don’t have to worry about building the underlying infrastructure for a text analysis model. MonkeyLearn does it for you. Performance, scalability, logging, architecture, tools, etc…MonkeyLearn has it all under control!

  • You don’t need to code:


  • Faster training:


We’ve made a strong case for why you should create your naive bayes text classifier with MonkeyLearn, but let’s see how easy it is to create your model. Here’s a step-by-step guide to help you get started.

1. Create a text classifier

Sign up for free to MonkeyLearn and go to the MonkeyLearn Dashboard, select ‘Create Model’ and click on the ‘Classifier‘ option:

Choose model type

2. Select ‘Topic Classification’


Type of classification

3. Upload your training data

You can easily upload texts from an Excel or CSV file, or seamlessly connect with third-party apps such as Gmail, Twitter or Zendesk… there’s a wide range of options. Remember, it’s always best to upload a large bunch of data to make sure your model has enough training data:

Import your text data

4. Create your tags


Define tags

5. Train your classifier


Tag data


6. Change to Naive Bayes


Select naive bayes algorithm


7. Test your Naive Bayes classifier

Is it ready? This phase will show you whether it’s time to put your model to work or continue training it. Click on the ‘Run’ tab and type something into the text box. Did your model predict the correct tag? Try a few more times to make sure you’re using a classifier that’s ready work for you:

Testing the classifier


8. Start working with your model


a) Using MonkeyLearn’s API with the programming language of your choice:

MonkeyLearn API


b) Go to the ‘Batch’ tab and upload a CSV or Excel file:

Batch processing

After running text data from your file through your model, you’ll get a new file in return with all the tag predictions, in just a few seconds.

c) Use third-party Integrations:

MonkeyLearn has various integrations, including Google Sheets, Zapier, Rapidminer or Zendesk, so that you can analyze texts without any hassle. Just go to the ‘Integrate’ section and take your pick:


Final words


Whether you’re a Machine Learning expert or not, you have the tools to build your own Naive Bayes classifier. You don’t need to spend hours studying and coding – just sign up to MonkeyLearn, create a model in just a few steps, and integrate it with your application.


Bruno Stecanella

Bruno Stecanella

Engineering student and primate enthusiast. I write code and blog about monkeys. Sometimes about other stuff, too.


Have something to say?

Text Analysis with Machine Learning

business processes and save hours of manual data processing.