Naive Bayes
A method that uses the probabilities of the words appearing in an email to classify it as spam or legitimate mail.
Countless emails arrive in your inbox every day. Some are important work messages, while others may be ads or scams. How does a computer read an email and decide that it is spam? Is the mere appearance of certain words enough?
In this way, the method that distinguishes legitimate mail from spam using the words in an email and the probability of belonging to each class is Naive Bayes. It learns the patterns in which words appear in emails that have already been classified, then calculates which group a new email is more likely to belong to.
KNN looks for similar emails, and SVM learns a decision boundary that separates the mail. Naive Bayes differs in that it combines the probabilities of words appearing in each class. Because it uses the simplifying assumption that words are independent of one another given the class, it computes quickly, but that assumption does not accurately reflect the structure of real language.
Naive Bayes is used in email spam filters, document topic classification, sorting customer inquiries by type and more. In this article we use a spam detection example to look at how it works, the probability calculations, preprocessing and smoothing, interpreting classification performance, how it differs from other analyses, and what to watch for in real applications.
What is Naive Bayes?
A probability-based classification method that uses Bayes' theorem and the conditional independence assumption.
Naive Bayes first determines the proportions of spam and legitimate mail among emails whose correct labels are already known. Next, it learns which words appear how often in each group. When a new email comes in, it classifies it by comparing how likely its words are to appear in spam versus in legitimate mail.
At this point, it assumes that the words are independent of one another given the class. In reality, words like “special” and “discount” tend to appear together, but to keep the calculation simple it combines the probabilities of the individual words. That is why it is called “naive” (simple).
Why do we need Naive Bayes?
It can quickly classify documents that contain many words.
Emails vary in length and are made up of many different words. Rather than turning the whole document into a single number, making features from the frequency or presence of the words that appear in an email gives you clues for telling spam from legitimate mail.
Training and prediction are relatively fast, and you can also see which words appear frequently in each class. On the other hand, it cannot fully capture the context of expressions or complex relationships between words, so it has limits with sentences whose meaning changes with context.
How does Naive Bayes work?
It compares groups by combining prior probabilities with the conditional probabilities of words.
① Turn the email into word features
Extract words from the subject and body of the email and, where needed, apply lowercasing, tokenization, removal of unnecessary symbols and so on. In multinomial Naive Bayes, word counts are the typical input values.
② Compute the prior probabilities of spam and legitimate mail
If 20% of the training data is spam and 80% is legitimate mail, the prior probabilities are 0.20 and 0.80, respectively. A new email is not classified from these alone; they are updated with the word information.
③ Estimate the conditional probability of each word appearing in each group
If the word “free” appears relatively often among the word tokens belonging to spam emails, that word can contribute more to the spam score. In the multinomial model, probabilities are calculated relative to the total number of word tokens in each class.
④ Compare the scores for each class
Combine the prior probability with the conditional probabilities of the words in the email to calculate a score for each of the two groups, and assign the email to the group with the higher posterior probability. In practice, log probabilities are summed to handle very small numbers stably.
P(spam | words) ∝ P(spam) × P(words | spam). Applying the conditional independence assumption, the second term is approximated by the product of the per-word conditional probabilities. To get the full posterior probability, normalize by dividing by the sum of the scores of all classes.
Real-world example: email spam detection
We use hypothetical email data to separate legitimate mail from spam.
Of 1,000 hypothetical emails assumed to have been collected from a mail server, 800 are legitimate and 200 are spam. We use 800 for training and set aside 200, with their labels hidden, for the final evaluation. Both sets were split so that they keep the ratio of 80% legitimate and 20% spam.
| Item | Example value | Role |
|---|---|---|
| Email content | “Check out your free offer” | Raw text |
| Word frequencies | free 1 · offer 1 · check 1 | Input features |
| True class | Legitimate / spam | Target variable |
| Training data | 800 emails (640 legitimate / 160 spam) | Training and choosing settings |
| Test data | 200 emails (160 legitimate / 40 spam) | Checking final performance |
The performance figures that follow are hypothetical results for learning constructed so that the calculations are consistent; they were not obtained by training or evaluating a model on real email data.
What should you check before the analysis?
Check text quality and data leakage together.
| Item to check | What do you check? | Why does it matter? |
|---|---|---|
| Labeling criteria | Is the definition of spam consistent? | Label quality |
| Duplicate emails | Are identical emails duplicated across training and test? | Prevents overestimating performance |
| Preprocessing | How tokenization, stop words and letter case are handled | Consistency of features |
| Train/test separation | Were the vocabulary and feature selection built from the training data only? | Prevents evaluation leakage |
| Class proportions | Is the share of spam similar to the operating environment? | Interpreting precision and predicted probabilities |
| Personal information | Policy for handling personal information contained in emails | Safe use of data |
In what order do you run a Naive Bayes analysis?
Proceed step by step, from preprocessing the emails to checking classification performance.
① Prepare the raw emails and labels
Organize the email bodies, subjects and spam labels, and check for duplicates and mislabeled emails.
② Split into training and test data
Keep the 200 test emails aside until the very end, and compare settings with cross-validation within the training data.
③ Build text features from the training data
Fit the vocabulary on the training data and turn each email into a word-frequency vector. For English emails, apply lowercasing and tokenization rules consistently.
④ Learn the probabilities and the smoothing parameter
Apply multinomial Naive Bayes, and choose the Laplace smoothing parameter α by cross-validation on the training data.
⑤ Check the errors on the final test data
Separate cases where spam was missed from cases where legitimate mail was wrongly blocked, and review the confusion matrix, precision, recall and F1 score.
Why do we need Laplace smoothing?
It adjusts the estimates so that a word not seen in the training data does not make the whole probability 0.
Suppose a new email contains a word that never appeared in the spam emails of the training data. If conditional probabilities are calculated from observed frequencies alone, that word's probability becomes 0, and the spam score obtained by multiplying probabilities also becomes 0. Smoothing eases this problem by adding a small value to each word's estimated frequency.
| Setting | Meaning | Interpretation |
|---|---|---|
| α = 0 | No smoothing | Zero-probability problem for unseen words |
| α = 1 | Laplace smoothing | Adds a pseudo-count of 1 for each word |
| 0 ＜ α ＜ 1 | Weaker smoothing | Keeps the influence of training-data frequencies relatively large |
| Large α | Stronger smoothing | Evens out differences in per-word probabilities |
In multinomial Naive Bayes, the estimated probability of each word is (frequency of the word in that class + α) ÷ (total number of words in that class + α × vocabulary size). α is not simply set large; it is chosen by validation within the training data.
How do you interpret spam detection results?
Check not only how many spam emails were caught but also how many legitimate emails were wrongly blocked.
Suppose the results of applying the model to 200 hypothetical test emails are as shown below. Rows are the actual class and columns are the predicted class.
| Actual ＼ Predicted | Legitimate | Spam | Total |
|---|---|---|---|
| Legitimate | 154 | 6 | 160 |
| Spam | 12 | 28 | 40 |
| Total | 166 | 34 | 200 |
Accuracy is 91.0% (182/200), spam precision is 82.4% (28/34), and spam recall is 70.0% (28/40). The spam F1 score is about 75.7%. Accuracy alone looks high, but the model misses 3 out of every 10 spam emails, so depending on the goal you may need to adjust the threshold or the model.
How do you read class probabilities and decision thresholds?
An output saying an email is likely spam and the decision whether to block it in practice are separate decisions.
Even if a predicted probability of 0.80 is displayed, it is not automatically guaranteed that about 80% of such emails are actually spam. If Naive Bayes's conditional independence assumption does not hold exactly in reality, the output probabilities can be skewed toward overconfidence.
| Operating goal | Metric to prioritize | Decision to consider |
|---|---|---|
| Minimize blocking of important mail | Spam precision · false positives | Block spam more cautiously |
| Minimize missed spam | Spam recall | Catch more suspicious emails |
| Balance the two errors | F1 score | Compare precision and recall together |
| Downstream decisions based on probability | Calibration curve · reliability | Consider probability calibration |
If you are actually deciding on automatic deletion, quarantine or warnings, you must also consider the harm done when legitimate mail is lost. It is safer to set the classification threshold in a separate validation process, not on the test data.
Things to watch out for with Naive Bayes
Performance can change when the context in which words appear and the time period change.
Are the words really independent?
Words in an email are related to one another, and sentence order matters too. Because Naive Bayes simplifies these relationships, it can misjudge sarcasm, negation and figurative expressions.
Is it wrongly blocking legitimate mail?
False positives, where legitimate mail is classified as spam, can lead to important work messages being missed. You should not run the system by looking at accuracy alone.
Are the words at training time different from those in operation?
Spam wording and sending patterns keep changing. Re-measure performance on mail collected after some time has passed, and retrain if necessary.
How do you manage personal information and bias?
Emails can contain sensitive information such as names, contact details and contract terms. Avoid unnecessary retention and use the data in line with access rights and purpose.
How does it differ from other analyses?
Even when the classification target is similar, the basis for prediction and the way of learning differ.
| Method | Core principle | Difference from Naive Bayes |
|---|---|---|
| Naive Bayes | Combines feature probabilities by class | Conditional independence assumption |
| Logistic regression | Models class probabilities from input variables | Learns feature coefficients jointly |
| SVM | Optimizes the margin of the class boundary | Focus on the decision boundary |
| KNN | Uses the labels of nearby observations | Distance-based |
| Decision tree | Repeatedly splits the data by rules | Predicts with condition rules |
For text data, multinomial Naive Bayes is a typical choice suited to word frequencies. There are also Bernoulli Naive Bayes, which handles whether a word appears, and Gaussian Naive Bayes, which assumes a normal distribution for continuous variables, used according to the form of the data.
How do you report Naive Bayes results?
Describe the data split, preprocessing and misclassifications so they can be reproduced.
The report should include the number of emails and the share of spam, tokenization and vocabulary construction, the type of Naive Bayes used, the smoothing parameter, the cross-validation scheme, and the confusion matrix and evaluation metrics on independent test data. Depending on the operating purpose, false positives and false negatives should also be presented.
A multinomial Naive Bayes classifier using word frequencies as input variables was applied for email spam detection. Of 1,000 hypothetical emails, 800 were used for training and 200 for final evaluation. On the final test data, accuracy was 91.0% (182/200), spam precision 82.4% (28/34), recall 70.0% (28/40) and F1 score 75.7%. Six legitimate emails were misclassified as spam and 12 spam emails were classified as legitimate. These figures are a hypothetical example for explaining the method.
In a real paper or report, also state the α value chosen on the training data and the feature extraction method, and when comparing with other models, use the same test data and metrics.
Where can Naive Bayes be used?
It can be applied to classification problems in which features appear in different patterns across categories.
| Field | Classification goal | Example input data |
|---|---|---|
| Email management | Legitimate / spam | Subject and body words |
| Customer support | Inquiry type | Inquiry text |
| Document organization | News and report topics | Word frequencies in documents |
| Review analysis | Positive / negative | Review words and expressions |
| Basic quality control | Normal / abnormal category | Continuous sensor features (consider the Gaussian variant) |
Naive Bayes is a good simple baseline to start from, but it is not the best model for every classification problem. It is appropriate to compare it with other methods, considering the structure and context of the data and the cost of errors in operation.
Naive Bayes classifies emails by combining word occurrence patterns as probabilities.
The probability calculation is simple, but when you actually use the results you must also look at mail that was wrongly blocked.
In email spam detection, Naive Bayes calculates which group a new email is more likely to belong to based on the frequencies of words in mail that has already been classified. It has the advantage of training quickly by simplifying the relationships between words, but its classifications can waver when the same word means different things in different contexts or when new spam expressions appear. Smoothing reduces the problem of probabilities becoming 0 because of words not in the training data, but it does not eliminate every error.
So when evaluating Naive Bayes, you should look not only at overall accuracy but also separately at how many spam emails were missed and how many legitimate emails were wrongly blocked. In this hypothetical case, accuracy was 91.0%, but spam recall stayed at 70.0%. Which error matters more to reduce depends on the purpose of the actual service. Results can be trusted only when preprocessing and model settings are decided on the training data alone and performance is also checked on emails from a new period.
Naive Bayes estimates the probability that a new email is spam using Bayes' theorem and the conditional independence assumption. Word frequencies, prior probabilities and Laplace smoothing affect the classification, and preprocessing must be fit on the training data only. Do not look at overall accuracy alone; check spam precision and recall together with wrongly blocked legitimate mail.