AdaBoost

If an insurer reviews a huge number of claims every day, which ones should it check first? Should every large claim automatically be suspect? What about a claim that isn’t large, but where similar claims keep recurring over a short period? Only by looking at several clues together can you separate normal claims from those that may be fraudulent.

AdaBoost (Adaptive Boosting) builds a final classifier by training a simple classifier several times, giving larger weights to the cases misclassified at earlier steps. Even if each individual model’s judgment is somewhat inaccurate, the idea is to improve classification performance by combining them in sequence.

Whereas a random forest trains many trees independently and pools their results, AdaBoost feeds the errors of the previous model into the training of the next. So which cases a new classifier focuses on depends on the results of the classifiers before it. Care is needed, though, because it can end up concentrating on mislabeled answers and outliers as well.

AdaBoost can be used for insurance claim review, financial transaction classification, quality inspection and more. In this lesson we’ll use an example of potentially fraudulent insurance claims to look at how it works, sample weights, key settings, interpreting classification results, how it differs from other ensemble methods and what to check when applying it in practice.

AdaBoost belongs to ensemble learning. Instead of using a single complex model, it trains a relatively simple base estimator many times to reach a final decision. Typical binary-classification AdaBoost uses shallow decision trees, especially decision stumps that split on a single condition.

Cases the first classifier gets wrong are given larger sample weights, and cases it gets right relatively smaller ones. The next classifier is trained taking the changed weights into account. At the end, the classifiers’ judgments aren’t treated equally; they are combined with weights that reflect each classifier’s performance.

For insurance claims, it’s hard to judge fraud from the claim amount alone. There are legitimate claims for expensive treatment, and there are suspicious transactions that are small but repeated. You need to look at several features together, such as claim frequency, type of incident and previous claim history.

AdaBoost first classifies the patterns that are easy to spot, then increases the weight of the cases that weren’t separated and learns new patterns. In this process several simple rules combine to form more varied classification boundaries than any single rule. That doesn’t mean, however, that it outperforms a single model in every situation.

① Start by giving every claim the same weight

If the training data contain 1,600 claims, the typical approach gives each claim the same initial weight. Here the weight isn’t the claim amount; it indicates how much importance that observation gets during training.

② Train the first weak classifier

A simple decision tree predicts normal or classifies possible fraud based on, say, the claim amount or past claim frequency. Its results are compared with the true labels to compute the weighted misclassification rate.

③ Raise the importance of misclassified cases

In the next step, misclassified observations get relatively larger weights and correctly classified ones get lower weights. The weights are then renormalized. So the next classifier is more influenced by the cases the previous classifier couldn’t separate well.

④ Combine the judgments of several classifiers

The process repeats, and the final classification is decided by reflecting each classifier’s performance. Note the distinction: what gets a high weight here is not a claim that is likely fraudulent but a claim the existing classifiers got wrong.

Suppose we analyze 2,000 hypothetical claims received by an insurer. 1,800 claims were confirmed as legitimate, and 200 were confirmed as fraudulent after investigation. We use 1,600 for training and set the remaining 400 aside until the final evaluation. Each set was made up of 90% legitimate and 10% fraudulent claims.

Here we assume that fraudulent claims are not merely ones the insurer suspected but cases confirmed by the investigation criteria. The figures in the tables and graphs that follow are hypothetical results constructed to aid learning, not results from actually training a model.

① Prepare the claim history and confirmed labels

Organize the features available at prediction time, such as incident type, claim amount and claim frequency, along with the labels confirmed by investigation. If the data contain repeat records for the same customer or span consecutive periods, split them by group or by time as appropriate.

② Do preprocessing and validation within the training data

Missing-value imputation, encoding of categorical variables and any feature selection should happen inside the training data. When using cross-validation, fit the preprocessing only on the training part of each fold.

③ Choose the weak classifier and AdaBoost settings

Use shallow decision trees as the base classifier and compare the number of classifiers, learning rate and tree depth. If fraudulent cases are few, validate with a focus on precision and recall, not just accuracy.

④ Train sequentially while updating the weights

At each step, adjust the weights of the misclassified claims and train a new weak classifier. Through this repetition, the rules for separating fraudulent from legitimate claims are refined.

⑤ Check final performance on the sealed evaluation data

After training is finished, compute the confusion matrix, precision, recall, F1, PR-AUC and so on for the evaluation data kept aside. If you change the settings again after seeing the final evaluation figures, the independent evaluation loses its meaning.

Adding classifiers lets the model handle more error patterns, but it can also focus too much on mislabeled or unusual claims. Lowering the learning rate shrinks each classifier’s influence, so more rounds may be needed. It makes sense to validate the two values together.

Suppose the hypothetical final evaluation data of 400 claims gave the following results. Rows of the table are the true labels and columns are the model’s predictions.

Accuracy is 93.0% (372/400), fraud precision is 70.0% (21/30), recall is 52.5% (21/40) and the F1 score is 60.0%. Because 19 of the 40 actual fraudulent claims were missed, you can’t conclude from accuracy alone that detection performance is good.

AdaBoost can provide a score for classifying each claim and, in some implementations, an estimated probability. But a high score alone doesn’t confirm actual fraud. Nor should probability outputs be read as real probabilities of occurrence without a separate calibration check.

For instance, even if claim amount turns out to be an important variable, that doesn’t mean raising the amount causes fraud. It may also be better to use the model’s alerts to set priorities for further review rather than to reject claims automatically.

What happens if labels are wrong?

If a claim that is actually legitimate was wrongly labeled as fraudulent, AdaBoost may keep treating it as a hard case. Check the quality of the investigation results so that the focus of repeated training doesn’t drift toward wrong labels.

If legitimate claims vastly outnumber the others, won’t accuracy come out high?

In this example 90% of claims are legitimate. Predicting every claim as legitimate would already give 90% accuracy, so the figure of 93% alone doesn’t show that fraud detection is adequate. Review recall, precision and the actual cost of investigations together.

How should past claims by the same customer be handled?

If similar claims from the same person or incident are mixed across training and evaluation, the ability to predict future claims can be overestimated. Consider time-based splits or customer-level splits that match how the system will actually be operated.

Could the model be overly suspicious of certain groups?

It can pick up the investigation practices or existing biases in the training data as they are. For important decisions, put in place checks on error differences across groups, explainability, an appeals process and review by a staff member.

AdaBoost is also a boosting method, but not every boosting model uses the same computation. In particular, it differs from gradient boosting in its optimization perspective and how it handles error, so you shouldn’t treat them as the same model just because the names are similar.

The report should describe the numbers of legitimate and fraudulent cases, the labeling criteria, the splitting scheme, input variables, the structure of the base classifier, the number of classifiers and learning rate, and the cross-validation procedure. Performance should be presented on independent evaluation data where possible, and the types of error should be reported as well.

In actual research it’s good to add the final chosen hyperparameter values, the label definition, the time of evaluation, the classification threshold, and confidence intervals or results from repeated validation. You should also report, separately, whether suspected-fraud flags actually led to investigations.

A classifier’s performance depends on the input features, label quality, the degree of imbalance and the evaluation design. So when applying AdaBoost, it is sensible to compare it with a baseline model and other ensemble methods under the same data-splitting conditions.

AdaBoost learns better classification rules by feeding each classifier’s errors into the next step. In the insurance example, it combines clues from claim amount, frequency and incident type in sequence to separate legitimate claims from potentially fraudulent ones. But because it repeatedly focuses on cases the earlier models got wrong, it can learn spurious patterns when investigation labels are wrong or unusual observations are mixed in. The base classifier, number of rounds and learning rate should be chosen by validation within the training data.

Accuracy in the example was 93.0%, yet 47.5% of actual fraudulent claims were missed. So rather than judging the results by accuracy alone, check fraud recall together with false alarms on legitimate claims. Classification results are best used not as a ruling that confirms fraud but to prioritize cases needing further review. In operation, re-evaluate performance on newer data over time and keep checking for fairness and data leakage.