Hyperparameter Tuning

Suppose you built a random forest model to predict whether customers will cancel a service. With 100 trees the accuracy was 87%, but when you increased the number of trees to 300 and adjusted the maximum depth, accuracy rose to 89%.

So would performance improve further with 1,000 trees? What would happen if you increased the maximum depth or changed other settings?

In machine learning, even with the same algorithm, predictive performance can differ depending on the training settings. But making the settings larger or the model more complex does not necessarily give better results. It may instead cause overfitting or simply increase training time.

The process of comparing settings like these to find a suitable combination is called hyperparameter tuning.

Hyperparameter tuning is used with many machine learning algorithms, including random forests, gradient boosting, neural networks (MLP), support vector machines (SVM), and ridge and lasso regression.

In this article, we look at hyperparameters from the basic concepts through the main search methods, the relationship with cross-validation, a real customer churn prediction example, computational budget, overfitting and data leakage, and interpreting the results.

For example, in a random forest you can set the number of trees, the maximum depth, the minimum samples per leaf and so on. In a neural network you can adjust the hidden layer structure, learning rate, regularization strength and so on. These settings are called hyperparameters.

Hyperparameters are generally specified before training the model or chosen through a separate search procedure. By contrast, values estimated directly from the training data, such as regression coefficients or neural network weights, are called model parameters.

The goal of tuning is not simply to maximize the score on the training data. It is to find settings that give good predictive performance on new data as well. So in hyperparameter tuning, what matters is not only performance on the training data but performance obtained from cross-validation or separate validation data.

Parameters are values the model estimates from the data during training, such as the regression coefficients of linear regression or the weights and biases of a neural network. Hyperparameters, on the other hand, are settings that determine how the model learns or how it is structured.

For example, in ridge regression the regression coefficients are parameters, and the regularization strength applied to them is a hyperparameter. If the hyperparameter changes, the learned parameters and final predictions can differ even with the same data. So distinguishing the two concepts is important for understanding model training and tuning.

You can build a predictive model with default settings, but there is no guarantee those settings suit every dataset. For example, data on 500 customers and data on 1 million customers differ greatly in size and structure. Likewise, a problem with 10 predictors and one with 1,000 may call for different levels of model complexity.

Hyperparameter tuning is used to adjust how the model learns in light of these differences. The main goals are as follows.

• Improve predictive performance on new data
• Reduce overfitting and underfitting
• Control model complexity
• Manage training time and computational cost
• Optimize the performance metric that suits the real-world task

However, not every problem needs elaborate tuning. If data are limited or a simple model already performs well enough, default settings or a limited search can give suitable results.

For example, if you set a random forest's maximum depth too large, individual trees can become complex as they split the training data very finely. Conversely, if you set it too small, the trees may not learn important relationships well enough.

In an MLP, increasing the number of hidden layers and neurons lets it represent complex relationships, but training time increases and the risk of overfitting can rise. So in hyperparameter tuning, it is important to understand how each setting affects the model and to choose an appropriate search range.

For example, suppose you compare the following settings for a random forest.

• Number of trees: 100, 300
• Max depth: 5, 10, 20
• Min samples per leaf: 1, 5

In this case there are 2 × 3 × 2 = 12 possible combinations. Grid search trains all 12 combinations and compares their validation performance.

Grid search example

The table shows only some of the 12 combinations.

The advantage of grid search is that it compares every specified candidate combination without exception. But as the number of hyperparameters to search grows, the number of combinations increases rapidly. For example, specifying 10 candidate values for each of five hyperparameters creates 100,000 combinations in total.

So grid search can be suitable when the search range is relatively small or when you want to compare a particular region in fine detail.

Whereas grid search compares every candidate combination, random search selects and evaluates a set number of combinations. For example, suppose you set the number of trees between 100 and 1,000 and the maximum depth between 3 and 30. Random search picks several combinations from this range and computes the validation performance of each.

Random search example

In this example, the fourth combination had the highest validation ROC-AUC. Because random search does not evaluate every combination, it has the advantage of covering a wide search space in limited time.

It can also be better than grid search at exploring diverse values when it is hard to know in advance which hyperparameters matter for performance. But because combinations are chosen at random, it may not explore high-performing regions thoroughly. So the number of trials and the hyperparameter ranges must be set appropriately.

Grid search and ordinary random search do not directly use the performance of previously evaluated combinations when choosing the next candidate. Bayesian optimization, by contrast, estimates the characteristics of the search space from the combinations already evaluated and their performance, and further explores regions where good performance is expected or uncertainty is large.

For example, if good performance appeared when the random forest's maximum depth was between 10 and 15, later searches can examine that range in more detail. But it does not search only the ranges that already gave good results. It also checks regions not yet evaluated enough, allowing for the possibility of finding a better combination.

So the balance between exploiting regions that currently perform well and exploring new regions is important.

Key features

Bayesian optimization can be useful when training and evaluating a single model is expensive. However, it requires a separate computation to choose the next candidate from the search results, and the results can vary with how the search space is set up and with the optimization algorithm.

Also, the name Bayesian optimization does not necessarily mean a Bayesian statistical model is used as the final predictive model. Bayesian optimization is a method for searching hyperparameters, and the final predictive model can be any of many algorithms, such as random forest, XGBoost or MLP.

No method always produces the best result. For example, if there are only two hyperparameters to tune and few candidate values, grid search can be simple and sufficient. If there are many settings to tune and the search range is wide, random search can be efficient.

If training a single model takes a long time, you can consider Bayesian optimization, which uses previous search results. What matters is not the name of the algorithm but setting up the search space, computational budget and evaluation method appropriately.

Of all customers, 650 canceled the service and 4,350 kept it, for an overall churn rate of 13%. The analysis used tenure, monthly charge, contract type, number of recent service uses, number of customer service contacts and more. The full data were split into 4,000 customers for training and 1,000 for final evaluation.

Assume the random forest's hyperparameters were searched using stratified 5-fold cross-validation within the training data. Because the share of churned customers—the event of interest—is low, PR-AUC was set as the tuning criterion.

Hyperparameters searched

Assume random search was used and a total of 30 combinations were evaluated. Each combination was evaluated on the same cross-validation splits so their performance could be compared.

In these results, the combination with 320 trees, a maximum depth of 12 and a minimum of 5 samples per leaf had the highest mean PR-AUC, 0.518. So you can choose this combination as the final candidate.

But the PR-AUC difference between #1 and #2 is only 0.003. Such a difference may be very small compared with the variation due to how the validation folds are composed.

So when performance differences are small, rather than simply picking the combination with the highest score, it is good to also consider model complexity, training time and the stability of the scores. Also, finding the best combination within the search range does not mean it is globally optimal among all possible hyperparameter combinations.

In a wider search space or with different data splits, other combinations may perform well.

For example, suppose the following results were obtained.

In this example, the tuned model's mean PR-AUC was 0.046 higher.

So after choosing the final model, it is good to check its performance on separately held-out test data. The tuned model may indeed perform better on the final test data, but the difference may be small, or it may even be lower than the default model.

Such results are also important for understanding the limits of the tuning process and the model's generalization performance.

For example, if a model with 100 trees were evaluated on customer group A and a model with 300 trees on customer group B, differences in data composition could make their performance hard to compare directly. Cross-validation helps by using the same split structure to compare several settings.

With 5-fold cross-validation, each hyperparameter combination is trained and validated five times and the average score is calculated. For example, evaluating 30 combinations may require 150 model fits in total.

This table assumes each combination is trained once on every fold. The actual number of computations can vary with early stopping, the search algorithm and whether additional refitting is done.

Cross-validation lets you check model performance across several data compositions, but if you choose the combination with the highest validation score, that score itself can contain selection bias. So it is good to evaluate the performance of the model selection procedure separately, using final test data or nested cross-validation.

Especially when training a model takes a long time or there are many hyperparameters to search, computational cost grows quickly. So before tuning, it is important to set limits such as the available time or number of evaluations. This is called the search budget.

For example, you can use search strategies like the following.

The numbers of combinations above are examples for illustration and do not imply actual run times or recommended settings.

Increasing the search budget lets you evaluate more candidates, but it does not always find a better model. For example, if the data lack the information needed for prediction, predictive performance may not improve much no matter how many hyperparameters you adjust.

Also, computational cost increases as the search budget grows, so consider the size of the performance gain and the cost together. In practice, rather than searching a very wide range from the start, you can also evaluate the default settings first and adjust the search range around the important hyperparameters.

For example, if you search a random forest's maximum depth only between 3 and 5, you cannot find a better setting outside that range even if one exists. Conversely, if the range is too wide, you may not examine promising regions sufficiently within a limited budget.

So it is good to set the search range with the algorithm's characteristics and the data size in mind.

Continuous hyperparameters

Settings that take continuous values, such as learning rate or regularization strength, need a carefully chosen search range. For example, if you search a learning rate from 0.0001 to 0.1, rather than picking candidates only at even intervals you can use a log scale to compare values of different magnitudes.

Integer hyperparameters

Number of trees, maximum depth, number of neurons and similar settings take integer values. You can evaluate several candidates for these within a reasonable range.

Categorical hyperparameters

Some settings choose one of several options, such as the activation function, optimization algorithm or split criterion. For example, you can compare ReLU and Tanh as the activation function of an MLP, or a linear kernel and an RBF kernel for an SVM.

These settings can be included in a single search space, but which combinations are actually supported depends on the algorithm and library.

For example, suppose you evaluated 1,000 setting combinations on the same validation data. Some of them may score high on the validation data by chance even if their real generalization performance is not outstanding. The more combinations you search, the greater the chance of picking such a chance result.

This phenomenon can be described as overfitting to the validation data, or model selection bias. To reduce it, consider the following.

• Use cross-validation within the training data during tuning
• Keep the final test data held out until model selection is finished
• Use nested cross-validation if needed
• When performance differences are small, also review model complexity and stability
• Keep the evaluation metric and data-splitting method consistent throughout the search

In particular, if you repeatedly revise hyperparameters based on the final test score because it came out low, the final test data also become part of the model selection process. So it is important to keep tuning and final performance evaluation clearly separate.

For example, if you impute missing values or standardize on the full data before cross-validation, information from the validation folds can leak into training. In hyperparameter search in particular, such leakage can affect the evaluation of every candidate combination.

So in proper tuning, split the data first and estimate the preprocessing parameters within each training fold.

Proper tuning procedure

1. Split the full data into development and final test sets.
2. Build cross-validation folds within the development data.
3. In each training fold, learn the preprocessing, such as missing-value imputation and standardization.
4. Train a model with each hyperparameter combination.
5. Compute performance on the corresponding validation fold.
6. Combine the validation results to choose the final settings.
7. Retrain on the full development data with the chosen settings.
8. Check final performance on the held-out test data.

In practice, building preprocessing and the model into a single pipeline helps prevent data leakage. Also, for data with repeated observations of the same customers or a time order, consider group-based or time-based splits instead of simple random cross-validation.

For example, if you tune a customer churn model on accuracy, a setting that correctly predicts the majority class—retained customers—may be chosen. If you tune on PR-AUC instead, a setting that provides prediction rankings useful for identifying churned customers may be chosen.

So it is important to choose a metric that fits the purpose of the analysis before starting the search.

Classification problems

Regression problems

For example, if missing customers who will default on a loan is costly, you might give recall more weight. In a marketing campaign that offers customers an expensive benefit, on the other hand, reducing the cost of false positives may be important.

So the evaluation metric should be chosen with an eye not only to statistical performance but also to how the predictions will actually be used.

But if the performance differences are very small, it is good to consider computational cost and model complexity as well. For example, suppose the following results were obtained.

Model B has the highest mean PR-AUC at 0.523. But its difference from Model A is only 0.003, while its training time is six times longer. In such a situation, choosing Model A may be reasonable, considering the variability of validation scores and the real operating environment.

Especially if the model must be retrained often or computational cost is limited, operational efficiency can matter more than a small performance difference. So the goal of hyperparameter tuning is not to choose the most complex, highest-scoring model at all costs, but to find a suitable balance among performance, stability and cost.

Using the customer churn prediction example from earlier, you could write it as follows.

In a real analysis, you should also present the performance obtained on the final test data. Recording the hyperparameter ranges searched, the number of trials, the random seed and the model's main settings also improves the reproducibility of the analysis. When performance differences are small, it is good to also check the score differences and variability among the top candidates.

It is used in many kinds of machine learning analysis where the model's training settings can be adjusted.

Optimizing classification models

It can be used to improve the classification performance of logistic regression, KNN, SVM, decision trees, random forests, gradient boosting, MLP and more.

Optimizing regression models

You can search for settings that reduce the prediction error of ridge, lasso, elastic net, random forest regression, gradient boosting regression, MLP regression and more.

Controlling complexity and overfitting

By adjusting a decision tree's maximum depth, the regularization strength of regularized regression, an MLP's hidden layer structure and so on, you can find a balance between model complexity and generalization performance.

Automated machine learning (AutoML)

Tuning is also an important process in AutoML, which automatically compares many algorithms and hyperparameter combinations. AutoML can be configured so that, once the user sets the search range, evaluation metric, time budget and so on, it compares many models and settings to find promising candidates.

However, AutoML does not evaluate every possible setting or always find the optimal model. The results depend on the search range, computational budget and characteristics of the data.

The main search methods are grid search, random search and Bayesian optimization. Grid search compares every specified combination, random search explores many combinations at random within a limited budget, and Bayesian optimization uses previous evaluation results to choose the next candidate.

All of these search methods can be used to improve a model's predictive performance, but no method is always the best. The right method depends on the size of the data, the search space, model training time and the computational budget.

Also, there is no guarantee that the combination with the highest tuning score is actually the best model. So compare combinations using cross-validation, and use the final test data only after model selection is finished. In particular, it is important to prevent data leakage and to choose an evaluation metric that fits the purpose of the analysis.