K-Means and K-Medoids

Suppose a museum analyzes the visit records of its members. Some members drop in briefly every time an exhibition changes, some come only occasionally but stay a long time, and others take part in education programs regularly. Without checking each of these differences one by one, can we group together members with similar usage patterns?

The method used for this is clustering. Clustering divides observations into groups based on how similar they are, without any given group labels to serve as the right answer.

K-Means and K-Medoids, two widely used distance-based methods, both find a specified number of clusters, but they define the center of each cluster differently. K-Means uses the mean of the observations as the center, while K-Medoids picks one of the actual observations as the representative point.

In this lesson we use museum members’ visit patterns as an example to cover the differences between the two methods and how they work, data preparation and standardization, choosing the number of clusters, interpreting the results table, the effect of outliers, and cautions for the analysis.

For example, looking at members’ number of visits, average length of stay and number of education programs attended together can reveal usage patterns that no single variable shows.

However, the groups produced by cluster analysis are not definitive customer types that already exist in the real world. The clusters can change depending on which variables you choose, how you scale them, and which distance and algorithm you use. The names and meanings of the clusters also have to be interpreted after the analysis, by looking at the characteristics of each group.

They are also similar in that they assign observations to the nearest center and update the centers that represent the clusters. The difference lies in what they call a center and which objective function they minimize.

K-Medoids is not fully robust to outliers either. If there are many extreme observations or the true cluster structure is complex, the results can change a great deal.

Instead of dividing members in advance by age or where they live, the goal is to find people whose actual usage behavior is similar.

In this analysis, the clusters are an exploratory grouping for describing visitor behavior, not a result that automatically predicts members’ satisfaction or future return visits.

In this example, each member is one row, and the four behavioral variables are the columns. You also need to be clear about how stay length and spending were calculated for members with almost no visits.

For example, if you arbitrarily record 0 minutes as the ‘average stay per visit’ for a member who never visited, they may become indistinguishable from members who actually stayed 0 minutes.

You should also check for missing values, duplicate members, abnormally long stays, negative spending due to refunds, and so on. Rather than automatically deleting outliers, it is important to first check whether they are genuine usage records. If you want to include categorical variables as well, you may need a distance suited to those variables or a different clustering method, rather than plain K-Means.

If average monthly visits range from 0 to 8 and stay length ranges from 20 to 240 minutes, then using the Euclidean distance on the raw data gives relatively large weight to differences in stay length.

So if you want the variables to count roughly equally, you can consider z-score standardization, which transforms each variable to have mean 0 and standard deviation 1.

Standardization here is a process of putting the variables on the same mathematical scale, not a demonstration that all the variables really are equally important. Depending on the purpose of the analysis, you can give certain variables more weight, or consider a robust scaling method instead of the mean and standard deviation, which are sensitive to outliers.

If you apply the clusters to new data, you must use the transformation parameters calculated from the training data as they are.

It then updates each centroid to the mean of the observations in its cluster, and repeats the assignment and update steps.

The usual K-Means objective is to minimize the sum of squared distances between the observations in each cluster and its centroid. Here μ_k is the mean vector of the observations in cluster k. Because it minimizes squared distances between observations and centroids, observations far from a centroid can have a relatively large influence on the result.

Because K-Means can converge to different solutions depending on the initial centroids, real analyses usually use K-Means++ and multiple initializations (n_init). Even when the iterations converge, there is no guarantee that the globally best clustering has been found.

For example, if you need to describe a cluster by finding one representative member, you can use an actual member’s record as the representative point rather than an ‘average member’ who does not exist in the data.

Each observation is assigned to its nearest medoid. A well-known algorithm is PAM (Partitioning Around Medoids), which can reduce the total distance cost by swapping medoid candidates.

It has the advantage of letting you choose representative points from actual members, but calculating the distances between all observations and searching through candidates can be demanding when the data are very large.

With K=1, the K-Means centroid is the arithmetic mean, so it is (2+3+4+5+30) ÷ 5 = 8.8 visits. Most members visited 2–5 times, but the value of 30 pulled the center to the right.

In one-dimensional K-Medoids that minimizes the sum of absolute distances, on the other hand, the actual observation 4 is the medoid. The sum of absolute distances to 4 is 2+1+0+1+26 = 30, which is smaller than when any other observation is chosen as the representative point.

This example shows that the two methods can respond differently to extreme values. In real data with many variables and many clusters, however, the differences are more complex, so you should not conclude that one algorithm always gives better results.

To choose a suitable K, compare several values, and rather than deciding on a single measure, it is good to check the stability and interpretability of the clusters as well.

In K-Means, the within-cluster sum of squared distances (inertia, or WCSS) decreases as K increases. The elbow method, which plots this for each K, looks for the point where the decrease levels off. A clear bend may not appear, however.

The silhouette score compares the distances within an observation’s own cluster with the distance to the nearest other cluster. It generally ranges from -1 to 1, and higher values indicate that the clusters are relatively well separated. Be careful interpreting it, though, for clusters that are not round or are unbalanced.

In these results, K=3 has the highest score. But rather than automatically choosing 3 clusters just because the score is highest, you should also check the size of each cluster and whether the actual behavior patterns are meaningfully distinct.

To make them easier to understand, the numbers in the table are cluster means shown back in the original units.

Cluster 1 has many members who visit often but stay relatively briefly, so it can be described as ‘short, frequent visits’. Cluster 2 visits less often but stays a long time per visit and spends more at the shop, so a name such as ‘long, leisurely visits’ could be considered. Cluster 3 attends noticeably more education programs, so it can be interpreted as ‘program participants’.

These names are not objective personality types discovered automatically by a statistical model, but descriptions the researcher gave based on each cluster’s descriptive statistics. In practice, you should also check the variance within each cluster and the differences among its members.

K-Means usually minimizes squared Euclidean distance, while K-Medoids minimizes the sum of the chosen distance, so you should not judge which method is better by directly comparing their different objective function values.

When comparing, you can look at silhouette scores calculated with the same distance, imbalance in cluster sizes, how much the results change with the starting values, stability when outliers are included or excluded, and how interpretable the results are in practice.

Also, a K-Means centroid is a hypothetical visitor profile summarized by the means of the four variables, whereas a K-Medoids representative point is the record of one actual member. If you need to present a representative member’s behavior as a case, K-Medoids can be convenient, but if computational efficiency on large data comes first, it may be practical to try K-Means first.

Clusters based on visits alone can look different from clusters based on visits, stay length and program participation together. If several variables carry much the same information, certain behavioral traits may be overrepresented.

Check what standardization and the distance mean.

If you compare variables with different units on their raw scales, the variables with larger ranges can dominate the distance. Standardization itself can also be affected by outliers, so choose a method that suits the data.

Review the number of clusters and the initialization results.

K-Means can give different results depending on the initial centroids, and K-Medoids can also be affected by the initial medoids or the search method. Comparing results across several initializations and numbers of clusters helps you gauge stability.

Check whether the clusters are actually separated.

If the silhouette score is low or several clusters have nearly the same characteristics, the data may have no distinct groups. Not every dataset necessarily contains meaningful clusters.

Don’t interpret clusters as causal effects or predictive performance.

Even if members in the program-participant cluster return more often, you cannot conclude that taking part in programs increased return visits. And cluster analysis alone does not guarantee accurate predictions of future behavior.

If you compared algorithms, you should also describe how stable the results were under the same evaluation criteria.

In real studies, you should also report the evaluation measures for each algorithm chosen, the spread within each cluster, and the stability of the results.

K-Means minimizes squared distances to the mean of each cluster, while K-Medoids chooses representative points from the actual observations and minimizes the sum of distances. As a result, they differ in what the center means, how they respond to outliers, and their computational cost.

Whichever method you choose, the quality of the results depends less on the algorithm’s name than on the choice of variables, standardization, the distance measure, the number of clusters, and the stability and interpretability of the results. Clusters are an exploratory classification found in the data, so before putting them to use, it is good to check how each group behaves and whether the grouping is reproduced consistently.