Clustering and customer segmentation

What is clustering and customer segmentation?

Clustering is grouping records so the members of a group resemble each other more than they resemble the rest, without anybody writing down the groups in advance. Google's clustering course calls it an unsupervised technique designed to group unlabeled examples based on their similarity to each other. You get back a cluster number per record. What that number means is your problem.

Customer segmentation is the business practice that pays for most clustering work inside a company. You want to stop treating 2,400 customers as one audience and start giving different attention to groups that behave differently.

Most segmentation in daily use is not a clustering model at all. In Dynamics 365 Customer Insights the everyday segment is a rule you write yourself out of conditions on attributes from the customer and purchase tables, such as customers who ordered in March and spent above an amount you set. Clustering is what you reach for when you do not know what the rule should be.

That is also the line between clustering and classification. In text classification you write the list of labels first and the model puts each item into one of them, so every item has a right answer and you can measure precision and recall against it. In clustering nobody wrote the list. The algorithm proposes the groups and you name them afterwards, or you fail to and the exercise stops there.

The algorithms you will meet

k-means. You say how many groups you want. It drops that many centre points into the data, assigns every record to the nearest one, moves each centre to the middle of what it caught, and repeats. Scikit-learn is blunt about the constraint: k-means requires the number of clusters to be specified. It is fast and it always converges, but its objective assumes clusters are convex and isotropic, so scikit-learn notes it responds poorly to elongated clusters or irregular shapes. On customer data that shows up as round blobs cut out of a cloud that never had round blobs in it.

Hierarchical clustering. Every customer starts as their own group, the two closest groups merge, and that repeats until everything sits in one group. Scikit-learn calls the resulting tree a dendrogram, with the root the single cluster gathering all samples and the leaves the clusters holding one sample each. You cut it at whatever height leaves a workable number of groups, so three, four and five are all readable from one run. Reading the tree by eye works on a few hundred rows and stops working on tens of thousands.

DBSCAN. Scikit-learn describes the view it takes: clusters are areas of high density separated by areas of low density, so the clusters it finds can be any shape, as opposed to k-means. It never asks how many groups you want, because you set a neighbourhood radius and a minimum number of neighbours instead. And it is allowed to say a record belongs nowhere: a point far enough from any dense area is labelled noise, while k-means has to force your one strange customer into a group and let it pull the centre along. The radius has to be chosen for your data. Too small and almost everything comes back as noise, too large and the clusters merge into one.

Clustering on embeddings. Free text has no numeric columns to measure distance on, so you turn each piece of text into an embedding first: a list of numbers where distance means relatedness. OpenAI's guide puts it that way and lists clustering, grouping text strings by similarity, among the use cases. That is the route from 4,000 free-text complaint fields to twelve recurring themes nobody had written down.

Preparation decides the result

Scale the columns, or you have clustered one of them. Hand k-means a table with revenue running from 200 to 400,000 euro next to an order count running from 1 to 180, and it measures distance almost entirely in euro. Two customers 10,000 euro apart are 100 million apart in squared distance on that column, while two customers 20 orders apart are 400 apart on theirs. Google states the requirement plainly, that similarity calculations need the features to have the same scale, and the usual fix is a z-score per column. Skip it and your four groups are revenue quartiles with extra steps.

Choose features that match the question. Spreading sales attention needs order rhythm, assortment breadth and recency. Planning delivery routes needs location and volume. Put both sets in one run and the groups mix the two questions and answer neither.

Take the identifiers out. Customer number, rep id, article code, postcode stored as a number. The distance calculation will happily use them, and the distance between customer 1002 and customer 1003 means nothing. Postcode is the one that slips through, because it looks geographic and it is ordered, so a group forms around a numeric band rather than a region anyone would draw on a map. Outliers deserve the same pass beforehand: Google warns that centroids can be dragged by outliers, or that outliers might get their own cluster instead of being ignored, and recommends removing or clipping them first.

How many groups, and how you decide honestly

The elbow comes first. Run k-means for k from 2 to 8 and plot the within-cluster sum of squares, which scikit-learn calls inertia and describes as a measure of how internally coherent clusters are. It falls as k rises, by construction, and you look for the bend where it stops falling fast. Sometimes the bend is obvious. Often the line is a smooth curve and three people read three different bends off it.

The silhouette comes second. Per record it is (b - a) / max(a, b), where a is the mean distance to the other members of its own cluster and b the mean distance to the nearest cluster it is not part of. It runs from -1 to 1: near 1 the record sits comfortably inside its group, near 0 it sits on a border, and a negative value means a different cluster suited it better. Average it and compare across values of k. Scikit-learn's own example is honest about how far that gets you, saying silhouette analysis is more ambivalent in deciding between 2 and 4 on its data, and it gives a reading worth copying: a value of k is a bad pick when it produces clusters scoring below the overall average and when the cluster sizes fluctuate wildly.

The test that settles it is neither of those. Put the group profiles in front of your sales manager and ask two questions per group. Can you describe this group in one sentence, and what would you do differently for it starting tomorrow. A group that gets a name and an action is a group. A group that gets neither is a colour on a chart, and folding it into its neighbour costs you nothing.

RFM, the baseline worth beating

Before any of the above there is a segmentation that needs three columns and no model. RFM scores every customer on recency (how long since the last order), frequency (how many orders inside the period) and monetary value (how much they spent). Optimove's version splits each of the three into four tiers, which gives 64 combinations, and you collapse those into a handful of groups people can name.

For most wholesalers and installers that beats a clustering model, for reasons that have nothing to do with accuracy. It is SQL, so it reruns every night on the same definition. A rep can read why a customer is in a group: last order 14 months ago, four orders ever. And it stays comparable, so this month's group means what last month's group meant. Rerun a clustering next quarter and cluster 3 can be a different set of people wearing the same number.

Optimove names the limit as well: the model looks at three factors and may be excluding others that matter as much, and it is historical by nature. Move to clustering when you have features RFM cannot hold, such as assortment breadth, seasonality, margin or how much service a customer costs you.

A worked example: 2,400 B2B customers

A technical wholesaler wants to know how to spread sales attention. Five features per customer, all over the last twelve months: revenue, number of orders, days since the last order, number of distinct article groups bought, and average discount percentage. Customer number, postcode and rep id stay out. All five get a z-score, so a euro and an order weigh the same going in.

The run covers k from 2 to 8. Inertia bends somewhere between three and five without committing to either. The average silhouette reads 0.41 at k = 3, 0.38 at k = 4 and 0.29 at k = 5, so k = 4 goes to the sales meeting.

Two of the four groups come back with the same description: three to eight orders a year, one or two article groups, nothing unusual in recency. They differ mainly on average discount, 4.1 percent against 5.6, and nobody in the room would run a different action off that gap. They get merged. Four honest clusters, three usable groups.

Core accounts, 180 customers, 62 percent of revenue. Twenty orders a year or more, across most of the assortment, last order inside the month. A named owner per account, a quarterly review, and an alert when the order rhythm breaks, because one of these customers going quiet for a quarter costs you more than a dozen customers from the third group order in a whole year.

Steady but narrow, 690 customers, 29 percent of revenue. Three to eight orders a year, almost all of it inside one or two article groups. They get called about the article groups they have never bought, and that campaign is measured on whether their number of distinct article groups goes up, not on how many people opened the mail.

Occasional and lapsing, 1,530 customers, 9 percent of revenue. One or two orders, the last of them more than nine months back. No rep time, a mail flow and the webshop instead, and a decision after two quarters about how much of this file is worth keeping warm.

Two of those three groups would have fallen out of a plain RFM run as well. The clustering earned its place on assortment breadth, the feature that separated the second group from the first and the one RFM does not carry.

What to watch out for with clustering and segmentation

A cluster is not a cause. The model found customers who order rarely and spend little. It did not find out why, and it cannot tell you whether a campaign will move anybody from the third group into the second. That question needs a test with a control group, and the clustering only tells you who to run it on.

The groups drift. Your customer base changes, so a segmentation from eighteen months ago describes a company that no longer exists. Rerunning is not free either, because cluster numbers do not carry over and group 2 in the new run is not group 2 in the old one. The practical move is to freeze the groups as readable rules once you have them, on thresholds like revenue above X and more than Y orders, and rerun the clustering once a year to check whether the rules still fit.

A segmentation nobody acts on is a slide. The failure has a recognisable shape: a deck with four coloured groups, an appreciative meeting, and a CRM where nothing was tagged and no campaign changed. Decide the action per group before you present the groups, and write the group back into the CRM as a field so it can drive a list.

Segmenting people is profiling. Article 4(4) of the GDPR defines profiling as any form of automated processing of personal data consisting of the use of personal data to evaluate certain personal aspects relating to a natural person, and the definition names economic situation, personal preferences, interests, reliability and behaviour. Grouping identified customers on their buying behaviour is inside that, so it needs a lawful basis and a line in your privacy statement. Business customers held under a company number are outside personal data; a sole trader and a named contact person are not. The AI Act does not make ordinary customer segmentation high risk on its own, but its Annex III list of high-risk uses names evaluating the creditworthiness of natural persons and risk assessment and pricing for life and health insurance, so a segmentation feeding a credit limit is a different conversation from one feeding a mail campaign.