Recommendation system

What is a recommendation system?

A recommendation system picks a handful of items out of a large catalogue and puts them in front of one person, in a specific order. That is the whole job. The block under a product on a webshop, the next article on a news site, the spare part that fits the machine somebody just ordered, the three extras a sales rep sees while building a quote: the catalogue changes, the problem does not.

Search starts with the customer telling you what they want. A recommendation system has to guess, from what this person did before and what similar customers did.

The four approaches you will meet

Popularity. Show everyone the same best sellers, possibly split by category. Amazon's Personalize documentation says to train a Popularity-Count model, which recommends the top K most popular items, purely as a baseline to compare against. Skip that and you cannot say afterwards whether your model beat the shop's own top-ten list.

Content-based filtering. Describe every item with attributes (category, brand, price band, material, compatibility) and recommend items that resemble what this customer already liked. Google's machine learning course gives it two advantages: it needs no data about other users, and it can recommend niche items very few other people want. The attributes are hand-engineered, so the model is only as good as the fields you filled in, and it barely expands on what the customer already showed interest in.

Collaborative filtering. Forget what the products are and look only at who interacted with what. "Customers who bought this also bought that" is the shape everyone recognises. Google credits it with needing no domain knowledge and with serendipity, since it can help someone discover a new interest.

Embedding-based ranking. The modern version puts customers and products in the same vector space, so a recommendation becomes a nearest-neighbour lookup: which product vectors sit closest to this customer's vector?

Content-based or collaborative: what each needs before it works

Content-based filtering needs a product catalogue with real attributes and almost nothing about behaviour. Google's retail documentation is explicit about the extreme case: its Similar Items model requires only information from the product catalog, and no user events are required. With three-word titles and empty attribute fields it fails no matter how much traffic you have.

Collaborative filtering is the mirror image. It needs a history of who did what and does not care what the products are. Five years of orders and a product table holding nothing but a code and a price is enough for it, while a shop that opened last month gives it nothing to learn from. Most systems in production run both.

What data you need before you start

A transaction history with a customer id on every line comes first, because plenty of webshops book guest orders without linking them to anything and without an id you cannot tie two purchases to the same person. Then product attributes worth matching on, filled in across the whole catalogue rather than for the 200 articles somebody cleaned up last year. Then views, not only purchases: a purchase is a strong signal and a rare one, while page views and add-to-carts arrive far more often.

Amazon puts numbers on the floor: at minimum 1000 item interactions and 25 unique user IDs with at least two interactions each before it trains anything, and at minimum 50,000 interactions from at least 1,000 users for what it calls quality recommendations. A shop with 400 orders a year will not get a model worth deploying.

Cold start: new customers and new products

Cold start is the state where recommendations are weak because the history is not there yet. Amazon describes it as the phase when recommendations are less relevant due to a lack of historical user data. It runs in two directions.

A new customer. Fall back on popularity for whatever segment you can infer, and start personalising within the session the moment they click. Device type, region, referring campaign and time of day are known on the first page view, and Amazon lists that kind of contextual metadata as a way to shorten the cold-start phase.

A new product. Nobody has bought it, so collaborative filtering has never seen it. Google states the mechanism plainly: if an item is not seen during training, the system can't create an embedding for it and can't query the model with this item. The answers are content-based similarity on its attributes, and exploration, where you reserve part of the slots for items with little interaction data so they get a chance to collect some. Amazon exposes that as an exploration weight with a default of 0.3.

Measuring whether it works

This is where recommendation projects quietly go wrong, and the failure has a recognisable shape: the offline metrics improve, the model ships, and revenue does not move.

Offline metrics come from replaying history. You hide part of each customer's past, ask the model to predict it, and score how often the hidden item turns up in the top five. Online metrics are what happens when real customers see the block: clicks, add-to-carts, order value, returns. Amazon's documentation notes that you are the one responsible for generating and recording those.

An offline score compares two models on the same data. It is not a promise about behaviour: a model can rank a customer's next purchase higher and still sell nothing extra, because the customer was going to buy that item anyway and the block took the place of something else they might have added. So split the traffic and measure the outcome you care about against two baselines: plain popularity, and the list your merchandiser would have written by hand.

A worked example

A webshop with 6,500 customers, 3,000 articles and around 12,000 orders a year wants a "customers also bought" block on the basket page. The offline test holds back each customer's last order and asks both models for four items. Plain best sellers hit the held-back article for 210 customers, a hit rate of 3.2 percent. A simple item-to-item collaborative model hits 495, or 7.6 percent. Well over twice as good on paper, and that is the number that ends up on the slide.

The A/B test over four weeks tells a smaller story. Each arm sees 10,000 basket pages. The popularity block gets 240 clicks and 96 items added, at an average of 34 euro. The model gets 305 clicks and 118 items added, at an average of 31 euro, because it is better at finding a matching accessory than an expensive one. That is 3,658 euro of added basket value against 3,264: a difference of 394 euro in four weeks, or roughly 5,000 euro a year.

Only the second number tells you whether the project earns its maintenance, and with a few hundred clicks per arm it is worth checking against chance before you celebrate.

Most SME cases are not Netflix

The tooling is built around consumer platforms with millions of users and a catalogue where taste drives the choice. Take a technical wholesaler instead: 4,000 customers, 20,000 articles, buyers who order the same things on a rhythm. They do not need a neural model to discover hidden taste. They need the query that says this customer ordered filter X four times in eighteen months, always about sixty days apart, and it has now been ninety-five days. A rep can act on that this afternoon, it is written in SQL, and on that catalogue it will out-earn a collaborative model for a long time.

Recommendation models start to pay when the catalogue is too large for anyone to hold in their head, when new customers keep arriving, and when nobody stands between the ranking and the customer. A language model fits alongside that, writing the explanation next to a recommendation or turning "something that goes with a black winter coat" into a filtered query, but the ranking stays with the model that has seen the order history.

What to watch out for with recommendation systems

Recommending what they already bought. A model trained on purchases will happily suggest the dishwasher somebody bought last week. Consumables should come back, durables should not, and that distinction lives in your product data rather than in the model. Amazon's own example of what a filter is for is exactly this one.

Recommending what you cannot ship. Out-of-stock items cost you the click and some trust with it, so stock has to be checked at serving time rather than at training time. Google's retail documentation uses its catalogue-only similar-items model as the fallback for precisely this case.

The feedback loop that narrows your catalogue. The system decides what gets shown, what gets shown gets bought, and what gets bought trains the next version. Left alone, that spiral concentrates sales in an ever smaller set of articles. Coverage tells you what proportion of unique items the system would ever recommend, so track it next to click-through and treat a falling number as a problem even while the clicks look fine.

Profiling has a legal definition. Under the GDPR, profiling is any form of automated processing of personal data used to evaluate personal aspects relating to a person, and the definition names personal preferences, interests and behaviour explicitly. A system built on an identified customer's purchase history is profiling, so it needs a lawful basis and a line in your privacy statement. Article 27 of the Digital Services Act adds a duty for platforms carrying third-party content: set out the main parameters of your recommender system in the terms and conditions, in plain and intelligible language. That section leaves micro and small enterprises out.