How to Build a Recommendation Engine That Converts, Not Just Scores Well

There are three well-established families, defined in the canonical 2005 survey by Adomavicius and Tuzhilin. You do not need to pick a favourite. You need the one that fits your data.

• Collaborative filtering: recommend items that people with similar tastes liked in the past. People like you also liked this. It needs a decent history of interactions to work.
• Content-based: recommend items similar to the ones a person already engaged with. More of what you already read, watched or bought. It leans on item attributes, so it copes better with thin interaction data.
• Hybrid: combine the two, plus sensible business rules. Hybrids exist because they avoid the specific limitations of each pure method, which is why they usually win in practice.

Every recommender hits the same wall: it cannot recommend well without data, and at the start it has none. Adomavicius and Tuzhilin name three versions of this. The new-user problem: the system must learn a person's preferences from ratings they have not given yet. The new-item problem: a collaborative system cannot surface a fresh product until enough people have interacted with it. And sparsity: the number of ratings you already have is tiny next to the number you need to predict.

The fixes are unglamorous and they work. Lean on content signals and item attributes for new users and new items, so you can recommend before you have behaviour to learn from. Add business rules for the cases that matter: new arrivals, stock you need to move, margin, editorial picks. Fall back to sensible popularity when you know nothing, then hand over to the personalised model as soon as you have signal. None of this is exotic. It is the difference between a recommender that works on day one and one that sulks until it has a year of data.

Left alone, a recommender tends to over-recommend what is already popular. A 2024 survey of popularity bias sets out the pattern. Recommenders often focus on popular items, which limits their value for both customers and providers, and can cause reinforcement effects over time. In one large field test on a North American retailer, the recommender did lift sales of niche items, but it lifted popular items even more, and aggregate sales diversity actually fell. Already popular items get more exposure, which earns them still more exposure: a feedback loop where the rich get richer.

This is often described as filter bubbles, though the evidence for real harm is mixed, and individual choice can matter more than the algorithm. So do not overclaim the danger, but do build the guardrails. Inject diversity so the same ten items do not fill every slot. Add freshness so the feed does not go stale. Keep business rules in the loop. The aim is a recommender that widens what people discover rather than narrowing it.

Here is the discipline that separates a recommender that converts from one that only scores well. Tune against real outcomes: conversion, order value and retention. Then keep testing, because tastes and inventory never stop changing. Offline metrics are a useful filter, not a verdict.

The Netflix team gives a concrete warning. They compared two sets of similar titles for a show. One set looked more relevant to a human eye. In an A/B test it lost to the set with a stronger popularity influence. Their conclusion is blunt: intuition, even collective intuition, often picks the wrong variant, and it is frequently impossible to tell good recommendations from great ones by eye. That is why they run A/B tests focused on member retention and medium-term engagement, not on offline accuracy.

If you have a catalogue and history sitting in your systems, you are closer than you think. The hard part is not the model. It is choosing the number to move, picking the simplest approach that moves it, and proving the lift with a test tied to revenue and retention.

See how we ship outcome-first machine learning on the MERIT Vision Case Studies page, then tell us the one number you need to move.

1. The Netflix Recommender System: Algorithms, Business Value, and Innovation (Gomez-Uribe and Hunt, ACM TMIS, 2015) (accessed 2026-09-07)
2. How retailers can keep up with consumers (MacKenzie, Meyer and Noble, McKinsey, 2013) (accessed 2026-09-07)
3. The value of getting personalization right or wrong is multiplying (McKinsey, 2021) (accessed 2026-09-07)
4. The Value of Personalized Recommendations: Evidence from Netflix (Zielnicki et al., arXiv:2511.07280, 2025) (accessed 2026-09-07)
5. Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions (Adomavicius and Tuzhilin, IEEE TKDE, 2005) (accessed 2026-09-07)
6. A survey on popularity bias in recommender systems (Klimashevskaia et al., User Modeling and User-Adapted Interaction, 2024) (accessed 2026-09-07)
7. Measuring the Business Value of Recommender Systems (Jannach and Jugovac, arXiv:1908.08328, 2019) (accessed 2026-09-07)
8. Filter Bubbles in Recommender Systems: Fact or Fallacy, A Systematic Review (arXiv:2307.01221) (accessed 2026-09-07)