Recommender Systems

Recommender Systems

Recommender systems are software that suggests products from what similar shoppers did. In published field tests where sales were measured directly, they added one to five per cent on average. Much larger figures usually come from weak comparisons, single categories or company statements with no stated method.

What it is

The best summary is a 2019 review by Dietmar Jannach, at the University of Klagenfurt, and Michael Jugovac, at TU Dortmund, in ACM Transactions on Management Information Systems. Gathering the published tests of what recommenders do for a business, they concluded that "in many cases where business effects are directly measured, increases in sales between one and five percent are reported on average." Larger effects appear in slices of a shop, such as a rise of more than 26 per cent for one category of an online grocery store, and switching recommenders off at an online bookshop cut revenue by 17 per cent. How large a reported gain looks depends on what the system replaced: sometimes no suggestions at all, sometimes a simple bestseller list, and only rarely another sophisticated method.

The review's most telling result comes from a news site. A more sophisticated method raised click-through by about 35 per cent; changing only the position and size of the box, with the method untouched, raised it by about 100 per cent. The review also notes that click-through is in most cases not a true measure of business value, and that popularity lists or random suggestions have matched clever methods on clicks in live trials.

The two most repeated figures, for Netflix viewing and Amazon sales, trace to a company blog post and a chief executive's remark. Netflix later published its own account in 2015: Carlos Gomez-Uribe and Neil Hunt wrote that the recommender influences choice for about 80 per cent of hours streamed, and that "we think" personalisation saves more than a billion dollars a year, with no calculation given.

In effect

Treat a suggestion as a pointer to what similar people bought, and a click on one as evidence about the page as much as about you. An aggregate lift of one to five per cent is compatible with the system being wrong about any particular shopper on any particular visit. Research by Elizabeth Aguirre and colleagues (2015) found more personalised ads raised click intentions only when data had been collected openly, and Tiffany Barnett White and colleagues (2008) found reactance when firms did not explain why an offer fitted, so a shop that says why it is suggesting something has earned more trust.

What it does not say

The figures come from tests firms chose to publish, and the review notes that firms usually do not share how they profit from recommendation. An average also hides the cases that matter: in a very large catalogue, a recommender is a real aid to finding things, and a category that rises by a quarter is a real effect for its shoppers. Recommenders can also change what gets bought, not only how much; see sales diversity. Trade books such as The Future of Shopping (2022) treat recommendation as a settled capability without putting a number on it.

Sources

  1. Jannach, D., & Jugovac, M. (2019). ACM Transactions on Management Information Systems, 10(4), 1-23. doi:10.1145/3370082.
  2. Gomez-Uribe, C. A., & Hunt, N. (2015). ACM Transactions on Management Information Systems, 6(4), Article 13. doi:10.1145/2843948.
  3. Aguirre, E., Mahr, D., Grewal, D., de Ruyter, K., & Wetzels, M. (2015). Journal of Retailing, 91(1), 34-49. doi:10.1016/j.jretai.2014.09.005. White, T. B., Zahay, D. L., Thorbjørnsen, H., & Shavitt, S. (2008), Marketing Letters, 19(1), 39-50.
  4. Snoeck, J., & Neerman, P. (2022). The Future of Shopping. LannooCampus.