Recommendations systems are powerful revenue engines: the better the system, the longer the users stay on the platform or the more the clients spend on the website. But systems are numerous and some perform better than others. In particular, some systems are automated and based on other users’ behaviours whereas others are more manual and based on experts. Which is best? For which objective? In which context? What are the strategic implications?
The research: The crowd against the few: Measuring the impact of expert recommendations. By Herm-Stapelberg and Rothlauf.
In this work, the authors tackle a stimulating question comparing the impacts of various recommendations systems on users behaviours of an online video-on-demand service. More than 6,000 users of this service accepted to be part of the experiment. They were randomly assigned either to the treatment group (receiving recommendations from experts) and the control group (receiving system-generated recommendations).
The main results
- Experts can significantly increase platform usage, are able to keep users longer on the website, and make users come back more frequently. Using experts leads to an increase in the number of watched clips (+9.8%), the use of recommendations (+17.1%), and whether people still used the platform after 10 weeks (+15% retain rate). Users visited the platform more often (+9%) and had more frequent visits where they watched more than one clip (+15.7%).
- Surprisingly, even though the treatment group significantly used the platform more often expert-based suggestions were used less than expected. This is in line with the finding that the probability of clicking on another recommendation decreases when the first clicked recommendation was generated by experts. Nevertheless, users who did not receive expert suggestions at all, have an even lower probability of clicking on another recommendation.
Even if expert-based recommendations are used less than expected, they have a large and significant impact on user behaviour. Expert recommendations are more diverse and better cover the taste of users.
What does it mean?
- Recommendation systems embed implicit (and often not debated) hypotheses about behaviours. For content-based systems (which use previously consumed items by the user), it is that people tend to favor what they already experienced themselves. For collaborative systems (based on other users consumptions), it is that people tend to mimic what their peers are consuming. For expert-based systems, it is that users have a wide range of preferences (diversity) and specific taste.
- There is no silver bullet. Some recommendations systems perform better than others in some industries, in a given context, for specific types of users or for particular objectives.
Useful questions for strategy-making
- Recommendation systems embed hypotheses about behaviours => How far have you analysed and understood your customers’ behaviors? Are they more sensitive to their peers or to experts?
- Recommendation systems can be a source for differentiation => How well do you know your competitors’ recommendation systems? What is your positioning (copying with a better performance or changing the logic of recommendation)?
- Recommendation systems are powerful revenue engines and we know more and more about their types and logic => Which systems are you using? Do you assess their performance against other systems (e.g.: user behaviour vs general trends)?
This article was first published on Prof. Benyayer’s blog.
This post gives the views of its author, not the position of ESCP Business School.
License and Republishing
The Choice - Republishing rules
We publish under a Creative Commons license with the following characteristics Attribution/Sharealike.
- You may not make any changes to the articles published on our site, except for dates, locations (according to the news, if necessary), and your editorial policy. The content must be reproduced and represented by the licensee as published by The Choice, without any cuts, additions, insertions, reductions, alterations or any other modifications.If changes are planned in the text, they must be made in agreement with the author before publication.
- Please make sure to cite the authors of the articles, ideally at the beginning of your republication.
- It is mandatory to cite The Choice and include a link to its homepage or the URL of thearticle. Insertion of The Choice’s logo is highly recommended.
- The sale of our articles in a separate way, in their entirety or in extracts, is not allowed , but you can publish them on pages including advertisements.
- Please request permission before republishing any of the images or pictures contained in our articles. Some of them are not available for republishing without authorization and payment. Please check the terms available in the image caption. However, it is possible to remove images or pictures used by The Choice or replace them with your own.
- Systematic and/or complete republication of the articles and content available on The Choice is prohibited.
- Republishing The Choice articles on a site whose access is entirely available by payment or by subscription is prohibited.
- For websites where access to digital content is restricted by a paywall, republication of The Choice articles, in their entirety, must be on the open access portion of those sites.
- The Choice reserves the right to enter into separate written agreements for the republication of its articles, under the non-exclusive Creative Commons licenses and with the permission of the authors. Please contact The Choice if you are interested at contact@the-choice.org.
Individual cases
Extracts: It is recommended that after republishing the first few lines or a paragraph of an article, you indicate "The entire article is available on ESCP’s media, The Choice" with a link to the article.
Citations: Citations of articles written by authors from The Choice should include a link to the URL of the authors’ article.
Translations: Translations may be considered modifications under The Choice's Creative Commons license, therefore these are not permitted without the approval of the article's author.
Modifications: Modifications are not permitted under the Creative Commons license of The Choice. However, authors may be contacted for authorization, prior to any publication, where a modification is planned. Without express consent, The Choice is not bound by any changes made to its content when republished.
Authorized connections / copyright assignment forms: Their use is not necessary as long as the republishing rules of this article are respected.
Print: The Choice articles can be republished according to the rules mentioned above, without the need to include the view counter and links in a printed version.
If you choose this option, please send an image of the republished article to The Choice team so that the author can review it.
Podcasts and videos: Videos and podcasts whose copyrights belong to The Choice are also under a Creative Commons license. Therefore, the same republishing rules apply to them.
