Personalized Rankings and User Engagement: An Empirical Evaluation of the Reddit News Feed

Digital platforms increasingly curate their content through personalized algorithmic rankings. Given the limited attention of their users and reliance on advertising, platforms have an incentive to promote content that increases the predicted engagement of each user. However, managers must also balance maximizing total engagement with the quality of content promoted on the platform due to advertiser concerns over brand safety and to satisfy policymakers. This paper studies how maximizing engagement for each user affects the quality of content with which users engage to understand the extent to which engagement-maximizing algorithms promote and incentivize low-quality content. In addition, I evaluate how the ranking algorithm itself can be designed to promote and encourage engagement with high quality content. To do this, I study the Reddit politics community and exploit a novel discontinuity – revealed in Reddit’s code repository – in how the ranking algorithm orders posts to identify the effect of a post’s rank on the number of comments it receives. I use this discontinuity to identify a discrete choice model of user comment decisions and estimate the distribution of news that users are exposed to and comment on under a personalized algorithm that maximizes engagement. This counterfactual demonstrates that personalization drives a wedge between users in terms of the quality of content – the credibility rating of an article’s publisher – with which are exposed and engage. Under the personalized ranking algorithm, users who ordinarily engage with high-credibility publishers continue to do so. However, users who ordinarily engage with lower-credibility publishers are exposed to and engage with an even larger share of low-credibility publishers under the personalized engagement-maximizing algorithm. Finally, I evaluate a credibility-aware algorithm that explicitly promotes credible news publishers and find that moving to the credibility-maximizing algorithm reduces total engagement by 5.0%, a meaningful decline. Yet, platforms can increase the share of the average user’s engagement with high-credibility publishers by 6.8 percentage points for only a 2.0% decrease in engagement. These findings suggest that algorithmic interventions can be a useful tool to promote higher-quality content to help satisfy both advertisers and policy makers.