An Empirical Audit of LLM Product Recommendations

Large language models (LLMs) are increasingly used by consumers to search for information, compare alternatives, and obtain purchasing advice. As conversational AI systems progressively complement traditional search engines and recommender systems, they may influence consumer choice by determining which products receive visibility and attention. Despite their growing importance, little is known about how LLMs construct product recommendations, whether they concentrate attention on a narrow set of products, or whether vertically integrated providers favor affiliated products and services. This paper conducts an audit study of leading LLMs, including ChatGPT, Gemini, Claude, DeepSeek, Grok, and Mistral. Using standardized prompts and repeated queries, we collect recommendations across a broad range of products and service categories that differ in market characteristics and the presence of vertically integrated products. We characterize recommendations along several dimensions, including breadth, first-position concentration, prominence, best-overall selections, and personalization. We find substantial heterogeneity across models in the breadth and structure of their recommendations. At the same time, first-position recommendations are highly concentrated in a substantial subset of markets. In 20 of the 100 product and service categories, a single product is listed first in more than three quarters of responses. We also find that some LLMs are likely to recommend products affiliated with their providers.