IDEAS home Printed from https://ideas.repec.org/p/diw/diwwpp/dp1730.html
   My bibliography  Save this paper

The Effect of Big Data on Recommendation Quality: The Example of Internet Search

Author

Listed:
  • Maximilian Schäfer
  • Geza Sapi
  • Szabolcs Lorincz

Abstract

Are there economies of scale to data in internet search? This paper is first to use real search engine query logs to empirically investigate how data drives the quality of internet search results. We find evidence that the quality of search results improve with more data on previous searches. Moreover, our results indicate that the type of data matters as well: personalized information is particularly valuable as it massively increases the speed of learning. We also provide some evidence that factors not directly related to data such as the general quality of the applied algorithms play an important role. The suggested methods to disentangle the effect of data from other factors driving the quality of search results can be applied to assess the returns to data in various recommendation systems in e-commerce, including product and information search. We also discuss the managerial, privacy, and competition policy implications of our findings.

Suggested Citation

  • Maximilian Schäfer & Geza Sapi & Szabolcs Lorincz, 2018. "The Effect of Big Data on Recommendation Quality: The Example of Internet Search," Discussion Papers of DIW Berlin 1730, DIW Berlin, German Institute for Economic Research.
    • Handle: RePEc:diw:diwwpp:dp1730
    as

    Download full text from publisher

    File URL: https://www.diw.de/documents/publikationen/73/diw_01.c.581628.de/dp1730.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Wendy W. Moe & Peter S. Fader, 2004. "Dynamic Conversion Behavior at E-Commerce Sites," Management Science, INFORMS, vol. 50(3), pages 326-335, March.
    2. Prabuddha De & Yu (Jeffrey) Hu & Mohammad S. Rahman, 2010. "Technology Usage and Online Sales: An Empirical Study," Management Science, INFORMS, vol. 56(11), pages 1930-1945, November.
    3. Cédric Argenton & Jens Prüfer, 2012. "Search Engine Competition With Network Externalities," Journal of Competition Law and Economics, Oxford University Press, vol. 8(1), pages 73-105.
    4. Maurice Stucke & Allen Grunes, 2015. "Debunking the Myths Over Big Data and Antitrust," Antitrust Chronicle, Competition Policy International, vol. 5.
    5. Lesley Chiou & Catherine Tucker, 2017. "Search Engines and Data Retention: Implications for Privacy and Antitrust," NBER Working Papers 23815, National Bureau of Economic Research, Inc.
    6. Patrick Bajari & Victor Chernozhukov & Ali Hortaçsu & Junichi Suzuki, 2019. "The Impact of Big Data on Firm Performance: An Empirical Investigation," AEA Papers and Proceedings, American Economic Association, vol. 109, pages 33-37, May.
    7. Pradeep Chintagunta & Dominique M. Hanssens & John R. Hauser, 2016. "Editorial—Marketing Science and Big Data," Marketing Science, INFORMS, vol. 35(3), pages 341-342, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Krämer, Jan & Shekhar, Shiva & Hofmann, Janina, 2022. "Regulating Algorithmic Learning in Digital Platform Ecosystems through Data Sharing and Data Siloing: Consequences for Innovation and Welfare," 31st European Regional ITS Conference, Gothenburg 2022: Reining in Digital Platforms? Challenging monopolies, promoting competition and developing regulatory regimes 265645, International Telecommunications Society (ITS).
    2. Graef, Inge & Prüfer, Jens, 2021. "Governance of data sharing: A law & economics proposal," Research Policy, Elsevier, vol. 50(9).
    3. Argentesi, Elena & Buccirossi, Paolo & Calvano, Emilio & Duso, Tomaso & Marrazzo, Alessia & Nava, Salvatore, 2021. "Merger Policy in Digital Markets: An Ex Post Assessment," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 17(1), pages 95-140.
    4. Hemant Bhargava & Antoine Dubus & David Ronayne & Shiva Shekhar, 2024. "The Strategic Value of Data Sharing in Interdependent Markets," CESifo Working Paper Series 10963, CESifo.
    5. de Cornière, Alexandre & Taylor, Greg, 2022. "Data and Competition: a Simple Framework with Applications to Mergers and Market Structure," CEPR Discussion Papers 14446, C.E.P.R. Discussion Papers.
    6. Kesler, Reinhold & Kummer, Michael E. & Schulte, Patrick, 2019. "Competition and privacy in online markets: Evidence from the mobile app industry," ZEW Discussion Papers 19-064, ZEW - Leibniz Centre for European Economic Research.
    7. Arnold, René & Marcus, J. Scott & Petropoulos, Georgios & Schneider, Anna, 2018. "Is data the new oil? Diminishing returns to scale," 29th European Regional ITS Conference, Trento 2018 184927, International Telecommunications Society (ITS).
    8. Calvano, Emilio & Polo, Michele, 2021. "Market power, competition and innovation in digital markets: A survey," Information Economics and Policy, Elsevier, vol. 54(C).
    9. de Cornière, Alexandre & Taylor, Greg, 2020. "Data and Competition: a General Framework with Applications to Mergers, Market Structure, and Privacy Policy," TSE Working Papers 20-1076, Toulouse School of Economics (TSE).
    10. Lenz, Fulko, 2020. "Plattformökonomie – zwischen Abwehr und Wunschdenken," Zeitthemen 03, Stiftung Marktwirtschaft / The Market Economy Foundation, Berlin.
    11. Jörg Claussen & Christian Peukert & Ananya Sen, 2019. "The Editor vs. the Algorithm: Returns to Data and Externalities in Online News," CESifo Working Paper Series 8012, CESifo.
    12. Schaefer, Maximilian & Sapi, Geza, 2023. "Complementarities in learning from data: Insights from general search," Information Economics and Policy, Elsevier, vol. 65(C).
    13. Georgios Petropoulos & Bertin Martens & Geoffrey Parker & Marshall Van Alstyne, 2023. "Platform Competition and Information Sharing," CESifo Working Paper Series 10663, CESifo.
    14. Ehsan Valavi & Joel Hestness & Newsha Ardalani & Marco Iansiti, 2022. "Time and the Value of Data," Papers 2203.09118, arXiv.org.
    15. Shan Huang & Michael Allan Ribers & Hannes Ullrich, 2021. "The Value of Data for Prediction Policy Problems: Evidence from Antibiotic Prescribing," Discussion Papers of DIW Berlin 1939, DIW Berlin, German Institute for Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Maximilian Schäfer & Geza Sapi, 2020. "Learning from Data and Network Effects: The Example of Internet Search," Discussion Papers of DIW Berlin 1894, DIW Berlin, German Institute for Economic Research.
    2. Ehsan Valavi & Joel Hestness & Newsha Ardalani & Marco Iansiti, 2022. "Time and the Value of Data," Papers 2203.09118, arXiv.org.
    3. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    4. Dipankar Das, 2023. "A Model of Competitive Assortment Planning Algorithm," Papers 2307.09479, arXiv.org.
    5. Lizhen Xu & Jason A. Duan & Andrew Whinston, 2014. "Path to Purchase: A Mutually Exciting Point Process Model for Online Advertising and Conversion," Management Science, INFORMS, vol. 60(6), pages 1392-1412, June.
    6. Graef, Inge & Prüfer, Jens, 2021. "Governance of data sharing: A law & economics proposal," Research Policy, Elsevier, vol. 50(9).
    7. Koski, Heli & Kässi, Otto & Braesemann, Fabian, 2020. "Killers on the Road of Emerging Start-ups – Implications for Market Entry and Venture Capital Financing," ETLA Working Papers 81, The Research Institute of the Finnish Economy.
    8. Schaefer, Maximilian & Sapi, Geza, 2023. "Complementarities in learning from data: Insights from general search," Information Economics and Policy, Elsevier, vol. 65(C).
    9. Catherine Tucker, 2019. "Digital Data, Platforms and the Usual [Antitrust] Suspects: Network Effects, Switching Costs, Essential Facility," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 54(4), pages 683-694, June.
    10. Guy Aridor & Yeon-Koo Che & Tobias Salz, 2020. "The Effect of Privacy Regulation on the Data Industry: Empirical Evidence from GDPR," NBER Working Papers 26900, National Bureau of Economic Research, Inc.
    11. Calvano, Emilio & Polo, Michele, 2021. "Market power, competition and innovation in digital markets: A survey," Information Economics and Policy, Elsevier, vol. 54(C).
    12. Steffen, Nico & Wiewiorra, Lukas & Kroon, Peter, 2021. "Wettbewerb und Regulierung in der Plattform- und Datenökonomie," WIK Discussion Papers 481, WIK Wissenschaftliches Institut für Infrastruktur und Kommunikationsdienste GmbH.
    13. Flavio Pino, 2022. "The microeconomics of data – a survey," Economia e Politica Industriale: Journal of Industrial and Business Economics, Springer;Associazione Amici di Economia e Politica Industriale, vol. 49(3), pages 635-665, September.
    14. Kesler, Reinhold & Kummer, Michael E. & Schulte, Patrick, 2019. "Competition and privacy in online markets: Evidence from the mobile app industry," ZEW Discussion Papers 19-064, ZEW - Leibniz Centre for European Economic Research.
    15. Ke Gong & Yi Peng & Yong Wang & Maozeng Xu, 2018. "Time series analysis for C2C conversion rate," Electronic Commerce Research, Springer, vol. 18(4), pages 763-789, December.
    16. Patrick Bajari & Victor Chernozhukov & Ali Hortaçsu & Junichi Suzuki, 2019. "The Impact of Big Data on Firm Performance: An Empirical Investigation," AEA Papers and Proceedings, American Economic Association, vol. 109, pages 33-37, May.
    17. Artige, Lionel, 2015. "Will Big Data Deliver its Promised Productivity Growth," Proceedings of the ENTRENOVA - ENTerprise REsearch InNOVAtion Conference (2015), Kotor, Montengero, in: Proceedings of the ENTRENOVA - ENTerprise REsearch InNOVAtion Conference, Kotor, Montengero, 10-11 September 2015, pages 193-200, IRENET - Society for Advancing Innovation and Research in Economy, Zagreb.
    18. Wang, Jiaxin & Zhao, Mu & Huang, Xiang & Song, Zilong & Sun, Di, 2024. "Supply chain diffusion mechanisms for AI applications: A perspective on audit pricing," International Review of Financial Analysis, Elsevier, vol. 93(C).
    19. Kinshuk Jerath & Anuj Kumar & Serguei Netessine, 2015. "An Information Stock Model of Customer Behavior in Multichannel Customer Support Services," Manufacturing & Service Operations Management, INFORMS, vol. 17(3), pages 368-383, July.
    20. Kris J. Ferreira & Sunanda Parthasarathy & Shreyas Sekar, 2022. "Learning to Rank an Assortment of Products," Management Science, INFORMS, vol. 68(3), pages 1828-1848, March.

    More about this item

    Keywords

    Big Data; Recommendation quality; Internet search; E-Commerce; Economies of Scale; Search engines;
    All these keywords.

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • L81 - Industrial Organization - - Industry Studies: Services - - - Retail and Wholesale Trade; e-Commerce
    • L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software
    • M15 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - IT Management

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:diw:diwwpp:dp1730. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bibliothek (email available below). General contact details of provider: https://edirc.repec.org/data/diwbede.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.