IDEAS home Printed from https://ideas.repec.org/a/eee/ijrema/v40y2023i1p54-74.html
   My bibliography  Save this article

Complementing human effort in online reviews: A deep learning approach to automatic content generation and review synthesis

Author

Listed:
  • Carlson, Keith
  • Kopalle, Praveen K.
  • Riddell, Allen
  • Rockmore, Daniel
  • Vana, Prasad

Abstract

Online product reviews are ubiquitous and helpful sources of information available to consumers for making purchase decisions. Consumers rely on both the quantitative aspects of reviews such as valence and volume as well as textual descriptions to learn about product quality and fit. In this paper we show how new achievements in natural language processing can provide an important assist for different kinds of review-related writing tasks. Working in the interesting context of wine reviews, we demonstrate that machines are capable of performing the critical marketing task of writing expert reviews directly from a fairly small amount of product attribute data (metadata). We conduct a kind of “Turing Test” to evaluate human response to our machine-written reviews and show strong support for the assertion that machines can write reviews that are indistinguishable from those written by experts. Rather than replacing the human review writer, we envision a workflow wherein machines take the metadata as inputs and generate a human readable review as a first draft of the review and thereby assist an expert reviewer in writing their review. We next modify and apply our machine-writing technology to show how machines can be used to write a synthesis of a set of product reviews. For this last application we work in the context of beer reviews (for which there is a large set of available reviews for each of a large number of products) and produce machine-written review syntheses that do a good job – measured again through human evaluation – of capturing the ideas expressed in the reviews of any given beer. For each of these applications, we adapt the Transformer neural net architecture. The work herein is broadly applicable in marketing, particularly in the context of online reviews. We close with suggestions for additional applications of our model and approach as well as other directions for future research.

Suggested Citation

  • Carlson, Keith & Kopalle, Praveen K. & Riddell, Allen & Rockmore, Daniel & Vana, Prasad, 2023. "Complementing human effort in online reviews: A deep learning approach to automatic content generation and review synthesis," International Journal of Research in Marketing, Elsevier, vol. 40(1), pages 54-74.
    • Handle: RePEc:eee:ijrema:v:40:y:2023:i:1:p:54-74
    DOI: 10.1016/j.ijresmar.2022.02.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S016781162200009X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijresmar.2022.02.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mantian (Mandy) Hu & Chu (Ivy) Dang & Pradeep K. Chintagunta, 2019. "Search and Learning at a Daily Deals Website," Marketing Science, INFORMS, vol. 38(4), pages 609-642, July.
    2. Chunhua Wu & Hai Che & Tat Y. Chan & Xianghua Lu, 2015. "The Economic Value of Online Reviews," Marketing Science, INFORMS, vol. 34(5), pages 739-754, September.
    3. Artem Timoshenko & John R. Hauser, 2019. "Identifying Customer Needs from User-Generated Content," Marketing Science, INFORMS, vol. 38(1), pages 1-20, January.
    4. Eric M. Schwartz & Eric T. Bradlow & Peter S. Fader, 2017. "Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments," Marketing Science, INFORMS, vol. 36(4), pages 500-522, July.
    5. David A. Reinstein & Christopher M. Snyder, 2005. "The Influence Of Expert Reviews On Consumer Demand For Experience Goods: A Case Study Of Movie Critics," Journal of Industrial Economics, Wiley Blackwell, vol. 53(1), pages 27-51, March.
    6. Moon, Sangkil & Kamakura, Wagner A., 2017. "A picture is worth a thousand words: Translating product reviews into a product positioning map," International Journal of Research in Marketing, Elsevier, vol. 34(1), pages 265-285.
    7. Kannan, P.K. & Li, Hongshuang “Alice”, 2017. "Digital marketing: A framework, review and research agenda," International Journal of Research in Marketing, Elsevier, vol. 34(1), pages 22-45.
    8. Jonah Berger & Alan T. Sorensen & Scott J. Rasmussen, 2010. "Positive Effects of Negative Publicity: When Negative Reviews Increase Sales," Marketing Science, INFORMS, vol. 29(5), pages 815-827, 09-10.
    9. Richard Friberg & Erik Grönqvist, 2012. "Do Expert Reviews Affect the Demand for Wine?," American Economic Journal: Applied Economics, American Economic Association, vol. 4(1), pages 193-211, January.
    10. Anindya Ghose & Panagiotis G. Ipeirotis & Beibei Li, 2012. "Designing Ranking Systems for Hotels on Travel Search Engines by Mining User-Generated and Crowdsourced Content," Marketing Science, INFORMS, vol. 31(3), pages 493-520, May.
    11. Ishita Chakraborty & Minkyung Kim & K. Sudhir, 2019. "Attribute Sentiment Scoring With Online Text Reviews : Accounting for Language Structure and Attribute Self-Selection," Cowles Foundation Discussion Papers 2176R2, Cowles Foundation for Research in Economics, Yale University, revised Jun 2021.
    12. Decker, Reinhold & Trusov, Michael, 2010. "Estimating aggregate consumer preferences from online product reviews," International Journal of Research in Marketing, Elsevier, vol. 27(4), pages 293-307.
    13. Camacho, N.M.A. & de Jong, M.G. & Stremersch, S., 2014. "The Effect of Customer Empowerment on Adherence to Expert Advice," ERIM Report Series Research in Management ERS-2014-005-MKT, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    14. Surendra N. Singh & Steve Hillmer & Ze Wang, 2011. "Efficient Methods for Sampling Responses from Large-Scale Qualitative Data," Marketing Science, INFORMS, vol. 30(3), pages 532-549, 05-06.
    15. Michael Haenlein & Andreas Kaplan & Chee-Wee Tan & Pengzhu Zhang, 2019. "Artificial intelligence (AI) and management analytics," Journal of Management Analytics, Taylor & Francis Journals, vol. 6(4), pages 341-343, October.
    16. Xiao Liu & Param Vir Singh & Kannan Srinivasan, 2016. "A Structured Analysis of Unstructured Big Data by Leveraging Cloud Computing," Marketing Science, INFORMS, vol. 35(3), pages 363-388, May.
    17. Yi Zhao & Sha Yang & Vishal Narayan & Ying Zhao, 2013. "Modeling Consumer Learning from Online Product Reviews," Marketing Science, INFORMS, vol. 32(1), pages 153-169, May.
    18. Kübler, Raoul V. & Colicev, Anatoli & Pauwels, Koen H., 2020. "Social Media's Impact on the Consumer Mindset: When to Use Which Sentiment Extraction Tool?," Journal of Interactive Marketing, Elsevier, vol. 50(C), pages 136-155.
    19. Oded Netzer & Ronen Feldman & Jacob Goldenberg & Moshe Fresko, 2012. "Mine Your Own Business: Market-Structure Surveillance Through Text Mining," Marketing Science, INFORMS, vol. 31(3), pages 521-543, May.
    20. Xueming Luo & Siliang Tong & Zheng Fang & Zhe Qu, 2019. "Frontiers: Machines vs. Humans: The Impact of Artificial Intelligence Chatbot Disclosure on Customer Purchases," Marketing Science, INFORMS, vol. 38(6), pages 937-947, November.
    21. Davide Proserpio & Georgios Zervas, 2017. "Online Reputation Management: Estimating the Impact of Management Responses on Consumer Reviews," Marketing Science, INFORMS, vol. 36(5), pages 645-665, September.
    22. Camacho, Nuno & De Jong, Martijn & Stremersch, Stefan, 2014. "The effect of customer empowerment on adherence to expert advice," International Journal of Research in Marketing, Elsevier, vol. 31(3), pages 293-308.
    23. Wendy W. Moe & David A. Schweidel, 2012. "Online Product Opinions: Incidence, Evaluation, and Evolution," Marketing Science, INFORMS, vol. 31(3), pages 372-386, May.
    24. Onishi, Hiroshi & Manchanda, Puneet, 2012. "Marketing activity, blogging and sales," International Journal of Research in Marketing, Elsevier, vol. 29(3), pages 221-234.
    25. Li, Xi & Shi, Mengze & Wang, Xin (Shane), 2019. "Video mining: Measuring visual information using automatic methods," International Journal of Research in Marketing, Elsevier, vol. 36(2), pages 216-231.
    26. Ma, Liye & Sun, Baohong, 2020. "Machine learning and AI in marketing – Connecting computing power to human insights," International Journal of Research in Marketing, Elsevier, vol. 37(3), pages 481-504.
    27. Judith A. Chevalier & Yaniv Dover & Dina MayzlinDina Mayzlin, 2018. "Channels of Impact: User Reviews When Quality Is Dynamic and Managers Respond," Marketing Science, INFORMS, vol. 37(5), pages 688-709, September.
    28. Liu Liu & Daria Dzyabura & Natalie Mizik, 2020. "Visual Listening In: Extracting Brand Image Portrayed on Social Media," Marketing Science, INFORMS, vol. 39(4), pages 669-686, July.
    29. Pradeep K. Chintagunta & Shyam Gopinath & Sriram Venkataraman, 2010. "The Effects of Online User Reviews on Movie Box Office Performance: Accounting for Sequential Rollout and Aggregation Across Local Markets," Marketing Science, INFORMS, vol. 29(5), pages 944-957, 09-10.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Arpan Kumar Kar & P. S. Varsha & Shivakami Rajan, 2023. "Unravelling the Impact of Generative Artificial Intelligence (GAI) in Industrial Applications: A Review of Scientific and Grey Literature," Global Journal of Flexible Systems Management, Springer;Global Institute of Flexible Systems Management, vol. 24(4), pages 659-689, December.
    2. Peres, Renana & Schreier, Martin & Schweidel, David & Sorescu, Alina, 2023. "On ChatGPT and beyond: How generative artificial intelligence may affect research, teaching, and practice," International Journal of Research in Marketing, Elsevier, vol. 40(2), pages 269-275.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ngai, Eric W.T. & Wu, Yuanyuan, 2022. "Machine learning in marketing: A literature review, conceptual framework, and research agenda," Journal of Business Research, Elsevier, vol. 145(C), pages 35-48.
    2. Bitty Balducci & Detelina Marinova, 2018. "Unstructured data in marketing," Journal of the Academy of Marketing Science, Springer, vol. 46(4), pages 557-590, July.
    3. Roelen-Blasberg, Tobias & Habel, Johannes & Klarmann, Martin, 2023. "Automated inference of product attributes and their importance from user-generated content: Can we replace traditional market research?," International Journal of Research in Marketing, Elsevier, vol. 40(1), pages 164-188.
    4. Boegershausen, Johannes & Datta, Hannes & Borah, Abhishek & Stephen, Andrew, 2022. "Fields of Gold: Web Scraping and APIs for Impactful Marketing Insights," Other publications TiSEM 5f1ed70a-48c3-422c-bc10-0, Tilburg University, School of Economics and Management.
    5. Wang, Xin (Shane) & Ryoo, Jun Hyun (Joseph) & Bendle, Neil & Kopalle, Praveen K., 2021. "The role of machine learning analytics and metrics in retailing research," Journal of Retailing, Elsevier, vol. 97(4), pages 658-675.
    6. Dominik Gutt & Jürgen Neumann & Steffen Zimmermann & Dennis Kundisch & Jianqing Chen, 2018. "Design of Review Systems - A Strategic Instrument to shape Online Review Behavior and Economic Outcomes," Working Papers Dissertations 42, Paderborn University, Faculty of Business Administration and Economics.
    7. Xiao Liu & Param Vir Singh & Kannan Srinivasan, 2016. "A Structured Analysis of Unstructured Big Data by Leveraging Cloud Computing," Marketing Science, INFORMS, vol. 35(3), pages 363-388, May.
    8. Ana Babić Rosario & Kristine Valck & Francesca Sotgiu, 2020. "Conceptualizing the electronic word-of-mouth process: What we know and need to know about eWOM creation, exposure, and evaluation," Journal of the Academy of Marketing Science, Springer, vol. 48(3), pages 422-448, May.
    9. Jang, Seongsoo & Chung, Jaihak & Rao, Vithala R., 2021. "The importance of functional and emotional content in online consumer reviews for product sales: Evidence from the mobile gaming market," Journal of Business Research, Elsevier, vol. 130(C), pages 583-593.
    10. Jacobsen, Grant D., 2015. "Consumers, experts, and online product evaluations: Evidence from the brewing industry," Journal of Public Economics, Elsevier, vol. 126(C), pages 114-123.
    11. Moon, Sangkil & Kamakura, Wagner A., 2017. "A picture is worth a thousand words: Translating product reviews into a product positioning map," International Journal of Research in Marketing, Elsevier, vol. 34(1), pages 265-285.
    12. Mengxia Zhang & Lan Luo, 2023. "Can Consumer-Posted Photos Serve as a Leading Indicator of Restaurant Survival? Evidence from Yelp," Management Science, INFORMS, vol. 69(1), pages 25-50, January.
    13. Katsumata, Sotaro & Nishimoto, Akihiro & Kannan, P.K., 2023. "Brand competitiveness and resilience to exogenous shock: Usage of smartphone apps during the COVID-19 pandemic," Journal of Retailing and Consumer Services, Elsevier, vol. 75(C).
    14. Marios Kokkodis & Theodoros Lappas & Gerald C. Kane, 2022. "Optional purchase verification in e‐commerce platforms: More representative product ratings and higher quality reviews," Production and Operations Management, Production and Operations Management Society, vol. 31(7), pages 2943-2961, July.
    15. Villarroel Ordenes, Francisco & Silipo, Rosaria, 2021. "Machine learning for marketing on the KNIME Hub: The development of a live repository for marketing applications," Journal of Business Research, Elsevier, vol. 137(C), pages 393-410.
    16. Uttara Ananthakrishnan & Davide Proserpio & Siddhartha Sharma, 2023. "I Hear You: Does Quality Improve with Customer Voice?," Marketing Science, INFORMS, vol. 42(6), pages 1143-1161, November.
    17. Pauwels, Koen & Aksehirli, Zeynep & Lackman, Andrew, 2016. "Like the ad or the brand? Marketing stimulates different electronic word-of-mouth content to drive online and offline performance," International Journal of Research in Marketing, Elsevier, vol. 33(3), pages 639-655.
    18. Li, Xi & Shi, Mengze & Wang, Xin (Shane), 2019. "Video mining: Measuring visual information using automatic methods," International Journal of Research in Marketing, Elsevier, vol. 36(2), pages 216-231.
    19. Ao Shen & Peng Wang & Yongyuan Ma, 2022. "When crowding‐in and when crowding‐out? The boundary conditions on the relationship between negative online reviews and online sales," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 43(6), pages 2016-2032, September.
    20. Peters, Kay & Chen, Yubo & Kaplan, Andreas M. & Ognibeni, Björn & Pauwels, Koen, 2013. "Social Media Metrics — A Framework and Guidelines for Managing Social Media," Journal of Interactive Marketing, Elsevier, vol. 27(4), pages 281-298.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ijrema:v:40:y:2023:i:1:p:54-74. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/international-journal-of-research-in-marketing/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.