IDEAS home Printed from https://ideas.repec.org/a/kap/expeco/v22y2019i4d10.1007_s10683-018-09600-z.html
   My bibliography  Save this article

Using machine learning for communication classification

Author

Listed:
  • Stefan P. Penczynski

    (University of East Anglia)

Abstract

The present study explores the value of machine learning techniques in the classification of communication content in experiments. Previously human-coded datasets are used to both train and test algorithm-generated models that relate word counts to categories. For various games, the computer models of the classification are able to match out-of-sample the human classification to a considerable extent. The analysis raises hope that the substantial effort going into such studies can be reduced by using computer algorithms for classification. This would enable a quick and replicable analysis of large-scale datasets at reasonable costs and widen the applicability of such approaches. The paper gives an easily accessible technical introduction into the computational method.

Suggested Citation

  • Stefan P. Penczynski, 2019. "Using machine learning for communication classification," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 1002-1029, December.
  • Handle: RePEc:kap:expeco:v:22:y:2019:i:4:d:10.1007_s10683-018-09600-z
    DOI: 10.1007/s10683-018-09600-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10683-018-09600-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10683-018-09600-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Vincent P. Crawford & Uri Gneezy & Yuval Rottenstreich, 2008. "The Power of Focal Points Is Limited: Even Minute Payoff Asymmetry May Yield Large Coordination Failures," American Economic Review, American Economic Association, vol. 98(4), pages 1443-1458, September.
    2. Stahl Dale O. & Wilson Paul W., 1995. "On Players' Models of Other Players: Theory and Experimental Evidence," Games and Economic Behavior, Elsevier, vol. 10(1), pages 218-254, July.
    3. Burchardi, Konrad B. & Penczynski, Stefan P., 2014. "Out of your mind: Eliciting individual reasoning in one shot games," Games and Economic Behavior, Elsevier, vol. 84(C), pages 39-57.
    4. David J. Cooper & John H. Kagel, 2005. "Are Two Heads Better Than One? Team versus Individual Play in Signaling Games," American Economic Review, American Economic Association, vol. 95(3), pages 477-509, June.
    5. Penczynski, Stefan P., 2017. "The nature of social learning: Experimental evidence," European Economic Review, Elsevier, vol. 94(C), pages 148-165.
    6. Anderson, Lisa R & Holt, Charles A, 1997. "Information Cascades in the Laboratory," American Economic Review, American Economic Association, vol. 87(5), pages 847-862, December.
    7. Nagel, Rosemarie, 1995. "Unraveling in Guessing Games: An Experimental Study," American Economic Review, American Economic Association, vol. 85(5), pages 1313-1326, December.
    8. Moellers, Claudia & Normann, Hans-Theo & Snyder, Christopher M., 2017. "Communication in vertical markets: Experimental evidence," International Journal of Industrial Organization, Elsevier, vol. 50(C), pages 214-258.
    9. Anna Lou Abatayo & John Lynham & Katerina Sherstyuk, 2018. "Facebook-to-Facebook: online communication and economic cooperation," Applied Economics Letters, Taylor & Francis Journals, vol. 25(11), pages 762-767, June.
    10. Matthew Gentzkow & Jesse M. Shapiro, 2010. "What Drives Media Slant? Evidence From U.S. Daily Newspapers," Econometrica, Econometric Society, vol. 78(1), pages 35-71, January.
    11. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    12. Erik Eyster & Matthew Rabin, 2010. "Naïve Herding in Rich-Information Settings," American Economic Journal: Microeconomics, American Economic Association, vol. 2(4), pages 221-243, November.
    13. Jacob K. Goeree & Leeat Yariv, 2011. "An Experimental Study of Collective Deliberation," Econometrica, Econometric Society, vol. 79(3), pages 893-921, May.
    14. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    15. Penczynski, Stefan P., 2016. "Strategic thinking: The influence of the game," Journal of Economic Behavior & Organization, Elsevier, vol. 128(C), pages 72-84.
    16. Penczynski, Stefan P., 2016. "Persuasion: An experimental study of team decision making," Journal of Economic Psychology, Elsevier, vol. 56(C), pages 244-261.
    17. Kenneth Benoit & Michael Laver & Slava Mikhaylov, 2009. "Treating Words as Data with Error: Uncertainty in Text Statements of Policy Positions," American Journal of Political Science, John Wiley & Sons, vol. 53(2), pages 495-513, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hausladen, Carina I. & Fochmann, Martin & Mohr, Peter, 2024. "Predicting compliance: Leveraging chat data for supervised classification in experimental research," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 109(C).
    2. Maximilian Andres & Lisa Bruttel & Jana Friedrichsen, 2020. "Choosing between explicit cartel formation and tacit collusion – An experiment," CEPA Discussion Papers 19, Center for Economic Policy Analysis.
    3. Arad, Ayala & Penczynski, Stefan P., 2024. "Multi-dimensional reasoning in competitive resource allocation games: Evidence from intra-team communication," Games and Economic Behavior, Elsevier, vol. 144(C), pages 355-377.
    4. Andres, Maximilian & Bruttel, Lisa & Friedrichsen, Jana, 2021. "The leniency rule revisited: Experiments on cartel formation with open communication," International Journal of Industrial Organization, Elsevier, vol. 76(C).
    5. Konstantinos Georgalos & John Hey, 2020. "Testing for the emergence of spontaneous order," Experimental Economics, Springer;Economic Science Association, vol. 23(3), pages 912-932, September.
    6. Andres, Maximilian & Bruttel, Lisa & Friedrichsen, Jana, 2023. "How communication makes the difference between a cartel and tacit collusion: A machine learning approach," European Economic Review, Elsevier, vol. 152(C).
    7. Benjamin Wegener, 2021. "How to Analyze Communication Data from Laboratory Experiments Without Being a Machine Learning Specialist," Journal of Economics and Behavioral Studies, AMH International, vol. 13(1), pages 32-56.
    8. Maximilian Andres & Lisa Bruttel & Jana Friedrichsen, 2019. "The Effect of a Leniency Rule on Cartel Formation and Stability: Experiments with Open Communication," Discussion Papers of DIW Berlin 1835, DIW Berlin, German Institute for Economic Research.
    9. Elten, Jonas van & Penczynski, Stefan P., 2020. "Coordination games with asymmetric payoffs: An experimental study with intra-group communication," Journal of Economic Behavior & Organization, Elsevier, vol. 169(C), pages 158-188.
    10. Andres, Maximilian & Bruttel, Lisa & Friedrichsen, Jana, 2021. "How do sanctions work? The choice between cartel formation and tacit collusion," VfS Annual Conference 2021 (Virtual Conference): Climate Economics 242372, Verein für Socialpolitik / German Economic Association.
    11. David J. Cooper & Ian Krajbich & Charles N. Noussair, 2019. "Choice-Process Data in Experimental Economics," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 1-13, August.
    12. Tebbe, Eva & Wegener, Benjamin, 2022. "Is natural language processing the cheap charlie of analyzing cheap talk? A horse race between classifiers on experimental communication data," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 96(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Elten, Jonas van & Penczynski, Stefan P., 2020. "Coordination games with asymmetric payoffs: An experimental study with intra-group communication," Journal of Economic Behavior & Organization, Elsevier, vol. 169(C), pages 158-188.
    2. Ayala Arad & Kevin P. Grubiak & Stefan P. Penczynski, 2024. "Does communicating within a team influence individuals’ reasoning and decisions?," Experimental Economics, Springer;Economic Science Association, vol. 27(1), pages 109-129, March.
    3. Arad, Ayala & Penczynski, Stefan P., 2024. "Multi-dimensional reasoning in competitive resource allocation games: Evidence from intra-team communication," Games and Economic Behavior, Elsevier, vol. 144(C), pages 355-377.
    4. Penczynski, Stefan P., 2017. "The nature of social learning: Experimental evidence," European Economic Review, Elsevier, vol. 94(C), pages 148-165.
    5. Alessandro Sontuoso & Sudeep Bhatia, 2021. "A notion of prominence for games with natural‐language labels," Quantitative Economics, Econometric Society, vol. 12(1), pages 283-312, January.
    6. Vincent P. Crawford & Miguel A. Costa-Gomes & Nagore Iriberri, 2010. "Strategic Thinking," Levine's Working Paper Archive 661465000000001148, David K. Levine.
    7. Penczynski, Stefan P., 2016. "Persuasion: An experimental study of team decision making," Journal of Economic Psychology, Elsevier, vol. 56(C), pages 244-261.
    8. Tong, Hanh T. & Freeman, David J., 2021. "Anchors of strategic reasoning in the traveler’s dilemma," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 28-38.
    9. Burchardi, Konrad B. & Penczynski, Stefan P., 2014. "Out of your mind: Eliciting individual reasoning in one shot games," Games and Economic Behavior, Elsevier, vol. 84(C), pages 39-57.
    10. Sitzia, Stefania & Zheng, Jiwei, 2019. "Group behaviour in tacit coordination games with focal points – an experimental investigation," Games and Economic Behavior, Elsevier, vol. 117(C), pages 461-478.
    11. Li, Wei & Tan, Xu, 2021. "Cognitively-constrained learning from neighbors," Games and Economic Behavior, Elsevier, vol. 129(C), pages 32-54.
    12. Chen, Chun-Ting & Huang, Chen-Ying & Wang, Joseph Tao-yi, 2018. "A window of cognition: Eyetracking the reasoning process in spatial beauty contest games," Games and Economic Behavior, Elsevier, vol. 111(C), pages 143-158.
    13. Bosch-Domènech, Antoni & Vriend, Nicolaas J., 2013. "On the role of non-equilibrium focal points as coordination devices," Journal of Economic Behavior & Organization, Elsevier, vol. 94(C), pages 52-67.
    14. Dieter Balkenborg & Rosemarie Nagel, 2016. "An Experiment on Forward vs. Backward Induction: How Fairness and Level k Reasoning Matter," German Economic Review, Verein für Socialpolitik, vol. 17(3), pages 378-408, August.
    15. Strzalecki, Tomasz, 2014. "Depth of reasoning and higher order beliefs," Journal of Economic Behavior & Organization, Elsevier, vol. 108(C), pages 108-122.
    16. Quement, Mark T. Le & Marcin, Isabel, 2020. "Communication and voting in heterogeneous committees: An experimental study," Journal of Economic Behavior & Organization, Elsevier, vol. 174(C), pages 449-468.
    17. Helland, Leif & Iachan, Felipe S. & Juelsrud, Ragnar E. & Nenov, Plamen T., 2021. "Information quality and regime change: Evidence from the lab," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 538-554.
    18. Baethge, Caroline, 2016. "Performance in the beauty contest: How strategic discussion enhances team reasoning," Passauer Diskussionspapiere, Betriebswirtschaftliche Reihe B-17-16, University of Passau, Faculty of Business and Economics.
    19. Feri, Francesco & Meléndez-Jiménez, Miguel A. & Ponti, Giovanni & Vega-Redondo, Fernando, 2011. "Error cascades in observational learning: An experiment on the Chinos game," Games and Economic Behavior, Elsevier, vol. 73(1), pages 136-146, September.
    20. Dorothea Kübler & Georg Weizsäcker, 2004. "Limited Depth of Reasoning and Failure of Cascade Formation in the Laboratory," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 71(2), pages 425-441.

    More about this item

    Keywords

    Communication; Classification; Machine learning;
    All these keywords.

    JEL classification:

    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:expeco:v:22:y:2019:i:4:d:10.1007_s10683-018-09600-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.