IDEAS home Printed from https://ideas.repec.org/a/eee/soceco/v86y2020ics2214804319303866.html
   My bibliography  Save this article

Classroom experiments as a replication device

Author

Listed:
  • So, Tony

Abstract

A string of failed experimental replications in many disciplines have shed light on the low levels of replicability of published research. There is an increasing call for more replications to be conducted to bring credibility back to academic research. Despite this, there are few incentives for researchers to conduct replicating studies. They are costly in terms of time and money, and are difficult to publish due to the competitive nature of publication, where journals seek a high degree of novelty and contribution. This paper proposes a low-cost method of replication: conducting replication experiments in a classroom context. As a case in point, we present results from a simple replication of Weber's (2003) “`Learning' with no feedback in a competitive guessing game”.

Suggested Citation

  • So, Tony, 2020. "Classroom experiments as a replication device," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 86(C).
  • Handle: RePEc:eee:soceco:v:86:y:2020:i:c:s2214804319303866
    DOI: 10.1016/j.socec.2020.101525
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S2214804319303866
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.socec.2020.101525?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alfredo Di Tillio & Marco Ottaviani & Peter Norman Sørensen, 2017. "Persuasion Bias in Science: Can Economics Help?," Economic Journal, Royal Economic Society, vol. 127(605), pages 266-304, October.
    2. Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
    3. Uri Gneezy & Stephan Meier & Pedro Rey-Biel, 2011. "When and Why Incentives (Don't) Work to Modify Behavior," Journal of Economic Perspectives, American Economic Association, vol. 25(4), pages 191-210, Fall.
    4. Peter Kuhn & Peter Kooreman & Adriaan Soetevent & Arie Kapteyn, 2011. "The Effects of Lottery Prizes on Winners and Their Neighbors: Evidence from the Dutch Postcode Lottery," American Economic Review, American Economic Association, vol. 101(5), pages 2226-2247, August.
    5. Gary Charness & David Masclet & Marie Claire Villeval, 2014. "The Dark Side of Competition for Status," Management Science, INFORMS, vol. 60(1), pages 38-55, January.
    6. Frank, Bjorn, 1997. "The Impact of Classroom Experiments on the Learning of Economics: An Empirical Investigation," Economic Inquiry, Western Economic Association International, vol. 35(4), pages 763-769, October.
    7. Lucas C. Coffman & Muriel Niederle & Alistair J. Wilson, 2017. "A Proposal to Organize and Promote Replications," American Economic Review, American Economic Association, vol. 107(5), pages 41-45, May.
    8. Roth, Alvin E. & Erev, Ido, 1995. "Learning in extensive-form games: Experimental data and simple dynamic models in the intermediate term," Games and Economic Behavior, Elsevier, vol. 8(1), pages 164-212.
    9. David Gill & Victoria Prowse, 2016. "Cognitive Ability, Character Skills, and Learning to Play Equilibrium: A Level-k Analysis," Journal of Political Economy, University of Chicago Press, vol. 124(6), pages 1619-1676.
    10. Agnès Festré & Pierre Garrouste, 2015. "Theory And Evidence In Psychology And Economics About Motivation Crowding Out: A Possible Convergence?," Journal of Economic Surveys, Wiley Blackwell, vol. 29(2), pages 339-356, April.
    11. Camelia M. Kuhnen & Agnieszka Tymula, 2012. "Feedback, Self-Esteem, and Performance in Organizations," Management Science, INFORMS, vol. 58(1), pages 94-113, January.
    12. Nagel, Rosemarie, 1995. "Unraveling in Guessing Games: An Experimental Study," American Economic Review, American Economic Association, vol. 85(5), pages 1313-1326, December.
    13. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    14. Smith, Vernon L, 1976. "Experimental Economics: Induced Value Theory," American Economic Review, American Economic Association, vol. 66(2), pages 274-279, May.
    15. Theodore C. Bergstrom & Eugene Kwok, 2005. "Extracting Valuable Data from Classroom Trading Pits," The Journal of Economic Education, Taylor & Francis Journals, vol. 36(3), pages 220-235, July.
    16. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1986. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project," American Economic Review, American Economic Association, vol. 76(4), pages 587-603, September.
    17. Burnham, Terence C. & Cesarini, David & Johannesson, Magnus & Lichtenstein, Paul & Wallace, Björn, 2009. "Higher cognitive ability is associated with lower entries in a p-beauty contest," Journal of Economic Behavior & Organization, Elsevier, vol. 72(1), pages 171-175, October.
    18. Gary Charness & David Masclet & Marie Claire Villeval, 2014. "The Dark Side of Competition for Status (preprint)," Working Papers halshs-01090241, HAL.
    19. David Card & Alexandre Mas & Enrico Moretti & Emmanuel Saez, 2012. "Inequality at Work: The Effect of Peer Salaries on Job Satisfaction," American Economic Review, American Economic Association, vol. 102(6), pages 2981-3003, October.
    20. Vernon L. Smith, 1962. "An Experimental Study of Competitive Market Behavior," Journal of Political Economy, University of Chicago Press, vol. 70, pages 111-111.
    21. Kelvin J. Lancaster, 1966. "A New Approach to Consumer Theory," Journal of Political Economy, University of Chicago Press, vol. 74, pages 132-132.
    22. Charles A. Holt, 1999. "Teaching Economics with Classroom Experiments: A Symposium," Southern Economic Journal, John Wiley & Sons, vol. 65(3), pages 603-610, January.
    23. Stefano DellaVigna & Devin Pope, 2018. "What Motivates Effort? Evidence and Expert Forecasts," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(2), pages 1029-1069.
    24. Ariel Rubinstein, 2007. "Instinctive and Cognitive Reasoning: A Study of Response Times," Economic Journal, Royal Economic Society, vol. 117(523), pages 1243-1259, October.
    25. Bruno S. Frey & Reto Jegen, 2001. "Motivation Crowding Theory," Journal of Economic Surveys, Wiley Blackwell, vol. 15(5), pages 589-611, December.
    26. McCullough, B. D. & McGeary, Kerry Anne & Harrison, Teresa D., 2006. "Lessons from the JMCB Archive," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(4), pages 1093-1107, June.
    27. Dietmar Fehr & Steffen Huck, 2016. "Who knows it is a game? On strategic awareness and cognitive ability," Experimental Economics, Springer;Economic Science Association, vol. 19(4), pages 713-726, December.
    28. Zacharias Maniadis & Fabio Tufano & John A. List, 2017. "To Replicate or Not To Replicate? Exploring Reproducibility in Economics through the Lens of a Model and a Pilot Study," Economic Journal, Royal Economic Society, vol. 127(605), pages 209-235, October.
    29. Lucas C. Coffman & Muriel Niederle, 2015. "Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 81-98, Summer.
    30. Weber, Roberto A., 2003. "'Learning' with no feedback in a competitive guessing game," Games and Economic Behavior, Elsevier, vol. 44(1), pages 134-144, July.
    31. Shubik, Martin, 2002. "Game theory and experimental gaming," Handbook of Game Theory with Economic Applications, in: R.J. Aumann & S. Hart (ed.), Handbook of Game Theory with Economic Applications, edition 1, volume 3, chapter 62, pages 2327-2351, Elsevier.
    32. Kräkel, Matthias, 2008. "Emotions in tournaments," Journal of Economic Behavior & Organization, Elsevier, vol. 67(1), pages 204-214, July.
    33. Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
    34. Rick, Scott & Weber, Roberto A., 2010. "Meaningful learning and transfer of learning in games played repeatedly without feedback," Games and Economic Behavior, Elsevier, vol. 68(2), pages 716-730, March.
    35. Paul Gertler & Sebastian Galiani & Mauricio Romero, 2018. "How to make replication the norm," Nature, Nature, vol. 554(7693), pages 417-419, February.
    36. Rubinstein, Ariel, 1999. "Experience from a Course in Game Theory: Pre- and Postclass Problem Sets as a Didactic Device," Games and Economic Behavior, Elsevier, vol. 28(1), pages 155-170, July.
    37. Tompkinson, Paul & Bethwaite, Judy, 1995. "The ultimatum game: raising the stakes," Journal of Economic Behavior & Organization, Elsevier, vol. 27(3), pages 439-451, August.
    38. Tisha L. N. Emerson & Beck A. Taylor, 2004. "Comparing Student Achievement across Experimental and Lecture-Oriented Sections of a Principles of Microeconomics Course," Southern Economic Journal, John Wiley & Sons, vol. 70(3), pages 672-693, January.
    39. Nagel, Rosemarie, 2008. "Experimental Beauty Contest Games: Levels of Reasoning and Convergence to Equilibrium," Handbook of Experimental Economics Results, in: Charles R. Plott & Vernon L. Smith (ed.), Handbook of Experimental Economics Results, edition 1, volume 1, chapter 45, pages 391-410, Elsevier.
    40. Yvonne Durham & Thomas Mckinnon & Craig Schulman, 2007. "Classroom Experiments: Not Just Fun And Games," Economic Inquiry, Western Economic Association International, vol. 45(1), pages 162-178, January.
    41. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    42. Avinash Dixit, 2005. "Restoring Fun to Game Theory," The Journal of Economic Education, Taylor & Francis Journals, vol. 36(3), pages 205-219, July.
    43. Zacharias Maniadis & Fabio Tufano & John A. List, 2017. "To Replicate or Not To Replicate? Exploring Reproducibility in Economics through the Lens of a Model and a Pilot Study," Economic Journal, Royal Economic Society, vol. 127(605), pages 209-235, October.
    44. Alfredo Di Tillio & Marco Ottaviani & Peter Norman Sørensen, 2017. "Persuasion Bias in Science: Can Economics Help?," Economic Journal, Royal Economic Society, vol. 127(605), pages 266-304, October.
    45. Ariel Rubinstein, 2007. "Instinctive and Cognitive Reasoning: Response Times Study," Levine's Bibliography 321307000000001011, UCLA Department of Economics.
    46. Veszteg, Róbert F. & Funaki, Yukihiko, 2018. "Monetary payoffs and utility in laboratory experiments," Journal of Economic Psychology, Elsevier, vol. 65(C), pages 108-121.
    47. Samuel Bowles & Sandra Polania-Reyes, 2012. "Economic Incentives and Social Preferences: Substitutes or Complements?," Journal of Economic Literature, American Economic Association, vol. 50(2), pages 368-425, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Farolfi, Stefano & Erdlenbruch, Katrin, 2020. "A classroom experimental game to improve the understanding of asymmetric common-pool resource dilemmas in irrigation water management," International Review of Economics Education, Elsevier, vol. 35(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    2. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
    3. Luigi Butera & Philip Grossman & Daniel Houser & John List & Marie-Claire Villeval, 2020. "A New Mechanism to Alleviate the Crises of Confidence in Science - With an Application to the Public Goods Game," Artefactual Field Experiments 00684, The Field Experiments Website.
    4. Grabiszewski, Konrad & Horenstein, Alex, 2020. "Effort is not a monotonic function of skills: Results from a global mobile experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 176(C), pages 634-652.
    5. Shu-Heng Chen & Ye-Rong Du & Lee-Xieng Yang, 2014. "Cognitive capacity and cognitive hierarchy: a study based on beauty contest experiments," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 9(1), pages 69-105, April.
    6. Sheen S. Levine & Mark Bernard & Rosemarie Nagel, 2018. "Strategic intelligence: The cognitive capability to anticipate competitor behaviour," Strategic Management Journal, Wiley Blackwell, vol. 39(2), pages 527-527, February.
    7. Erkal, Nisvan & Gangadharan, Lata & Koh, Boon Han, 2018. "Monetary and non-monetary incentives in real-effort tournaments," European Economic Review, Elsevier, vol. 101(C), pages 528-545.
    8. Nagel, Rosemarie & Bühren, Christoph & Frank, Björn, 2017. "Inspired and inspiring: Hervé Moulin and the discovery of the beauty contest game," Mathematical Social Sciences, Elsevier, vol. 90(C), pages 191-207.
    9. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    10. Duffy, Sean & Smith, John, 2011. "Cognitive load in the multi-player prisoner's dilemma game," MPRA Paper 30856, University Library of Munich, Germany.
    11. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," Research Policy, Elsevier, vol. 48(1), pages 62-83.
    12. Carlos Alós-Ferrer & Johannes Buckenmaier, 2021. "Cognitive sophistication and deliberation times," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 558-592, June.
    13. Taylor, Matthew P., 2020. "Heterogeneous motivation and cognitive ability in the lab," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 85(C).
    14. Duffy, Sean & Naddeo, JJ & Owens, David & Smith, John, 2016. "Cognitive load and mixed strategies: On brains and minimax," MPRA Paper 71878, University Library of Munich, Germany.
    15. Luigino Bruni & Vittorio Pelligra & Tommaso Reggiani & Matteo Rizzolli, 2020. "The Pied Piper: Prizes, Incentives, and Motivation Crowding-in," Journal of Business Ethics, Springer, vol. 166(3), pages 643-658, October.
    16. Duffy, Sean & Smith, John, 2014. "Cognitive load in the multi-player prisoner's dilemma game: Are there brains in games?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 51(C), pages 47-56.
    17. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    18. Chen, Chia-Ching & Chiu, I-Ming & Smith, John & Yamada, Tetsuji, 2013. "Too smart to be selfish? Measures of cognitive ability, social preferences, and consistency," Journal of Economic Behavior & Organization, Elsevier, vol. 90(C), pages 112-122.
    19. Cadsby, C. Bram & Song, Fei & Engle-Warnick, Jim & Fang, Tony, 2019. "Invoking social comparison to improve performance by ranking employees: The moderating effects of public ranking, rank pay, and individual risk attitude," Journal of Economic Psychology, Elsevier, vol. 72(C), pages 64-79.
    20. Brinkerink, Jasper & De Massis, Alfredo & Kellermanns, Franz, 2022. "One finding is no finding: Toward a replication culture in family business research," Journal of Family Business Strategy, Elsevier, vol. 13(4).

    More about this item

    Keywords

    Classroom experiments; Replication; Induced value theory; Intrinsic motivation; Guessing game;
    All these keywords.

    JEL classification:

    • A20 - General Economics and Teaching - - Economic Education and Teaching of Economics - - - General
    • C73 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Stochastic and Dynamic Games; Evolutionary Games
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:soceco:v:86:y:2020:i:c:s2214804319303866. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/620175 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.