IDEAS home Printed from https://ideas.repec.org/a/eee/jeborg/v93y2013icp186-195.html
   My bibliography  Save this article

Claims and confounds in economic experiments

Author

Listed:
  • Zizzo, Daniel John

Abstract

We present a distinctiveness, relevance and plausibility (DRP) method for systematically evaluating potential experimental confounds. A claim is a statement being inferred on the basis of experimental data analysis. A potential confound is a statement providing a prima facie reason why the claim is not justified (other than internal weakness). In evaluating whether a potential confound is problematic, we can start by asking whether the potential confound is distinctive from the claim; we can then ask whether it is relevant for the claim; and we can conclude by asking whether it is plausible in the light of the evidence.

Suggested Citation

  • Zizzo, Daniel John, 2013. "Claims and confounds in economic experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 93(C), pages 186-195.
  • Handle: RePEc:eee:jeborg:v:93:y:2013:i:c:p:186-195
    DOI: 10.1016/j.jebo.2013.05.006
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167268113001364
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jebo.2013.05.006?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hoffman Elizabeth & McCabe Kevin & Shachat Keith & Smith Vernon, 1994. "Preferences, Property Rights, and Anonymity in Bargaining Games," Games and Economic Behavior, Elsevier, vol. 7(3), pages 346-380, November.
    2. Rutström, E. Elisabet & Wilcox, Nathaniel T., 2009. "Stated beliefs versus inferred beliefs: A methodological inquiry and experimental test," Games and Economic Behavior, Elsevier, vol. 67(2), pages 616-632, November.
    3. Haisley, Emily C. & Weber, Roberto A., 2010. "Self-serving interpretations of ambiguity in other-regarding behavior," Games and Economic Behavior, Elsevier, vol. 68(2), pages 614-625, March.
    4. John D. Hey, 2018. "Why We Should Not Be Silent About Noise," World Scientific Book Chapters, in: Experiments in Economics Decision Making and Markets, chapter 13, pages 309-329, World Scientific Publishing Co. Pte. Ltd..
    5. Matthias Sutter & Martin G. Kocher & Sabine Strauss, 2009. "Individuals and teams in auctions," Oxford Economic Papers, Oxford University Press, vol. 61(2), pages 380-394, April.
    6. Ananish Chaudhuri & Andrew Schotter & Barry Sopher, 2009. "Talking Ourselves to Efficiency: Coordination in Inter‐Generational Minimum Effort Games with Private, Almost Common and Common Knowledge of Advice," Economic Journal, Royal Economic Society, vol. 119(534), pages 91-122, January.
    7. Simon Gächter & Elke Renner, 2010. "The effects of (incentivized) belief elicitation in public goods experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(3), pages 364-377, September.
    8. Marco Casari & John C. Ham & John H. Kagel, 2007. "Selection Bias, Demographic Effects, and Ability Effects in Common Value Auction Experiments," American Economic Review, American Economic Association, vol. 97(4), pages 1278-1304, September.
    9. Todd L. Cherry & Peter Frykblom & Jason F. Shogren, 2002. "Hardnose the Dictator," American Economic Review, American Economic Association, vol. 92(4), pages 1218-1221, September.
    10. Cooper, David J. & Kagel, John H., 2003. "The impact of meaningful context on strategic play in signaling games," Journal of Economic Behavior & Organization, Elsevier, vol. 50(3), pages 311-337, March.
    11. Zizzo, Daniel John & Fleming, Piers, 2011. "Can experimental measures of sensitivity to social pressure predict public good contribution?," Economics Letters, Elsevier, vol. 111(3), pages 239-242, June.
    12. Smith, Vernon L., 2010. "Theory and experiment: What are the questions?," Journal of Economic Behavior & Organization, Elsevier, vol. 73(1), pages 3-15, January.
    13. Daniel J. Benjamin & James J. Choi & A. Joshua Strickland, 2010. "Social Identity and Preferences," American Economic Review, American Economic Association, vol. 100(4), pages 1913-1928, September.
    14. Olivier Armantier, 2006. "Do Wealth Differences Affect Fairness Considerations?," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 47(2), pages 391-429, May.
    15. Davidson, Russell & MacKinnon, James G., 1993. "Estimation and Inference in Econometrics," OUP Catalogue, Oxford University Press, number 9780195060119, Decembrie.
    16. Martin Jones, 2008. "On the autonomy of experiments in economics," Journal of Economic Methodology, Taylor & Francis Journals, vol. 15(4), pages 391-407.
    17. Edward L. Millner & Michael D. Pratt & Robert J. Reilly, 1990. "Contestability in Real-Time Experimental Flow Markets," RAND Journal of Economics, The RAND Corporation, vol. 21(4), pages 584-599, Winter.
    18. Harrison, Glenn W. & Lau, Morten I. & Elisabet Rutström, E., 2009. "Risk attitudes, randomization to treatment, and self-selection into experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 70(3), pages 498-507, June.
    19. David Reinstein & Gerhard Riener, 2012. "Reputation and influence in charitable giving: an experiment," Theory and Decision, Springer, vol. 72(2), pages 221-243, February.
    20. Glenn Harrison, 2007. "House money effects in public good experiments: Comment," Experimental Economics, Springer;Economic Science Association, vol. 10(4), pages 429-437, December.
    21. Ernst Fehr & Jean-Robert Tyran, 2001. "Does Money Illusion Matter?," American Economic Review, American Economic Association, vol. 91(5), pages 1239-1262, December.
    22. David J. Cooper & John H. Kagel, 2009. "The Role of Context and Team Play in Cross-Game Learning," Journal of the European Economic Association, MIT Press, vol. 7(5), pages 1101-1139, September.
    23. Branas-Garza, Pablo, 2006. "Poverty in dictator games: Awakening solidarity," Journal of Economic Behavior & Organization, Elsevier, vol. 60(3), pages 306-320, July.
    24. Huck, Steffen & Normann, Hans-Theo & Oechssler, Jorg, 1999. "Learning in Cournot Oligopoly--An Experiment," Economic Journal, Royal Economic Society, vol. 109(454), pages 80-95, March.
    25. Berg Joyce & Dickhaut John & McCabe Kevin, 1995. "Trust, Reciprocity, and Social History," Games and Economic Behavior, Elsevier, vol. 10(1), pages 122-142, July.
    26. Till Requate & Israel Waichman, 2011. "“A profit table or a profit calculator?” A note on the design of Cournot oligopoly experiments," Experimental Economics, Springer;Economic Science Association, vol. 14(1), pages 36-46, March.
    27. John A. List, 2007. "On the Interpretation of Giving in Dictator Games," Journal of Political Economy, University of Chicago Press, vol. 115, pages 482-493.
    28. Huck, Steffen & Normann, Hans-Theo & Oechssler, Jorg, 2004. "Two are few and four are many: number effects in experimental oligopolies," Journal of Economic Behavior & Organization, Elsevier, vol. 53(4), pages 435-446, April.
    29. Rigdon, Mary & Ishii, Keiko & Watabe, Motoki & Kitayama, Shinobu, 2009. "Minimal social cues in the dictator game," Journal of Economic Psychology, Elsevier, vol. 30(3), pages 358-367, June.
    30. Fong, Christina M. & Luttmer, Erzo F.P., 2011. "Do fairness and race matter in generosity? Evidence from a nationally representative charity experiment," Journal of Public Economics, Elsevier, vol. 95(5-6), pages 372-394, June.
    31. Smith, Vernon L, 1982. "Microeconomic Systems as an Experimental Science," American Economic Review, American Economic Association, vol. 72(5), pages 923-955, December.
    32. Friedman,Daniel & Sunder,Shyam, 1994. "Experimental Methods," Cambridge Books, Cambridge University Press, number 9780521456821, October.
    33. R. Cookson, 2000. "Framing Effects in Public Goods Experiments," Experimental Economics, Springer;Economic Science Association, vol. 3(1), pages 55-79, June.
    34. Nikiforakis, Nikos, 2008. "Punishment and counter-punishment in public good games: Can we really govern ourselves," Journal of Public Economics, Elsevier, vol. 92(1-2), pages 91-112, February.
    35. Jen Shang & Rachel Croson, 2009. "A Field Experiment in Charitable Contribution: The Impact of Social Information on the Voluntary Provision of Public Goods," Economic Journal, Royal Economic Society, vol. 119(540), pages 1422-1439, October.
    36. Daniel Zizzo, 2010. "Experimenter demand effects in economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(1), pages 75-98, March.
    37. Robert Slonim & Alvin E. Roth, 1998. "Learning in High Stakes Ultimatum Games: An Experiment in the Slovak Republic," Econometrica, Econometric Society, vol. 66(3), pages 569-596, May.
    38. Francesco Guala, 2002. "On the scope of experiments in economics: comments on Siakantaris," Cambridge Journal of Economics, Oxford University Press, vol. 26(2), pages 261-267, March.
    39. Matthias Sutter & Martin Kocher & Sabine Strauß, "undated". "Individuals and teams in UMTS-license auctions," Working Papers 2007-23, Faculty of Economics and Statistics, Universität Innsbruck.
    40. Oberholzer-Gee Felix & Eichenberger Reiner, 2008. "Fairness in Extended Dictator Game Experiments," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 8(1), pages 1-21, July.
    41. Simon Gächter & Daniele Nosenzo & Martin Sefton, 2013. "Peer Effects In Pro-Social Behavior: Social Norms Or Social Preferences?," Journal of the European Economic Association, European Economic Association, vol. 11(3), pages 548-573, June.
    42. Arthur Schram, 2005. "Artificiality: The tension between internal and external validity in economic experiments," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 225-237.
    43. Nicholas Bardsley, 2005. "Experimental economics and the artificiality of alteration," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 239-251.
    44. Ferraro Paul J & Vossler Christian A, 2010. "The Source and Significance of Confusion in Public Goods Experiments," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 10(1), pages 1-42, July.
    45. Guala, Francesco & Mittone, Luigi, 2010. "Paradigmatic experiments: The Dictator Game," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 39(5), pages 578-584, October.
    46. Nava Ashraf & Iris Bohnet & Nikita Piankov, 2006. "Decomposing trust and trustworthiness," Experimental Economics, Springer;Economic Science Association, vol. 9(3), pages 193-208, September.
    47. Guala,Francesco, 2005. "The Methodology of Experimental Economics," Cambridge Books, Cambridge University Press, number 9780521618618, October.
    48. Robert Sugden, 2005. "Experiments as exhibits and experiments as tests," Journal of Economic Methodology, Taylor & Francis Journals, vol. 12(2), pages 291-302.
    49. Vernon L. Smith, 1994. "Economics in the Laboratory," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 113-131, Winter.
    50. James Andreoni & John Miller, 2002. "Giving According to GARP: An Experimental Test of the Consistency of Preferences for Altruism," Econometrica, Econometric Society, vol. 70(2), pages 737-753, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Grolleau, Gilles & Mungan, Murat C. & Mzoughi, Naoufel, 2022. "Seemingly irrelevant information? The impact of legal team size on third party perceptions," International Review of Law and Economics, Elsevier, vol. 71(C).
    2. Sean Fahle & Santiago I. Sautua, 2021. "How do risk attitudes affect pro-social behavior? Theory and experiment," Theory and Decision, Springer, vol. 91(1), pages 101-122, July.
    3. Jonathan H.W. Tan & Zhao Zichen & Daniel John Zizzo, 2023. "Scientific Inference from Field and Laboratory Economic Experiments: Empirical Evidence," Discussion Papers Series 663, School of Economics, University of Queensland, Australia.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniel John Zizzo, 2013. "Do dictator games measure altruism?," Chapters, in: Luigino Bruni & Stefano Zamagni (ed.), Handbook on the Economics of Reciprocity and Social Enterprise, chapter 10, pages 108-111, Edward Elgar Publishing.
    2. Matteo M. Galizzi & Daniel Navarro-Martinez, 2019. "On the External Validity of Social Preference Games: A Systematic Lab-Field Study," Management Science, INFORMS, vol. 65(3), pages 976-1002, March.
    3. Daniel Zizzo, 2010. "Experimenter demand effects in economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(1), pages 75-98, March.
    4. James C. Cox & Vjollca Sadiraj, 2018. "Incentives," Experimental Economics Center Working Paper Series 2018-01, Experimental Economics Center, Andrew Young School of Policy Studies, Georgia State University.
    5. Bogliacino, Francesco & Codagnone, Cristiano, 2021. "Microfoundations, behaviour, and evolution: Evidence from experiments," Structural Change and Economic Dynamics, Elsevier, vol. 56(C), pages 372-385.
    6. Murnighan, J. Keith & Wang, Long, 2016. "The social world as an experimental game," Organizational Behavior and Human Decision Processes, Elsevier, vol. 136(C), pages 80-94.
    7. Fiore, Annamaria, 2009. "Experimental Economics: Some Methodological Notes," MPRA Paper 12498, University Library of Munich, Germany.
    8. Eckel, Catherine & Gintis, Herbert, 2010. "Blaming the messenger: Notes on the current state of experimental economics," Journal of Economic Behavior & Organization, Elsevier, vol. 73(1), pages 109-119, January.
    9. Daniel John Zizzo, 2012. "Inducing natural group identity: A RDP analysis," Working Paper series, University of East Anglia, Centre for Behavioural and Experimental Social Science (CBESS) 12-01, School of Economics, University of East Anglia, Norwich, UK..
    10. Blair Cleave & Nikos Nikiforakis & Robert Slonim, 2013. "Is there selection bias in laboratory experiments? The case of social and risk preferences," Experimental Economics, Springer;Economic Science Association, vol. 16(3), pages 372-382, September.
    11. Cox, James C. & Friedman, Daniel & Gjerstad, Steven, 2007. "A tractable model of reciprocity and fairness," Games and Economic Behavior, Elsevier, vol. 59(1), pages 17-45, April.
    12. Masiliūnas, Aidas & Nax, Heinrich H., 2020. "Framing and repeated competition," Games and Economic Behavior, Elsevier, vol. 124(C), pages 604-619.
    13. Guala, Francesco & Mittone, Luigi, 2010. "Paradigmatic experiments: The Dictator Game," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 39(5), pages 578-584, October.
    14. Steven D. Levitt & John A. List, 2007. "Viewpoint: On the generalizability of lab behaviour to the field," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 40(2), pages 347-370, May.
    15. Michalis Drouvelis & Benjamin M. Marx, 2021. "Dimensions of donation preferences: the structure of peer and income effects," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 274-302, March.
    16. Umer, Hamza, 2020. "Revisiting generosity in the dictator game: Experimental evidence from Pakistan," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 84(C).
    17. Zhang, Le & Ortmann, Andreas, 2016. "Pro-social or anti-social, or both? A within- and between-subjects study of social preferences," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 62(C), pages 23-32.
    18. Levati, Maria Vittoria & Miettinen, Topi & Rai, Birendra, 2011. "Context and interpretation in laboratory experiments: The case of reciprocity," Journal of Economic Psychology, Elsevier, vol. 32(5), pages 846-856.
    19. Anna Dreber & Tore Ellingsen & Magnus Johannesson & David Rand, 2013. "Do people care about social context? Framing effects in dictator games," Experimental Economics, Springer;Economic Science Association, vol. 16(3), pages 349-371, September.
    20. Cleave, Blair L. & Nikiforakis, Nikos & Slonim, Robert, 2010. "Is There Selection Bias in Laboratory Experiments?," Working Papers 2010-01, University of Sydney, School of Economics.

    More about this item

    Keywords

    Confounds; Claims; Experimental design; Methodology; Internal validity; External validity;
    All these keywords.

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:93:y:2013:i:c:p:186-195. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jebo .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.