IDEAS home Printed from https://ideas.repec.org/a/eee/jeborg/v134y2017icp48-59.html
   My bibliography  Save this article

Experimental methods: When and why contextual instructions are important

Author

Listed:
  • Alekseev, Aleksandr
  • Charness, Gary
  • Gneezy, Uri

Abstract

An important methodological issue in experimental research is the extent to which one should use context-rich or abstract language in the instructions for an experiment. The traditional use of abstract context in experimental economics is commonly viewed as a way to achieve experimental control. However, there are some advantages to using context-framed instructions, such as “employer and worker” instead of “player 1 and player 2.” Meaningful context can enhance understanding of an environment and reduce confusion among participants, particularly when a task requires sophisticated reasoning, and hence may yield responses of better quality. In emotionally-charged research questions, such as pollution or bribes, contextual instructions may affect behavior in the experiment, but this effect may be appropriate as it relates to the research question. Our review of the evidence from the literature indicates that in the great majority of cases meaningful language is either useful or produces no change in behavior. Nevertheless, a few important considerations are worth keeping in mind when using rich context. Finally, we see the choice of context as being an expansion of the experimenter’s toolkit and a factor to consider in experimental design.

Suggested Citation

  • Alekseev, Aleksandr & Charness, Gary & Gneezy, Uri, 2017. "Experimental methods: When and why contextual instructions are important," Journal of Economic Behavior & Organization, Elsevier, vol. 134(C), pages 48-59.
  • Handle: RePEc:eee:jeborg:v:134:y:2017:i:c:p:48-59
    DOI: 10.1016/j.jebo.2016.12.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167268116302876
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde, 2009. "Homo Reciprocans: Survey Evidence on Behavioural Outcomes," Economic Journal, Royal Economic Society, vol. 119(536), pages 592-612, March.
    2. Burnham, Terence & McCabe, Kevin & Smith, Vernon L., 2000. "Friend-or-foe intentionality priming in an extensive form trust game," Journal of Economic Behavior & Organization, Elsevier, vol. 43(1), pages 57-73, September.
    3. Berninghaus, Siegfried K. & Haller, Sven & Krüger, Tyll & Neumann, Thomas & Schosser, Stephan & Vogt, Bodo, 2013. "Risk attitude, beliefs, and information in a Corruption Game – An experimental analysis," Journal of Economic Psychology, Elsevier, vol. 34(C), pages 46-60.
    4. Dufwenberg, Martin & Gächter, Simon & Hennig-Schmidt, Heike, 2011. "The framing of games and the psychology of play," Games and Economic Behavior, Elsevier, vol. 73(2), pages 459-478.
    5. Heike Hennig-Schmidt & Bettina Rockenbach & Abdolkarim Sadrieh, 2010. "In Search Of Workers' Real Effort Reciprocity-A Field and a Laboratory Experiment," Journal of the European Economic Association, MIT Press, vol. 8(4), pages 817-837, June.
    6. Itzhak Gilboa & David Schmeidler, 1995. "Case-Based Decision Theory," The Quarterly Journal of Economics, Oxford University Press, vol. 110(3), pages 605-639.
    7. David J. Cooper, 1999. "Gaming against Managers in Incentive Systems: Experimental Results with Chinese Students and Chinese Managers," American Economic Review, American Economic Association, vol. 89(4), pages 781-804, September.
    8. Daniele Nosenzo, 2013. "Pay Secrecy And Effort Provision," Economic Inquiry, Western Economic Association International, vol. 51(3), pages 1779-1794, July.
    9. Glenn W. Harrison & John A. List, 2004. "Field Experiments," Journal of Economic Literature, American Economic Association, vol. 42(4), pages 1009-1055, December.
    10. David Cooper & John Kagel & Qing Liang Gu & Wei Lo, 1999. "Gaming against managers in incentive systems: Experimental results with chinese students and chinese managers," Artefactual Field Experiments 00038, The Field Experiments Website.
    11. Abigail Barr & Danila Serra, 2009. "The effects of externalities and framing on bribery in a petty corruption experiment," Experimental Economics, Springer;Economic Science Association, vol. 12(4), pages 488-503, December.
    12. Jehiel, Philippe, 2005. "Analogy-based expectation equilibrium," Journal of Economic Theory, Elsevier, vol. 123(2), pages 81-104, August.
    13. Avrahami, Judith & Kareev, Yaakov & Todd, Peter M. & Silverman, Boaz, 2014. "Allocation of resources in asymmetric competitions: How do the weak maintain a chance of winning?," Journal of Economic Psychology, Elsevier, vol. 42(C), pages 161-174.
    14. Ritwik Banerjee, 2016. "On the interpretation of bribery in a laboratory corruption game: moral frames and social norms," Experimental Economics, Springer;Economic Science Association, vol. 19(1), pages 240-267, March.
    15. Abbink, Klaus & Dasgupta, Utteeyo & Gangadharan, Lata & Jain, Tarun, 2014. "Letting the briber go free: An experiment on mitigating harassment bribes," Journal of Public Economics, Elsevier, vol. 111(C), pages 17-28.
    16. M. Fernanda Rivas, 2013. "An Experiment On Corruption And Gender," Bulletin of Economic Research, Wiley Blackwell, vol. 65(1), pages 10-42, January.
    17. Andrei Bremzeny & Elena Khokhlovaz & Anton Suvorov & Jeroen van de Ven, 2011. "Bad News: An Experimental Study on the Informational Effects of Rewards," Working Papers w0164, New Economic School (NES).
    18. Glenn W. Harrison & Morten I. Lau & E. Elisabet Rutström, 2007. "Estimating Risk Attitudes in Denmark: A Field Experiment," Scandinavian Journal of Economics, Wiley Blackwell, vol. 109(2), pages 341-368, June.
    19. Klaus Abbink & Heike Hennig-Schmidt, 2006. "Neutral versus loaded instructions in a bribery experiment," Experimental Economics, Springer;Economic Science Association, vol. 9(2), pages 103-121, June.
    20. Engel, Christoph & Rand, David G., 2014. "What does “clean” really mean? The implicit framing of decontextualized experiments," Economics Letters, Elsevier, vol. 122(3), pages 386-389.
    21. Cox, James C & Oaxaca, Ronald L, 1989. "Laboratory Experiments with a Finite-Horizon Job-Search Model," Journal of Risk and Uncertainty, Springer, vol. 2(3), pages 301-329, September.
    22. Ellingsen, Tore & Johannesson, Magnus & Mollerstrom, Johanna & Munkhammar, Sara, 2012. "Social framing effects: Preferences or beliefs?," Games and Economic Behavior, Elsevier, vol. 76(1), pages 117-130.
    23. Rajna Gibson & Carmen Tanner & Alexander F. Wagner, 2013. "Preferences for Truthfulness: Heterogeneity among and within Individuals," American Economic Review, American Economic Association, vol. 103(1), pages 532-548, February.
    24. Pevnitskaya, Svetlana & Ryvkin, Dmitry, 2013. "Environmental context and termination uncertainty in games with a dynamic public bad," Environment and Development Economics, Cambridge University Press, vol. 18(1), pages 27-49, February.
    25. Alm, James & McClelland, Gary H. & Schulze, William D., 1992. "Why do people pay taxes?," Journal of Public Economics, Elsevier, vol. 48(1), pages 21-38, June.
    26. David J. Cooper & John H. Kagel, 2009. "The Role of Context and Team Play in Cross-Game Learning," Journal of the European Economic Association, MIT Press, vol. 7(5), pages 1101-1139, September.
    27. Elliott, Catherine S. & Hayward, Donald M. & Canon, Sebastian, 1998. "Institutional framing: Some experimental evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 35(4), pages 455-464, May.
    28. Andrei Bremzen & Elena Khokhlova & Anton Suvorov & Jeroen van de Ven, 2015. "Bad News: An Experimental Study on the Informational Effects Of Rewards," The Review of Economics and Statistics, MIT Press, vol. 97(1), pages 55-70, March.
    29. Landeo, Claudia M. & Nikitin, Maxim & Babcock, Linda, 2007. "Split-awards and disputes: An experimental study of a strategic model of litigation," Journal of Economic Behavior & Organization, Elsevier, vol. 63(3), pages 553-572, July.
    30. Philippe Jehiel, 2005. "Analogy-based Expectation Equilibrium," Post-Print halshs-00754070, HAL.
    31. Jana Krajcova & Andreas Ortmann, 2008. "Testing Leniency Programs Experimentally: The Impact of “Natural” Framing," CERGE-EI Working Papers wp372, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    32. Christine Harbring & Bernd Irlenbusch, 2011. "Sabotage in Tournaments: Evidence from a Laboratory Experiment," Management Science, INFORMS, vol. 57(4), pages 611-627, April.
    33. Jordi Brandts & David Cooper, 2006. "Observability and overcoming coordination failure in organizations: An experimental study," Experimental Economics, Springer;Economic Science Association, vol. 9(4), pages 407-423, December.
    34. Li, Danyang & Hawley, Zackary & Schnier, Kurt, 2013. "Increasing organ donation via changes in the default choice or allocation rule," Journal of Health Economics, Elsevier, vol. 32(6), pages 1117-1129.
    35. Jana Krajcova, 2008. "Testing Leniency Programs Experimentally: The Impact of Change in Parameterization," CERGE-EI Working Papers wp370, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    36. Eileen Chou & Margaret McConnell & Rosemarie Nagel & Charles Plott, 2009. "The control of game form recognition in experiments: understanding dominant strategy failures in a simple two person “guessing” game," Experimental Economics, Springer;Economic Science Association, vol. 12(2), pages 159-179, June.
    37. Michael Kirchler & Jurgen Huber & Thomas Stockl, 2012. "Thar She Bursts: Reducing Confusion Reduces Bubbles," American Economic Review, American Economic Association, vol. 102(2), pages 865-883, April.
    38. Vivi Alatas & Lisa Cameron & Ananish Chaudhuri & Nisvan Erkal & Lata Gangadharan, 2009. "Gender, Culture, and Corruption: Insights from an Experimental Analysis," Southern Economic Journal, Southern Economic Association, vol. 75(3), pages 663-680, January.
    39. Rachel Croson & Uri Gneezy, 2009. "Gender Differences in Preferences," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 448-474, June.
    40. Cooper, David J. & Kagel, John H., 2003. "The impact of meaningful context on strategic play in signaling games," Journal of Economic Behavior & Organization, Elsevier, vol. 50(3), pages 311-337, March.
    41. Samuelson, Charles D. & Allison, Scott T., 1994. "Cognitive Factors Affecting the Use of Social Decision Heuristics in Resource-Sharing Tasks," Organizational Behavior and Human Decision Processes, Elsevier, vol. 58(1), pages 1-27, April.
    42. Lambsdorff, Johann Graf & Frank, Björn, 2011. "Corrupt reciprocity - Experimental evidence on a men's game," International Review of Law and Economics, Elsevier, vol. 31(2), pages 116-125, June.
    43. Smith, Vernon L, 1976. "Experimental Economics: Induced Value Theory," American Economic Review, American Economic Association, vol. 66(2), pages 274-279, May.
    44. Smith, Vernon L & Suchanek, Gerry L & Williams, Arlington W, 1988. "Bubbles, Crashes, and Endogenous Expectations in Experimental Spot Asset Markets," Econometrica, Econometric Society, vol. 56(5), pages 1119-1151, September.
    45. Kimmo Eriksson & Pontus Strimling, 2014. "Spontaneous associations and label framing have similar effects in the public goods game," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 9(5), pages 360-372, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Erik Ansink & Mark Koetse & Jetske Bouma & Dominic Hauck & Daan van Soest, 2017. "Crowdfunding public goods: An experiment," Tinbergen Institute Discussion Papers 17-119/VIII, Tinbergen Institute.
    2. Lydia Mechtenberg & Gerd Muehlheusser & Andreas Roider, 2017. "Whistle-Blower Protection: Theory and Experimental Evidence," CESifo Working Paper Series 6394, CESifo.
    3. Shaun P. Hargreaves Heap & Kei Tsutsui & Daniel J. Zizzo, 2020. "Vote and voice: an experiment on the effects of inclusive governance rules," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 54(1), pages 111-139, January.
    4. Camille Cornand & Frank Heinemann, 2018. "Experiments on macroeconomics: methods and applications," Post-Print halshs-01809937, HAL.
    5. Niu, Xiaofei & Li, Jianbiao, 2019. "Incentivizing organ donation by swearing an oath: The role of signature and ritual," EconStor Preprints 203243, ZBW - Leibniz Information Centre for Economics.
    6. Christoph Huber & Jürgen Huber, 2020. "Bad bankers no more? Truth-telling and (dis)honesty in the finance industry," Working Papers 2020-03, Faculty of Economics and Statistics, University of Innsbruck.
    7. Subhasish M. Chowdhury & Anwesha Mukherjee & Theodore L. Turocy, 2020. "That’s the ticket: explicit lottery randomisation and learning in Tullock contests," Theory and Decision, Springer, vol. 88(3), pages 405-429, April.
    8. Ihli, Hanna Julia & Gassner, Anja & Musshoff, Oliver, 2018. "Experimental insights on the investment behavior of small-scale coffee farmers in central Uganda under risk and uncertainty," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 75(C), pages 31-44.
    9. Li, Jing, 2018. "Plastic surgery or primary care? Altruistic preferences and expected specialty choice of U.S. medical students," Journal of Health Economics, Elsevier, vol. 62(C), pages 45-59.
    10. Lydia Mechtenberg & Gerd Muehlheusser & Andreas Roider, 2017. "Whistle-Blower Protection: Theory and Experimental Evidence," CESifo Working Paper Series 6394, CESifo.
    11. Camille Cornand & Frank Heinemann, 2018. "Experiments on macroeconomics: methods and applications," Working Papers 1810, Groupe d'Analyse et de Théorie Economique Lyon St-Étienne (GATE Lyon St-Étienne), Université de Lyon.
    12. Margarita Gáfaro & César Mantilla, 2019. "Preferences, Uncertainty, and Biases in Land Division: A Bargaining Experiment in the Field," Borradores de Economia 1092, Banco de la Republica de Colombia.
    13. Meraner, Manuela & Musshoff, Oliver & Finger, Robert, 2018. "Using involvement to reduce inconsistencies in risk preference elicitation," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 73(C), pages 22-33.
    14. Yu‐Chin Hsiao & Simon Kemp & Maroš Servátka, 2020. "On the Importance of Context in Sequential Search," Southern Economic Journal, John Wiley & Sons, vol. 86(4), pages 1510-1530, April.
    15. Hsiao, Yu Chin & Kemp, Simon & Servátka, Maroš, 2019. "On the Importance of Context in Sequential Decision-Making," MPRA Paper 94027, University Library of Munich, Germany.
    16. Giuseppe Danese & Luigi Mittone, 2020. "On pledging one's trustworthiness through gifts: an experimental inquiry," CEEL Working Papers 2001, Cognitive and Experimental Economics Laboratory, Department of Economics, University of Trento, Italia.
    17. Cagala, Tobias & Glogowsky, Ulrich & Grimm, Veronika & Rincke, Johannes & Tuset-Cueva, Amanda, 2019. "Rent extraction and prosocial behavior," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 709-723.
    18. Maria Vittoria Levati & Chiara Nardi, 2019. "The power of words in a petty corruption experiment," Working Papers 18/2019, University of Verona, Department of Economics.
    19. Yu-Chin Hsiao & Simon Kemp, 2020. "The effect of incentive structure on search in the secretary problem," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(1), pages 82-92, January.
    20. David J. Freeman & Erik O. Kimbrough & Garrett M. Petersen & Hanh T. Tong, 2018. "Instructions," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 4(2), pages 165-179, December.
      • David J. Freeman & Erik O. Kimbrough & Garrett M. Petersen & Hanh T. Tong, 2017. "Instructions," Discussion Papers dp17-12, Department of Economics, Simon Fraser University.

    More about this item

    Keywords

    Experiments; Methodology; Context; Meaningful language; Quality of responses;

    JEL classification:

    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C92 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Group Behavior
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • C99 - Mathematical and Quantitative Methods - - Design of Experiments - - - Other

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:134:y:2017:i:c:p:48-59. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Haili He). General contact details of provider: http://www.elsevier.com/locate/jebo .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.