IDEAS home Printed from https://ideas.repec.org/a/eee/gamebe/v157y2026icp153-185.html

Strategy vs. direct-response method: evidence from a large online experiment on simple social dilemmas

Author

Listed:
  • Roel, Marcus
  • Chen, Zhuoqiong

Abstract

This paper examines behavioral differences in sequential games that arise when choices are elicited via the direct-response method, where players observe the choices made by those who acted earlier and respond with a single choice, or strategy method, where they make their choice(s) in response to all possible choices by those who acted before regardless of whether such actions were taken. We conducted a 2 × 2 between-subject large-scale online experiment with over 8000 participants on Amazon MTurk, manipulating both the elicitation method and participants’ ex-ante beliefs about player 1’s choices via an information-provision treatment. In neutrally framed binary-choice games, a sequential Prisoners’ Dilemma and a mini-Ultimatum Game, we document that the strategy method does not appear to alter player 2’s preferences, i.e., their tendency to reward cooperation or reject unfair offers. However, it reduces the tendency to reward defection and reject fair offers, which we interpret as a reduction in mistakes.

Suggested Citation

  • Roel, Marcus & Chen, Zhuoqiong, 2026. "Strategy vs. direct-response method: evidence from a large online experiment on simple social dilemmas," Games and Economic Behavior, Elsevier, vol. 157(C), pages 153-185.
    • Handle: RePEc:eee:gamebe:v:157:y:2026:i:c:p:153-185
    DOI: 10.1016/j.geb.2025.10.010
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0899825625001617
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.geb.2025.10.010?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. John Horton & David Rand & Richard Zeckhauser, 2011. "The online laboratory: conducting experiments in a real labor market," Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 399-425, September.
    2. Yukun Zhao & Xiaobo Zhao, 2018. "On elicitation-method effect in game experiments: a competing newsvendor perspective," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 69(4), pages 541-555, April.
    3. Benjamin Enke, 2020. "What You See Is All There Is," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 135(3), pages 1363-1398.
    4. Guth, Werner & Tietz, Reinhard, 1990. "Ultimatum bargaining behavior : A survey and comparison of experimental results," Journal of Economic Psychology, Elsevier, vol. 11(3), pages 417-449, September.
    5. Jordi Brandts & Gary Charness, 2011. "The strategy versus the direct-response method: a first survey of experimental comparisons," Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 375-398, September.
    6. Battigalli, Pierpaolo & Dufwenberg, Martin & Smith, Alec, 2019. "Frustration, aggression, and anger in leader-follower games," Games and Economic Behavior, Elsevier, vol. 117(C), pages 15-39.
    7. Armin Falk & Ernst Fehr & Urs Fischbacher, 2005. "Driving Forces Behind Informal Sanctions," Econometrica, Econometric Society, vol. 73(6), pages 2017-2030, November.
    8. Loewenstein, George, 1996. "Out of Control: Visceral Influences on Behavior," Organizational Behavior and Human Decision Processes, Elsevier, vol. 65(3), pages 272-292, March.
    9. Paolacci, Gabriele & Chandler, Jesse & Ipeirotis, Panagiotis G., 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Cambridge University Press, vol. 5(5), pages 411-419, August.
    10. Ernesto Reuben & Sigrid Suetens, 2012. "Revisiting strategic versus non-strategic cooperation," Experimental Economics, Springer;Economic Science Association, vol. 15(1), pages 24-43, March.
    11. Oxoby, Robert J. & McLeish, Kendra N., 2004. "Sequential decision and strategy vector methods in ultimatum bargaining: evidence on the strength of other-regarding behavior," Economics Letters, Elsevier, vol. 84(3), pages 399-405, September.
    12. Ignacio Esponda & Emanuel Vespa, 2018. "Endogenous sample selection: A laboratory study," Quantitative Economics, Econometric Society, vol. 9(1), pages 183-216, March.
    13. Minozzi, William & Woon, Jonathan, 2020. "Direct response and the strategy method in an experimental cheap talk game," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 85(C).
    14. Ofra Amir & David G Rand & Ya'akov Kobi Gal, 2012. "Economic Games on the Internet: The Effect of $1 Stakes," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-4, February.
    15. Güth, Werner & Kocher, Martin G., 2014. "More than thirty years of ultimatum bargaining experiments: Motives, variations, and a survey of the recent literature," Journal of Economic Behavior & Organization, Elsevier, vol. 108(C), pages 396-409.
    16. Khalmetski, Kiryl & Ockenfels, Axel & Werner, Peter, 2015. "Surprising gifts: Theory and laboratory evidence," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 163-208.
    17. David Amdur & Ethan Schmick, 2013. "Does the direct-response method induce guilt aversion in a trust game?," Economics Bulletin, AccessEcon, vol. 33(1), pages 687-693.
    18. Gary Charness & Matthew Rabin, 2002. "Understanding Social Preferences with Simple Tests," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(3), pages 817-869.
    19. Guth, Werner & Huck, Steffen & Muller, Wieland, 2001. "The Relevance of Equal Splits in Ultimatum Games," Games and Economic Behavior, Elsevier, vol. 37(1), pages 161-169, October.
    20. Fischbacher, Urs & Gachter, Simon & Fehr, Ernst, 2001. "Are people conditionally cooperative? Evidence from a public goods experiment," Economics Letters, Elsevier, vol. 71(3), pages 397-404, June.
    21. Luyao Li & Xiaobo Zhao & Dong Xie & Xue Xiao, 2024. "On difference between direct-response method and strategy method in decision-making: behavioural and neural evidence in a reward-punishment game," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 75(9), pages 1681-1698, September.
    22. Daniel L. Chen & Martin Schonger, 2024. "Invariance of equilibrium to the strategy method I: theory," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 10(1), pages 15-30, June.
    23. Recalde, María P. & Riedl, Arno & Vesterlund, Lise, 2018. "Error-prone inference from response time: The case of intuitive generosity in public-good games," Journal of Public Economics, Elsevier, vol. 160(C), pages 132-147.
    24. Alejandro Martínez-Marquina & Muriel Niederle & Emanuel Vespa, 2019. "Failures in Contingent Reasoning: The Role of Uncertainty," American Economic Review, American Economic Association, vol. 109(10), pages 3437-3474, October.
    25. Swee-Hoon Chuah & Robert Hoffmann & Jeremy Larner, 2014. "Elicitation effects in a multi-stage bargaining experiment," Experimental Economics, Springer;Economic Science Association, vol. 17(2), pages 335-345, June.
    26. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    27. Martinsson, Peter & Pham-Khanh, Nam & Villegas-Palacio, Clara, 2013. "Conditional cooperation and disclosure in developing countries," Journal of Economic Psychology, Elsevier, vol. 34(C), pages 148-155.
    28. Machina, Mark J, 1989. "Dynamic Consistency and Non-expected Utility Models of Choice under Uncertainty," Journal of Economic Literature, American Economic Association, vol. 27(4), pages 1622-1668, December.
    29. Daniel L. Chen & Martin Schonger, 2024. "Invariance of equilibrium to the strategy method II: experimental evidence," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 10(1), pages 31-43, June.
    30. Columbus, Simon & Böhm, Robert, 2021. "Norm shifts under the strategy method," Judgment and Decision Making, Cambridge University Press, vol. 16(5), pages 1267-1289, September.
    31. Anna Dreber & Tore Ellingsen & Magnus Johannesson & David Rand, 2013. "Do people care about social context? Framing effects in dictator games," Experimental Economics, Springer;Economic Science Association, vol. 16(3), pages 349-371, September.
    32. Jordi Brandts & Gary Charness, 2003. "Truth or Consequences: An Experiment," Management Science, INFORMS, vol. 49(1), pages 116-130, January.
    33. Ellingsen, Tore & Johannesson, Magnus & Mollerstrom, Johanna & Munkhammar, Sara, 2012. "Social framing effects: Preferences or beliefs?," Games and Economic Behavior, Elsevier, vol. 76(1), pages 117-130.
    34. Ockenfels, Axel & Werner, Peter, 2014. "Scale manipulation in dictator games," Journal of Economic Behavior & Organization, Elsevier, vol. 97(C), pages 138-142.
    35. Aina, Chiara & Battigalli, Pierpaolo & Gamba, Astrid, 2020. "Frustration and anger in the Ultimatum Game: An experiment," Games and Economic Behavior, Elsevier, vol. 122(C), pages 150-167.
    36. Jordi Brandts & Gary Charness, 2000. "Hot vs. Cold: Sequential Responses and Preference Stability in Experimental Games," Experimental Economics, Springer;Economic Science Association, vol. 2(3), pages 227-238, March.
    37. Giovanni Bartolomeo & Stefano Papa, 2016. "Trust and reciprocity: extensions and robustness of triadic design," Experimental Economics, Springer;Economic Science Association, vol. 19(1), pages 100-115, March.
    38. M. Kathleen Ngangoué & Georg Weizsäcker, 2021. "Learning from Unrealized versus Realized Prices," American Economic Journal: Microeconomics, American Economic Association, vol. 13(2), pages 174-201, May.
    39. Croson, Rachel T. A., 1999. "The Disjunction Effect and Reason-Based Choice in Games, , , , , , , , , , , , ," Organizational Behavior and Human Decision Processes, Elsevier, vol. 80(2), pages 118-133, November.
    40. Jillian Jordan & Katherine McAuliffe & David Rand, 2016. "The effects of endowment size and strategy method on third party punishment," Experimental Economics, Springer;Economic Science Association, vol. 19(4), pages 741-763, December.
    41. David K. Levine, 1998. "Modeling Altruism and Spitefulness in Experiment," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(3), pages 593-622, July.
    42. Guillaume R. Fréchette & Kim Sarnoff & Leeat Yariv, 2022. "Experimental Economics: Past and Future," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 777-794, August.
    43. Gabriele Paolacci & Jesse Chandler & Panagiotis G. Ipeirotis, 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 5(5), pages 411-419, August.
    44. Bolle, Friedel & Ockenfels, Peter, 1990. "Prisoners' Dilemma as a game with incomplete information," Journal of Economic Psychology, Elsevier, vol. 11(1), pages 69-84, March.
    45. Johannes Moser, 2019. "Hypothetical thinking and the winner’s curse: an experimental investigation," Theory and Decision, Springer, vol. 87(1), pages 17-56, July.
    46. Roth, Alvin E. & Vesna Prasnikar & Masahiro Okuno-Fujiwara & Shmuel Zamir, 1991. "Bargaining and Market Behavior in Jerusalem, Ljubljana, Pittsburgh, and Tokyo: An Experimental Study," American Economic Review, American Economic Association, vol. 81(5), pages 1068-1095, December.
    47. Joseph K. Goodman & Gabriele Paolacci, 2017. "Crowdsourcing Consumer Research," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 44(1), pages 196-210.
    48. Ignacio Esponda & Emanuel Vespa, 2024. "Contingent Thinking and the Sure-Thing Principle: Revisiting Classic Anomalies in the Laboratory," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(5), pages 2806-2831.
    49. Ingar Haaland & Christopher Roth & Johannes Wohlfart, 2023. "Designing Information Provision Experiments," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 3-40, March.
    50. Daniel L. Chen & Martin Schonger & Chris Wickens, 2016. "oTree - An open-source platform for laboratory, online, and field experiments," Post-Print hal-04315125, HAL.
    51. Guth, Werner & Schmittberger, Rolf & Schwarze, Bernd, 1982. "An experimental analysis of ultimatum bargaining," Journal of Economic Behavior & Organization, Elsevier, vol. 3(4), pages 367-388, December.
    52. Niederle, Muriel & Vespa, Emanuel, 2023. "Cognitive Limitations: Failures of Contingent Thinking," University of California at San Diego, Economics Working Paper Series qt5q14p1np, Department of Economics, UC San Diego.
    53. Matthew Rabin & Georg Weizsacker, 2009. "Narrow Bracketing and Dominated Choices," American Economic Review, American Economic Association, vol. 99(4), pages 1508-1543, September.
    54. Gary Charness & Dan Levin, 2009. "The Origin of the Winner's Curse: A Laboratory Study," American Economic Journal: Microeconomics, American Economic Association, vol. 1(1), pages 207-236, February.
    55. Yukun Zhao & Xiaobo Zhao & Lihong Wang & Yefen Chen & Xinmeng Zhang, 2016. "Does Elicitation Method Matter? Behavioral and Neuroimaging Evidence from Capacity Allocation Game," Production and Operations Management, Production and Operations Management Society, vol. 25(5), pages 919-934, May.
    56. Timothy N. Cason & Charles R. Plott, 2014. "Misconceptions and Game Form Recognition: Challenges to Theories of Revealed Preference and Framing," Journal of Political Economy, University of Chicago Press, vol. 122(6), pages 1235-1270.
    57. Ignacio Esponda Jr. & Emanuel Vespa Jr., 2014. "Hypothetical Thinking and Information Extraction in the Laboratory," American Economic Journal: Microeconomics, American Economic Association, vol. 6(4), pages 180-202, November.
    58. Muriel Niederle & Emanuel Vespa, 2023. "Cognitive Limitations: Failures of Contingent Thinking," Annual Review of Economics, Annual Reviews, vol. 15(1), pages 307-328, September.
    59. Fischbacher, Urs & Gächter, Simon & Quercia, Simone, 2012. "The behavioral validity of the strategy method in public good experiments," Journal of Economic Psychology, Elsevier, vol. 33(4), pages 897-913.
    60. David Johnson & John Barry Ryan, 2020. "Amazon Mechanical Turk workers can provide consistent and economically meaningful data," Southern Economic Journal, John Wiley & Sons, vol. 87(1), pages 369-385, July.
    61. Yukun Zhao & Xiaobo Zhao & Zuo-Jun Max Shen, 2018. "The hot-versus-cold effect in a punishment game: a multi-round experimental study," Annals of Operations Research, Springer, vol. 268(1), pages 333-355, September.
    62. Alexandros Karakostas & Nhu Tran & Daniel John Zizzo, 2022. "Experimental Insights on Anti-Social Behavior: Two Meta-Analyses," Discussion Papers Series 658, School of Economics, University of Queensland, Australia.
    63. Lu Dong & Maria Montero & Alex Possajennikov, 2018. "Communication, leadership and coordination failure," Theory and Decision, Springer, vol. 84(4), pages 557-584, June.
    64. Chen, Daniel L. & Schonger, Martin & Wickens, Chris, 2016. "oTree—An open-source platform for laboratory, online, and field experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 9(C), pages 88-97.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Columbus, Simon & Böhm, Robert, 2021. "Norm shifts under the strategy method," Judgment and Decision Making, Cambridge University Press, vol. 16(5), pages 1267-1289, September.
    2. Chen, Peikun & Li, Jianbiao & Pan, Jingjing & Zhu, Chengkang, 2025. "Expectations matter in bottom-line setting: Theory and evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 231(C).
    3. Ignacio Esponda & Emanuel Vespa, 2024. "Contingent Thinking and the Sure-Thing Principle: Revisiting Classic Anomalies in the Laboratory," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(5), pages 2806-2831.
    4. Gächter, Simon & Kölle, Felix & Quercia, Simone, 2022. "Preferences and perceptions in Provision and Maintenance public goods," Games and Economic Behavior, Elsevier, vol. 135(C), pages 338-355.
    5. Güth, Werner & Kocher, Martin G., 2014. "More than thirty years of ultimatum bargaining experiments: Motives, variations, and a survey of the recent literature," Journal of Economic Behavior & Organization, Elsevier, vol. 108(C), pages 396-409.
    6. Chiara Nardi, 2018. "Play Versus Strategy Method: Behavior and the Role of Emotions in the Ultimatum Game," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 4(1), pages 91-106, March.
    7. Miguel A. Costa‐Gomes & Yuan Ju & Jiawen Li, 2019. "Role‐Reversal Consistency: An Experimental Study Of The Golden Rule," Economic Inquiry, Western Economic Association International, vol. 57(1), pages 685-704, January.
    8. Niederle, Muriel & Vespa, Emanuel, 2023. "Cognitive Limitations: Failures of Contingent Thinking," University of California at San Diego, Economics Working Paper Series qt5q14p1np, Department of Economics, UC San Diego.
    9. Ignacio Esponda & Emanuel Vespa & Sevgi Yuksel, 2024. "Mental Models and Learning: The Case of Base-Rate Neglect," American Economic Review, American Economic Association, vol. 114(3), pages 752-782, March.
    10. Hyndman, Kyle & Walker, Matthew J., 2022. "Fairness and risk in ultimatum bargaining," Games and Economic Behavior, Elsevier, vol. 132(C), pages 90-105.
    11. Jillian Jordan & Katherine McAuliffe & David Rand, 2016. "The effects of endowment size and strategy method on third party punishment," Experimental Economics, Springer;Economic Science Association, vol. 19(4), pages 741-763, December.
    12. Yukun Zhao & Xiaobo Zhao & Zuo-Jun Max Shen, 2018. "The hot-versus-cold effect in a punishment game: a multi-round experimental study," Annals of Operations Research, Springer, vol. 268(1), pages 333-355, September.
    13. Aina, Chiara & Battigalli, Pierpaolo & Gamba, Astrid, 2020. "Frustration and anger in the Ultimatum Game: An experiment," Games and Economic Behavior, Elsevier, vol. 122(C), pages 150-167.
    14. Koppel, Lina & Andersson, David & Johannesson, Magnus & Strømland, Eirik & Tinghög, Gustav, 2025. "Comprehension in economic games," Journal of Economic Behavior & Organization, Elsevier, vol. 234(C).
    15. Chen, Daniel L. & Schonger, Martin, 2016. "A Theory of Experiments: Invariance of Equilibrium to the Strategy Method of Elicitation and Implications for Social Preferences," TSE Working Papers 16-724, Toulouse School of Economics (TSE), revised Feb 2020.
    16. Daniel L. Chen & Martin Schonger, 2024. "Invariance of equilibrium to the strategy method I: theory," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 10(1), pages 15-30, June.
    17. Manapat, Michael L. & Nowak, Martin A. & Rand, David G., 2013. "Information, irrationality, and the evolution of trust," Journal of Economic Behavior & Organization, Elsevier, vol. 90(S), pages 57-75.
    18. Daniel L. Chen & Martin Schonger, 2023. "Invariance of equilibrium to the strategy method I: theory," Post-Print hal-04550734, HAL.
    19. Falk, Armin & Fischbacher, Urs, 2006. "A theory of reciprocity," Games and Economic Behavior, Elsevier, vol. 54(2), pages 293-315, February.
    20. Chowdhury Mohammad Sakib Anwar & Konstantinos Georgalos, 2024. "Position uncertainty in a sequential public goods game: an experiment," Experimental Economics, Springer;Economic Science Association, vol. 27(4), pages 820-853, September.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C72 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Noncooperative Games
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C92 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Group Behavior
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:gamebe:v:157:y:2026:i:c:p:153-185. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/622836 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.