IDEAS home Printed from https://ideas.repec.org/a/eee/soceco/v94y2021ics2214804321000719.html

Cognitive misers on the web: An online-experiment of incentives, cheating, and cognitive reflection

Author

Listed:
  • Ludwig, Jonas
  • Achtziger, Anja

Abstract

The Cognitive Reflection Test (CRT) has become one of the most widely used psychological tests in decision research. It also acquired considerable fame in the news, social media, and classrooms around the globe. Given its popularity, it comes as no surprise that correct answers on the CRT questions can be easily found online. How does the availability of information about the test influence its validity as a measure of cognitive reflection, particularly, if it is deployed in an online study? We present evidence that participants searched for answers on the internet. We compared two CRT versions. The crucial difference was that correct responses could be easily found online for the standard CRT, but not its modified variant. In addition, we examined if performance-based payment affected test performance and participants’ propensity to cheat. The latter was operationalized as the online search for correct answers on the CRT questions. Our results suggested that cheating occurred during the experiment, and that doing so improved performance if correct responses were easily found on the web. When the CRT items were re-phrased so that they could not be found online, performance was not improved by incentives. Hence, incentives boosted cheating rather than cognitive reflection.

Suggested Citation

  • Ludwig, Jonas & Achtziger, Anja, 2021. "Cognitive misers on the web: An online-experiment of incentives, cheating, and cognitive reflection," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 94(C).
    • Handle: RePEc:eee:soceco:v:94:y:2021:i:c:s2214804321000719
    DOI: 10.1016/j.socec.2021.101731
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S2214804321000719
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.socec.2021.101731?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Camerer, Colin F & Hogarth, Robin M, 1999. "The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 7-42, December.
    2. Brañas-Garza, Pablo & Kujal, Praveen & Lenkei, Balint, 2019. "Cognitive reflection test: Whom, how, when," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 82(C).
    3. Angela Fagerlin & Brian J. Zikmund-Fisher & Peter A. Ubel & Aleksandra Jankovic & Holly A. Derry & Dylan M. Smith, 2007. "Measuring Numeracy without a Math Test: Development of the Subjective Numeracy Scale," Medical Decision Making, , vol. 27(5), pages 672-680, September.
    4. Anja Achtziger & Carlos Alós-Ferrer, 2014. "Fast or Rational? A Response-Times Study of Bayesian Updating," Management Science, INFORMS, vol. 60(4), pages 923-938, April.
    5. Michael N. Stagnaro & Gordon Pennycook & David G. Rand, 2018. "Performance on the Cognitive Reflection Test is stable across time," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 13(3), pages 260-267, May.
    6. Oechssler, Jörg & Roider, Andreas & Schmitz, Patrick W., 2009. "Cognitive abilities and behavioral biases," Journal of Economic Behavior & Organization, Elsevier, vol. 72(1), pages 147-152, October.
    7. Kajackaite, Agne & Gneezy, Uri, 2017. "Incentives and cheating," Games and Economic Behavior, Elsevier, vol. 102(C), pages 433-444.
    8. Hoppe, Eva I. & Kusterer, David J., 2011. "Behavioral biases and cognitive reflection," Economics Letters, Elsevier, vol. 110(2), pages 97-100, February.
    9. Urs Fischbacher & Franziska Föllmi-Heusi, 2013. "Lies In Disguise—An Experimental Study On Cheating," Journal of the European Economic Association, European Economic Association, vol. 11(3), pages 525-547, June.
    10. Uri Gneezy, 2005. "Deception: The Role of Consequences," American Economic Review, American Economic Association, vol. 95(1), pages 384-394, March.
    11. Keela S. Thomson & Daniel M. Oppenheimer, 2016. "Investigating an alternate form of the cognitive reflection test," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 11(1), pages 99-113, January.
    12. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    13. Andrew Meyer & Elizabeth Zhou & Shane Frederick, 2018. "The non-effects of repeated exposure to the Cognitive Reflection Test," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 13(3), pages 246-259, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ludwig, Jonas & Cavagnaro, Daniel R. & Regenwetter, Michel, 2023. "Order-constrained inference to supplement experimental data analytics in behavioral economics: A motivational case study," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 107(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brañas-Garza, Pablo & Kujal, Praveen & Lenkei, Balint, 2019. "Cognitive reflection test: Whom, how, when," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 82(C).
    2. Ruffle, Bradley J. & Wilson, Anne E., 2019. "Tat will tell: Tattoos and time preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 566-585.
    3. Ludwig, Jonas & Cavagnaro, Daniel R. & Regenwetter, Michel, 2023. "Order-constrained inference to supplement experimental data analytics in behavioral economics: A motivational case study," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 107(C).
    4. Benjamin Enke & Uri Gneezy & Brian Hall & David Martin & Vadim Nelidov & Theo Offerman & Jeroen van de Ven, 2023. "Cognitive Biases: Mistakes or Missing Stakes?," The Review of Economics and Statistics, MIT Press, vol. 105(4), pages 818-832, July.
    5. Taylor, Matthew P., 2020. "Heterogeneous motivation and cognitive ability in the lab," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 85(C).
    6. Sascha Grehl & Andreas Tutić, 2022. "Intuition, reflection, and prosociality: Evidence from a field experiment," PLOS ONE, Public Library of Science, vol. 17(2), pages 1-14, February.
    7. Nicolas Eber & Patrick Roger & Tristan Roger, 2024. "Finance and intelligence: An overview of the literature," Journal of Economic Surveys, Wiley Blackwell, vol. 38(2), pages 503-554, April.
    8. Benjamin Enke & Uri Gneezy & Brian Hall & David Martin & Vadim Nelidov & Theo Offerman & Jeroen van de Ven, 2020. "Cognitive Biases: Mistakes or Missing Stakes?," CESifo Working Paper Series 8168, CESifo.
    9. Fossen, Frank M. & Neyse, Levent & Schröder, Carsten, 2025. "Does Cognitive Reflection Relate to Preferences and Socioeconomic Outcomes?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 3(2), pages 303-343.
    10. Antonio M. Espin & Valerio Capraro & Brice Corgnet & Simon Gachter & Roberto Hernan-Gonzalez & Praveen Kujal & Stephen Rassenti, 2021. "Differences in Cognitive Reflection Mediate Gender Differences in Social Preferences," Working Papers 21-22, Chapman University, Economic Science Institute.
    11. Alós-Ferrer, Carlos & Hügelschäfer, Sabine, 2016. "Faith in intuition and cognitive reflection," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 64(C), pages 61-70.
    12. Jinrui Pan & Jason Shachat & Sijia Wei, 2020. "Cognitive reflection and economic order quantity inventory management: An experimental investigation," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 41(6), pages 998-1009, September.
    13. Kai Duttle & Keigo Inukai, 2015. "Complexity Aversion: Influences of Cognitive Abilities, Culture and System of Thought," Economics Bulletin, AccessEcon, vol. 35(2), pages 846-855.
    14. Matthieu Raoelison & Wim De Neys, 2019. "Do we de-bias ourselves?: The impact of repeated presentation on the bat-and-ball problem," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(2), pages 170-178, March.
    15. Chen, Chia-Ching & Chiu, I-Ming & Smith, John & Yamada, Tetsuji, 2013. "Too smart to be selfish? Measures of cognitive ability, social preferences, and consistency," Journal of Economic Behavior & Organization, Elsevier, vol. 90(C), pages 112-122.
    16. Ayşegül Engin, 2021. "The cognitive ability and working memory framework: Interpreting cognitive reflection test results in the domain of the cognitive experiential theory," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 29(1), pages 227-245, March.
    17. Alós-Ferrer, Carlos & Hügelschäfer, Sabine, 2012. "Faith in intuition and behavioral biases," Journal of Economic Behavior & Organization, Elsevier, vol. 84(1), pages 182-192.
    18. Jimenez, Natalia & Rodriguez-Lara, Ismael & Tyran, Jean-Robert & Wengström, Erik, 2018. "Thinking fast, thinking badly," Economics Letters, Elsevier, vol. 162(C), pages 41-44.
    19. Rodrigo de Oliveira Leite & Ricardo Lopes Cardoso, 2020. "The Theory of Cognitive-Conditional Conservatism in Accounting," Mathematics, MDPI, vol. 8(9), pages 1-7, September.
    20. Andrew Meyer & Elizabeth Zhou & Shane Frederick, 2018. "The non-effects of repeated exposure to the Cognitive Reflection Test," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 13(3), pages 246-259, May.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:soceco:v:94:y:2021:i:c:s2214804321000719. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/620175 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.