IDEAS home Printed from https://ideas.repec.org/a/jdm/journl/v11y2016i1p99-113.html
   My bibliography  Save this article

Investigating an alternate form of the cognitive reflection test

Author

Listed:
  • Keela S. Thomson
  • Daniel M. Oppenheimer

Abstract

Much research in cognitive psychology has focused on the tendency to conserve limited cognitive resources. The CRT is the predominant measure of such miserly information processing, and also predicts a number of frequently studied decision-making traits (such as belief bias and need for cognition). However, many subjects from common subject populations have already been exposed to the questions, which might add considerable noise to data. Moreover, the CRT has been shown to be confounded with numeracy. To increase the pool of available questions and to try to address numeracy confounds, we developed and tested the CRT-2. CRT-2 questions appear to rely less on numeracy than the original CRT but appear to measure closely related constructs in other respects. Crucially, substantially fewer subjects from Amazon’s Mechanical Turk have been previously exposed to CRT-2 questions. Though our primary purpose was investigating the CRT-2, we also found that belief bias questions appear suitable as an additional source of new items. Implications and remaining Keywords: cognitive reflection, CRT.

Suggested Citation

  • Keela S. Thomson & Daniel M. Oppenheimer, 2016. "Investigating an alternate form of the cognitive reflection test," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 11(1), pages 99-113, January.
  • Handle: RePEc:jdm:journl:v:11:y:2016:i:1:p:99-113
    as

    Download full text from publisher

    File URL: http://journal.sjdm.org/15/151029/jdm151029.pdf
    Download Restriction: no

    File URL: http://journal.sjdm.org/15/151029/jdm151029.html
    Download Restriction: no

    References listed on IDEAS

    as
    1. Oechssler, Jörg & Roider, Andreas & Schmitz, Patrick W., 2009. "Cognitive abilities and behavioral biases," Journal of Economic Behavior & Organization, Elsevier, vol. 72(1), pages 147-152, October.
    2. Neil Stewart & Christoph Ungemach & Adam J. L. Harris & Daniel M. Bartels & Ben R. Newell & Gabriele Paolacci & Jesse Chandler, "undated". "The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers," Mathematica Policy Research Reports f97b669c7b3e4c2ab95c9f805, Mathematica Policy Research.
    3. Edward T. Cokely & Colleen M. Kelley, 2009. "Cognitive abilities and superior decision making under risk: A protocol analysis and process model evaluation," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 4(1), pages 20-33, February.
    4. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    5. Hoppe, Eva I. & Kusterer, David J., 2011. "Behavioral biases and cognitive reflection," Economics Letters, Elsevier, vol. 110(2), pages 97-100, February.
    6. Neil Stewart & Christoph Ungemach & Adam J. L. Harris & Daniel M. Bartels & Ben R. Newell & Gabriele Paolacci & Jesse Chandler, 2015. "The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 10(5), pages 479-491, September.
    7. Berinsky, Adam J. & Huber, Gregory A. & Lenz, Gabriel S., 2012. "Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk," Political Analysis, Cambridge University Press, vol. 20(3), pages 351-368, July.
    8. Jesse Chandler & Gabriele Paolacci & Eyal Peer & Pam Mueller & Kate A. Ratliff, 2015. "Using Nonnaive Participants Can Reduce Effect Sizes," Mathematica Policy Research Reports bffac982a56e4cfba3659e74a, Mathematica Policy Research.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jinrui Pan & Jason Shachat & Sijia Wei, 2020. "Cognitive reflection and economic order quantity inventory management: An experimental investigation," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 41(6), pages 998-1009, September.
    2. Ray Saadaoui Mallek & Mohamed Albaity, 2019. "Individual differences and cognitive reflection across gender and nationality the case of the United Arab Emirates," Cogent Economics & Finance, Taylor & Francis Journals, vol. 7(1), pages 1567965-156, January.
    3. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    4. Chen, Chia-Ching & Chiu, I-Ming & Smith, John & Yamada, Tetsuji, 2013. "Too smart to be selfish? Measures of cognitive ability, social preferences, and consistency," Journal of Economic Behavior & Organization, Elsevier, vol. 90(C), pages 112-122.
    5. Jonathan Robinson & Cheskie Rosenzweig & Aaron J Moss & Leib Litman, 2019. "Tapped out or barely tapped? Recommendations for how to harness the vast and largely unused potential of the Mechanical Turk participant pool," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-29, December.
    6. Alós-Ferrer, Carlos & Hügelschäfer, Sabine, 2016. "Faith in intuition and cognitive reflection," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 64(C), pages 61-70.
    7. Corgnet, Brice & DeSantis, Mark & Porter, David, 2020. "The distribution of information and the price efficiency of markets," Journal of Economic Dynamics and Control, Elsevier, vol. 110(C).
    8. Antoni Bosch-Domènech & Pablo Brañas-Garza & Antonio M. Espín, 2013. "Can exposure to prenatal sex hormones (2D:4D) predict cognitive reflection?," Working Papers 698, Barcelona Graduate School of Economics.
    9. Kai Duttle & Keigo Inukai, 2015. "Complexity Aversion: Influences of Cognitive Abilities, Culture and System of Thought," Economics Bulletin, AccessEcon, vol. 35(2), pages 846-855.
    10. Mohammad Noori, 2016. "Cognitive reflection as a predictor of susceptibility to behavioral anomalies," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 11(1), pages 114-120, January.
    11. Ruffle, Bradley J. & Wilson, Anne E., 2019. "Tat will tell: Tattoos and time preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 166(C), pages 566-585.
    12. Antonio Mastrogiorgio & Enrico Petracca, 2014. "Numerals as triggers of System 1 and System 2 in the ‘bat and ball’ problem," Mind & Society: Cognitive Studies in Economics and Social Sciences, Springer;Fondazione Rosselli, vol. 13(1), pages 135-148, June.
    13. Benjamin Enke & Uri Gneezy & Brian Hall & David Martin & Vadim Nelidov & Theo Offerman & Jeroen van de Ven, 2020. "Cognitive Biases: Mistakes or Missing Stakes?," CESifo Working Paper Series 8168, CESifo.
    14. Joshua Zonca & Giorgio Coricelli & Luca Polonio, 2020. "Gaze patterns disclose the link between cognitive reflection and sophistication in strategic interaction," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(2), pages 230-245, March.
    15. Ji Yong Lee & Rodolfo M. Nayga & Cary Deck & Andreas C. Drichoutis, 2020. "Cognitive Ability and Bidding Behavior in Second Price Auctions: An Experimental Study," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(5), pages 1494-1510, October.
    16. Andersson, Ola & Ingebretsen Carlson, Jim & Wengström, Erik, 2016. "Differences Attract: An Experimental Study of Focusing in Economic Choice," Working Papers 2016:15, Lund University, Department of Economics.
    17. Brañas-Garza, Pablo & Kujal, Praveen & Lenkei, Balint, 2019. "Cognitive reflection test: Whom, how, when," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 82(C).
    18. Cueva, Carlos & Iturbe-Ormaetxe, Iñigo & Mata-Pérez, Esther & Ponti, Giovanni & Sartarelli, Marcello & Yu, Haihan & Zhukova, Vita, 2016. "Cognitive (ir)reflection: New experimental evidence," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 64(C), pages 81-93.
    19. Brice Corgnet & Mark Desantis & David Porter, 2018. "What Makes a Good Trader? On the Role of Intuition and Reflection on Trader Performance," Journal of Finance, American Finance Association, vol. 73(3), pages 1113-1137, June.
    20. Lee, Ji Yong & Nayga, Rodolfo M. & Deck, Cary & Drichoutis, Andreas, 2017. "Cognitive Ability and Bidding Behavior in Experimental Auction," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 258347, Agricultural and Applied Economics Association.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jdm:journl:v:11:y:2016:i:1:p:99-113. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Jonathan Baron). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.