IDEAS home Printed from https://ideas.repec.org/a/bla/jconsa/v56y2022i2p938-956.html
   My bibliography  Save this article

Measurement error in research on financial literacy: How much error is there and how does it influence effect size estimates?

Author

Listed:
  • Gilles E. Gignac
  • Elizabeth Ooi

Abstract

Financial literacy is often measured with only three to five questions, suggesting challenges with achieving respectable levels of internal consistency reliability. Based on our review, we found financial literacy tests composed of three and five test questions yielded mean reliability estimates of 0.40 (k = 7; N = 167,075) and 0.54 (k = 8; N = 57,937), respectively: values less than minimally acceptable for even exploratory research. Based on our more comprehensive review of 52 samples and a variety of financial literacy tests (3–45 questions), researchers are recommended to measure financial literacy with a minimum of 13–15 questions. Finally, we conclude that the potential impact of financial literacy on various outcome variables has been underestimated substantially in many previous investigations, as the relatively low levels of internal consistency reliability in the financial literacy test scores attenuated the obtained effects estimated from the observed scores.

Suggested Citation

  • Gilles E. Gignac & Elizabeth Ooi, 2022. "Measurement error in research on financial literacy: How much error is there and how does it influence effect size estimates?," Journal of Consumer Affairs, Wiley Blackwell, vol. 56(2), pages 938-956, June.
    • Handle: RePEc:bla:jconsa:v:56:y:2022:i:2:p:938-956
    DOI: 10.1111/joca.12417
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/joca.12417
    Download Restriction: no
    File URL: https://libkey.io/10.1111/joca.12417?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Elizabeth Ooi, 2020. "Give mind to the gap: Measuring gender differences in financial knowledge," Journal of Consumer Affairs, Wiley Blackwell, vol. 54(3), pages 931-950, September.
    2. van Rooij, Maarten & Lusardi, Annamaria & Alessie, Rob, 2011. "Financial literacy and stock market participation," Journal of Financial Economics, Elsevier, vol. 101(2), pages 449-472, August.
    3. Dee Warmath & David Zimmerman, 2019. "Financial Literacy as More than Knowledge: The Development of a Formative Scale through the Lens of Bloom's Domains of Knowledge," Journal of Consumer Affairs, Wiley Blackwell, vol. 53(4), pages 1602-1629, December.
    4. Florian Deuflhard & Dimitris Georgarakos & Roman Inderst, 2019. "Financial Literacy and Savings Account Returns," Journal of the European Economic Association, European Economic Association, vol. 17(1), pages 131-164.
    5. Sam Allgood & William B. Walstad, 2016. "The Effects Of Perceived And Actual Financial Literacy On Financial Behaviors," Economic Inquiry, Western Economic Association International, vol. 54(1), pages 675-697, January.
    6. Daniel Fernandes & John G. Lynch & Richard G. Netemeyer, 2014. "Financial Literacy, Financial Education, and Downstream Financial Behaviors," Management Science, INFORMS, vol. 60(8), pages 1861-1883, August.
    7. Milo Bianchi, 2018. "Financial Literacy and Portfolio Dynamics," Journal of Finance, American Finance Association, vol. 73(2), pages 831-859, April.
    8. G. Kuder & M. Richardson, 1937. "The theory of the estimation of test reliability," Psychometrika, Springer;The Psychometric Society, vol. 2(3), pages 151-160, September.
    9. Brown, Martin & Henchoz, Caroline & Spycher, Thomas, 2018. "Culture and financial literacy: Evidence from a within-country language border," Journal of Economic Behavior & Organization, Elsevier, vol. 150(C), pages 62-85.
    10. Almenberg, Johan & Dreber, Anna, 2015. "Gender, stock market participation and financial literacy," Economics Letters, Elsevier, vol. 137(C), pages 140-142.
    11. William B. Walstad & Ken Rebeck, 2017. "The : Development and measurement characteristics," The Journal of Economic Education, Taylor & Francis Journals, vol. 48(2), pages 113-122, April.
    12. Justine S. Hastings & Brigitte C. Madrian & William L. Skimmyhorn, 2013. "Financial Literacy, Financial Education, and Economic Outcomes," Annual Review of Economics, Annual Reviews, vol. 5(1), pages 347-373, May.
    13. Carrie R. Houts & Melissa A. Z. Knoll, 2020. "The Financial Knowledge Scale: New Analyses, Findings, and Development of a Short Form," Journal of Consumer Affairs, Wiley Blackwell, vol. 54(2), pages 775-800, June.
    14. Yoshihiko Kadoya & Mostafa Saidur Rahim Khan, 2018. "Can financial literacy reduce anxiety about life in old age?," Journal of Risk Research, Taylor & Francis Journals, vol. 21(12), pages 1533-1550, December.
    15. Chen, Haiyang & Volpe, Ronald P., 1998. "An Analysis of Personal Financial Literacy Among College Students," Financial Services Review, Elsevier, vol. 7(2), pages 107-128.
    16. Lee Cronbach, 1951. "Coefficient alpha and the internal structure of tests," Psychometrika, Springer;The Psychometric Society, vol. 16(3), pages 297-334, September.
    17. Maarten C.J. van Rooij & Annamaria Lusardi & Rob J.M. Alessie, 2012. "Financial Literacy, Retirement Planning and Household Wealth," Economic Journal, Royal Economic Society, vol. 122(560), pages 449-478, May.
    18. Cote, Joseph A & Buckley, M Ronald, 1988. "Measurement Error and Theory Testing in Consumer Research: An Illustration of the Importance of Construct Validation," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 14(4), pages 579-582, March.
    19. Gopi Shah Goda & Matthew Levy & Colleen Flaherty Manchester & Aaron Sojourner & Joshua Tasoff, 2019. "Predicting Retirement Savings Using Survey Measures Of Exponential‐Growth Bias And Present Bias," Economic Inquiry, Western Economic Association International, vol. 57(3), pages 1636-1658, July.
    20. Susanne M. Schennach, 2016. "Recent Advances in the Measurement Error Literature," Annual Review of Economics, Annual Reviews, vol. 8(1), pages 341-377, October.
    21. Lusardi, Annamaria & Mitchell, Olivia S. & Curto, Vilsa, 2014. "Financial literacy and financial sophistication in the older population," Journal of Pension Economics and Finance, Cambridge University Press, vol. 13(4), pages 347-366, October.
    22. Davidoff, Thomas & Gerhard, Patrick & Post, Thomas, 2017. "Reverse mortgages: What homeowners (don’t) know and how it matters," Journal of Economic Behavior & Organization, Elsevier, vol. 133(C), pages 151-171.
    23. Irina Kunovskaya & Brenda Cude & Natalia Alexeev, 2014. "Evaluation of a Financial Literacy Test Using Classical Test Theory and Item Response Theory," Journal of Family and Economic Issues, Springer, vol. 35(4), pages 516-531, December.
    24. Albert Hizgilov & Jacques Silber, 2020. "On Multidimensional Approaches to Financial Literacy Measurement," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 148(3), pages 787-830, April.
    25. Rob Ranyard & Simon McNair & Gianni Nicolini & Darren Duxbury, 2020. "An item response theory approach to constructing and evaluating brief and in‐depth financial literacy scales," Journal of Consumer Affairs, Wiley Blackwell, vol. 54(3), pages 1121-1156, September.
    26. Melissa A. Z. Knoll & Carrie R. Houts, 2012. "The Financial Knowledge Scale: An Application of Item Response Theory to the Assessment of Financial Literacy," Journal of Consumer Affairs, Wiley Blackwell, vol. 46(3), pages 381-410, September.
    27. Mehmet Celiktas & Neslihan Yilmaz, 2021. "Money illusion, financial literacy and implications of self-perceptions," Applied Economics Letters, Taylor & Francis Journals, vol. 28(6), pages 447-450, March.
    28. Viswesvaran, Chockalingam & Ones, Deniz S. & Schmidt, Frank L. & Le, Huy & Oh, In-Sue, 2014. "Measurement Error Obfuscates Scientific Knowledge: Path to Cumulative Knowledge Requires Corrections for Unreliability and Psychometric Meta-Analyses," Industrial and Organizational Psychology, Cambridge University Press, vol. 7(4), pages 507-518, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Weiss-Cohen, Leonardo & Newall, Philip Warren Stirling & Ranyard, Rob & Ayton, Peter & Clacher, Iain, 2023. "Revalidating Fernandes et al.’s 2014 financial literacy scale in response to ongoing legislative and behavioral changes," OSF Preprints 493x7, Center for Open Science.
    2. Callis, Zoe & Gerrans, Paul & Walker, Dana L. & Gignac, Gilles E., 2023. "The association between intelligence and financial literacy: A conceptual and meta-analytic review," Intelligence, Elsevier, vol. 100(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Azra Zaimovic & Anes Torlakovic & Almira Arnaut-Berilo & Tarik Zaimovic & Lejla Dedovic & Minela Nuhic Meskovic, 2023. "Mapping Financial Literacy: A Systematic Literature Review of Determinants and Recent Trends," Sustainability, MDPI, vol. 15(12), pages 1-30, June.
    2. Oscar A. Stolper & Andreas Walter, 2017. "Financial literacy, financial advice, and financial behavior," Journal of Business Economics, Springer, vol. 87(5), pages 581-643, July.
    3. J. C. Hauff & A. Carlander & T. Gärling & G. Nicolini, 2020. "Retirement Financial Behaviour: How Important Is Being Financially Literate?," Journal of Consumer Policy, Springer, vol. 43(3), pages 543-564, September.
    4. Marie-Hélène BROIHANNE, 2021. "Testing the gender gap in subjective financial literacy of spouses," Working Papers of LaRGE Research Center 2021-08, Laboratoire de Recherche en Gestion et Economie (LaRGE), Université de Strasbourg.
    5. Hanson, Thomas A. & Olson, Peter M., 2018. "Financial literacy and family communication patterns," Journal of Behavioral and Experimental Finance, Elsevier, vol. 19(C), pages 64-71.
    6. Callis, Zoe & Gerrans, Paul & Walker, Dana L. & Gignac, Gilles E., 2023. "The association between intelligence and financial literacy: A conceptual and meta-analytic review," Intelligence, Elsevier, vol. 100(C).
    7. Bannier, Christina E. & Schwarz, Milena, 2018. "Gender- and education-related effects of financial literacy and confidence on financial wealth," Journal of Economic Psychology, Elsevier, vol. 67(C), pages 66-86.
    8. Francisco J. Oliver-Márquez & Almudena Guarnido-Rueda & Ignacio Amate-Fortes, 2021. "Measuring financial knowledge: a macroeconomic perspective," International Economics and Economic Policy, Springer, vol. 18(1), pages 177-222, February.
    9. Kaiser, Tim & Menkhoff, Lukas, 2018. "Active Learning Improves Financial Education:," Rationality and Competition Discussion Paper Series 131, CRC TRR 190 Rationality and Competition.
    10. Noemi Oggero & Maria Cristina Rossi & Elisa Ughetto, 2020. "Entrepreneurial spirits in women and men. The role of financial literacy and digital skills," Small Business Economics, Springer, vol. 55(2), pages 313-327, August.
    11. Tabea Bucher-Koenen & Annamaria Lusardi & Rob Alessie & Maarten van Rooij, 2017. "How Financially Literate Are Women? An Overview and New Insights," Journal of Consumer Affairs, Wiley Blackwell, vol. 51(2), pages 255-283, July.
    12. Gerrans, Paul, 2021. "Undergraduate student financial education interventions: Medium term evidence of retention, decay, and confidence in financial literacy," Pacific-Basin Finance Journal, Elsevier, vol. 67(C).
    13. Kawamura, Tetsuya & Mori, Tomoharu & Motonishi, Taizo & Ogawa, Kazuhito, 2021. "Is Financial Literacy Dangerous? Financial Literacy, Behavioral Factors, and Financial Choices of Households," Journal of the Japanese and International Economies, Elsevier, vol. 60(C).
    14. Isidore, Renu & Arun, C. Joe, 2023. "The Moderating Effect of Financial Literacy on the Relationship Between Decision-Making Tools and Equity Returns in the Indian Secondary Equity Market," Revista Finanzas y Politica Economica, Universidad Católica de Colombia, vol. 15(1), pages 185-211, January.
    15. Susanna Levantesi & Giulia Zacchia, 2021. "Machine Learning and Financial Literacy: An Exploration of Factors Influencing Financial Knowledge in Italy," JRFM, MDPI, vol. 14(3), pages 1-21, March.
    16. Tsun‐Feng Chiang, 2021. "Financial capability and investment management of Chinese households: An application of hybrid item response theory," Journal of Consumer Affairs, Wiley Blackwell, vol. 55(4), pages 1442-1463, December.
    17. Zhou, Yang & Yang, Manfang & Gan, Xu, 2023. "Education and financial literacy: Evidence from compulsory schooling law in China," The Quarterly Review of Economics and Finance, Elsevier, vol. 89(C), pages 335-346.
    18. Elizabeth Ooi, 2020. "Give mind to the gap: Measuring gender differences in financial knowledge," Journal of Consumer Affairs, Wiley Blackwell, vol. 54(3), pages 931-950, September.
    19. Tiina Koskelainen & Panu Kalmi & Eusebio Scornavacca & Tero Vartiainen, 2023. "Financial literacy in the digital age—A research agenda," Journal of Consumer Affairs, Wiley Blackwell, vol. 57(1), pages 507-528, January.
    20. Marc Oliver Rieger, 2020. "How to Measure Financial Literacy?," JRFM, MDPI, vol. 13(12), pages 1-14, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jconsa:v:56:y:2022:i:2:p:938-956. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0022-0078 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.