IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0271191.html

How representative are student convenience samples? A study of literacy and numeracy skills in 32 countries

Author

Listed:
  • Heather Wild
  • Aki-Juhani Kyröläinen
  • Victor Kuperman

Abstract

Psychological research, including research into adult reading, is frequently based on convenience samples of undergraduate students. This practice raises concerns about the external validity of many accepted findings. The present study seeks to determine how strong this student sampling bias is in literacy and numeracy research. We use the nationally representative cross-national data from the Programme for the International Assessment of Adult Competencies to quantify skill differences between (i) students and the general population aged 16–65, and (ii) students and age-matched non-students aged 16–25. The median effect size for the comparison (i) of literacy scores across 32 countries was d = .56, and for comparison (ii) d = .55, which exceeds the average effect size in psychological experiments (d = .40). Numeracy comparisons (i) and (ii) showed similarly strong differences. The observed differences indicate that undergraduate students are not representative of the general population nor age-matched non-students.

Suggested Citation

  • Heather Wild & Aki-Juhani Kyröläinen & Victor Kuperman, 2022. "How representative are student convenience samples? A study of literacy and numeracy skills in 32 countries," PLOS ONE, Public Library of Science, vol. 17(7), pages 1-22, July.
    • Handle: RePEc:plo:pone00:0271191
    DOI: 10.1371/journal.pone.0271191
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0271191
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0271191&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0271191?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Richard Desjardins & Arne Jonas Warnke, 2012. "Ageing and Skills: A Review and Analysis of Skill Gain and Skill Loss Over the Lifespan and Over Time," OECD Education Working Papers 72, OECD Publishing.
    2. Anke Grotlüschen & David Mallows & Stephen Reder & John Sabatini, 2016. "Adults with Low Proficiency in Literacy or Numeracy," OECD Education Working Papers 131, OECD Publishing.
    3. Saima Ghazal & Edward T. Cokely & Rocio Garcia-Retamero, 2014. "Predicting biases in very highly educated samples: Numeracy and metacognition," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 9(1), pages 15-34, January.
    4. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    5. Barrett, Garry F. & Riddell, W. Craig, 2019. "Ageing and Skills: The Case of Literacy Skills," IZA Discussion Papers 12073, IZA Network @ LISER.
    6. Green, David A. & Riddell, W. Craig, 2013. "Ageing and literacy skills: Evidence from Canada, Norway and the United States," Labour Economics, Elsevier, vol. 22(C), pages 16-29.
    7. Erik Snowberg & Leeat Yariv, 2021. "Testing the Waters: Behavior across Participant Pools," American Economic Review, American Economic Association, vol. 111(2), pages 687-719, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Claudia Reiter, 2022. "Changes in Literacy Skills as Cohorts Age," Population and Development Review, The Population Council, Inc., vol. 48(1), pages 217-246, March.
    2. Barrett, Garry F. & Riddell, W. Craig, 2019. "Ageing and Skills: The Case of Literacy Skills," IZA Discussion Papers 12073, IZA Network @ LISER.
    3. Ray Saadaoui Mallek & Mohamed Albaity, 2019. "Individual differences and cognitive reflection across gender and nationality the case of the United Arab Emirates," Cogent Economics & Finance, Taylor & Francis Journals, vol. 7(1), pages 1567965-156, January.
    4. André Mata & Tiago Almeida, 2014. "Using metacognitive cues to infer others' thinking," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 9(4), pages 349-359, July.
    5. Samuel Vézina & Alain Bélanger, 2019. "Impacts of education and immigration on the size and skills of the future workforce," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 41(12), pages 331-366.
    6. Jelnov, Pavel & Weiss, Yoram, 2022. "Influence in economics and aging," Labour Economics, Elsevier, vol. 77(C).
    7. Thiemann, Petra & Schulz, Jonathan & Sunde, Uwe & Thöni, Christian, 2022. "Selection into experiments: New evidence on the role of preferences, cognition, and recruitment protocols," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
    8. Johannes G. Jaspersen & Marc A. Ragin & Justin R. Sydnor, 2022. "Insurance demand experiments: Comparing crowdworking to the lab," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 89(4), pages 1077-1107, December.
    9. Skagerlund, Kenny & Lind, Thérèse & Strömbäck, Camilla & Tinghög, Gustav & Västfjäll, Daniel, 2018. "Financial literacy and the role of numeracy–How individuals’ attitude and affinity with numbers influence financial literacy," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 74(C), pages 18-25.
    10. Karsten Albæk, 2015. "A test of the ‘lose it or use it’ hypothesis in labour markets around the world," Working Papers 2015/24, Institut d'Economia de Barcelona (IEB).
    11. Sven Grüner & Oliver Mußhoff, 2025. "Behavioral climate change: does thinking about future consequences of climate change affect risk preferences and cooperation?," Climatic Change, Springer, vol. 178(7), pages 1-21, July.
    12. Audra Bowlus & Lance Lochner & Chris Robinson & Eda Suleymanoglu, 2023. "Wages, Skills, and Skill-Biased Technical Change: The Canonical Model Revisited," Journal of Human Resources, University of Wisconsin Press, vol. 58(6), pages 1783-1819.
    13. Sobkow, Agata & Olszewska, Angelika & Traczyk, Jakub, 2020. "Multiple numeric competencies predict decision outcomes beyond fluid intelligence and cognitive reflection," Intelligence, Elsevier, vol. 80(C).
    14. Nobuyuki Hanaki & Takahiro Hoshino & Kohei Kubota & Fabrice Murtin & Masao Ogaki & Fumio Ohtake & Naoko Okuyama, 2022. "Comparing data gathered in an online and a laboratory experiment using the Trustlab platform," ISER Discussion Paper 1168r, Institute of Social and Economic Research, The University of Osaka, revised Jun 2022.
    15. Ronayne, David & Sgroi, Daniel & Tuckwell, Anthony, 2021. "Evaluating the sunk cost effect," Journal of Economic Behavior & Organization, Elsevier, vol. 186(C), pages 318-327.
    16. Muñoz-Murillo, Melisa & Álvarez-Franco, Pilar B. & Restrepo-Tobón, Diego A., 2020. "The role of cognitive abilities on financial literacy: New experimental evidence," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 84(C).
    17. Jorge Calero & Inés P. Murillo Huertas & Josep Lluís Raymond Bara, 2016. "Education, age and skills: an analysis using the PIAAC survey," Working Papers 2016/3, Institut d'Economia de Barcelona (IEB).
    18. Nobuyuki Hanaki & Keigo Inukai & Takehito Masuda & Yuta Shimodaira, 2022. "Comparing behavior between a large sample of smart students and a representative sample of Japanese adults," ISER Discussion Paper 1160, Institute of Social and Economic Research, The University of Osaka.
    19. Siddiqi, Umema, 2023. "The Intersection of Financial Literacy, Cognitive Ability, and Numeracy Skills in Pakistani Adults," MPRA Paper 119781, University Library of Munich, Germany.
    20. Gruener, Sven & Mußhoff, Oliver, 2025. "Behavioral climate change: Does thinking about future consequences of climate change affect risk preferences and cooperation?," OSF Preprints 7vdu6_v1, Center for Open Science.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0271191. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.