IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0271191.html
   My bibliography  Save this article

How representative are student convenience samples? A study of literacy and numeracy skills in 32 countries

Author

Listed:
  • Heather Wild
  • Aki-Juhani Kyröläinen
  • Victor Kuperman

Abstract

Psychological research, including research into adult reading, is frequently based on convenience samples of undergraduate students. This practice raises concerns about the external validity of many accepted findings. The present study seeks to determine how strong this student sampling bias is in literacy and numeracy research. We use the nationally representative cross-national data from the Programme for the International Assessment of Adult Competencies to quantify skill differences between (i) students and the general population aged 16–65, and (ii) students and age-matched non-students aged 16–25. The median effect size for the comparison (i) of literacy scores across 32 countries was d = .56, and for comparison (ii) d = .55, which exceeds the average effect size in psychological experiments (d = .40). Numeracy comparisons (i) and (ii) showed similarly strong differences. The observed differences indicate that undergraduate students are not representative of the general population nor age-matched non-students.

Suggested Citation

  • Heather Wild & Aki-Juhani Kyröläinen & Victor Kuperman, 2022. "How representative are student convenience samples? A study of literacy and numeracy skills in 32 countries," PLOS ONE, Public Library of Science, vol. 17(7), pages 1-22, July.
    • Handle: RePEc:plo:pone00:0271191
    DOI: 10.1371/journal.pone.0271191
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0271191
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0271191&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0271191?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Richard Desjardins & Arne Jonas Warnke, 2012. "Ageing and Skills: A Review and Analysis of Skill Gain and Skill Loss Over the Lifespan and Over Time," OECD Education Working Papers 72, OECD Publishing.
    2. Green, David A. & Riddell, W. Craig, 2013. "Ageing and literacy skills: Evidence from Canada, Norway and the United States," Labour Economics, Elsevier, vol. 22(C), pages 16-29.
    3. Anke Grotlüschen & David Mallows & Stephen Reder & John Sabatini, 2016. "Adults with Low Proficiency in Literacy or Numeracy," OECD Education Working Papers 131, OECD Publishing.
    4. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    5. Barrett, Garry F. & Riddell, W. Craig, 2019. "Ageing and Skills: The Case of Literacy Skills," IZA Discussion Papers 12073, Institute of Labor Economics (IZA).
    6. Erik Snowberg & Leeat Yariv, 2021. "Testing the Waters: Behavior across Participant Pools," American Economic Review, American Economic Association, vol. 111(2), pages 687-719, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Claudia Reiter, 2022. "Changes in Literacy Skills as Cohorts Age," Population and Development Review, The Population Council, Inc., vol. 48(1), pages 217-246, March.
    2. Barrett, Garry F. & Riddell, W. Craig, 2019. "Ageing and Skills: The Case of Literacy Skills," IZA Discussion Papers 12073, Institute of Labor Economics (IZA).
    3. Audra Bowlus & Lance Lochner & Chris Robinson & Eda Suleymanoglu, 2023. "Wages, Skills, and Skill-Biased Technical Change: The Canonical Model Revisited," Journal of Human Resources, University of Wisconsin Press, vol. 58(6), pages 1783-1819.
    4. Brice Corgnet & Mark DeSantis & Christoph Siemroth, 2023. "Algorithmic Trading, Price Efficiency and Welfare: An Experimental Approach," Working Papers 2313, Groupe d'Analyse et de Théorie Economique Lyon St-Étienne (GATE Lyon St-Étienne), Université de Lyon.
    5. Ronayne, David & Sgroi, Daniel & Tuckwell, Anthony, 2021. "Evaluating the sunk cost effect," Journal of Economic Behavior & Organization, Elsevier, vol. 186(C), pages 318-327.
    6. Nobuyuki Hanaki & Keigo Inukai & Takehito Masuda & Yuta Shimodaira, 2021. "Participants’ Characteristics at ISER-Lab in 2020," ISER Discussion Paper 1141, Institute of Social and Economic Research, The University of Osaka.
    7. Samuel Vézina & Alain Bélanger, 2019. "Impacts of education and immigration on the size and skills of the future workforce," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 41(12), pages 331-366.
    8. Pin, Paolo & Rotesi, Tiziano, 2023. "App-based experiments," Journal of Economic Psychology, Elsevier, vol. 99(C).
    9. Eric A. Hanushek & Lavinia Kinne & Frauke Witthoeft & Ludger Woessmann, 2024. "Age and Cognitive Skills: Use It or Lose It," Papers 2410.00790, arXiv.org.
    10. Jelnov, Pavel & Weiss, Yoram, 2022. "Influence in economics and aging," Labour Economics, Elsevier, vol. 77(C).
    11. Thiemann, Petra & Schulz, Jonathan & Sunde, Uwe & Thöni, Christian, 2022. "Selection into experiments: New evidence on the role of preferences, cognition, and recruitment protocols," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
    12. Johannes G. Jaspersen & Marc A. Ragin & Justin R. Sydnor, 2022. "Insurance demand experiments: Comparing crowdworking to the lab," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 89(4), pages 1077-1107, December.
    13. Te Bao & Brice Corgnet & Nobuyuki Hanaki & Katsuhiko Okada & Yohanes E. Riyanto & Jiahua Zhu, 2022. "Financial Forecasting in the Lab and the Field: Qualified Professionals vs. Smart Students," ISER Discussion Paper 1156r, Institute of Social and Economic Research, The University of Osaka, revised Sep 2024.
    14. Albæk, Karsten, 2016. "A Test of the ?Use it or Lose It? Hypothesis in Labour Markets around the World/Una prueba de la hipótesis "usarlo o perderlo" en los mercados de trabajo del mundo," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 34, pages 323-352, Mayo.
    15. Karsten Albæk, 2015. "A test of the ‘lose it or use it’ hypothesis in labour markets around the world," Working Papers 2015/24, Institut d'Economia de Barcelona (IEB).
    16. Flisi, Sara & Goglio, Valentina & Meroni, Elena Claudia & Vera-Toscano, Esperanza, 2019. "Cohort patterns in adult literacy skills: How are new generations doing?," Journal of Policy Modeling, Elsevier, vol. 41(1), pages 52-65.
    17. Piotr Evdokimov & Umberto Garfagnini, 2023. "Cognitive Ability and Perceived Disagreement in Learning," Rationality and Competition Discussion Paper Series 381, CRC TRR 190 Rationality and Competition.
    18. Nobuyuki Hanaki & Takahiro Hoshino & Kohei Kubota & Fabrice Murtin & Masao Ogaki & Fumio Ohtake & Naoko Okuyama, 2022. "Comparing data gathered in an online and a laboratory experiment using the Trustlab platform," ISER Discussion Paper 1168r, Institute of Social and Economic Research, The University of Osaka, revised Jun 2022.
    19. Jorge Calero & Inés P. Murillo Huertas & Josep Lluís Raymond Bara, 2016. "Education, age and skills: an analysis using the PIAAC survey," Working Papers 2016/3, Institut d'Economia de Barcelona (IEB).
    20. Nobuyuki Hanaki & Keigo Inukai & Takehito Masuda & Yuta Shimodaira, 2022. "Comparing behavior between a large sample of smart students and a representative sample of Japanese adults," ISER Discussion Paper 1160, Institute of Social and Economic Research, The University of Osaka.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0271191. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.