IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp15252.html
   My bibliography  Save this paper

Measuring Knowledge

Author

Listed:
  • Heckman, James J.

    (University of Chicago)

  • Zhou, Jin

    (University of Chicago)

Abstract

Empirical studies in the economics of education, the measurement of skill gaps across demographic groups, and the impacts of interventions on skill formation rely on psychometrically validated test scores that record the proportion of items correctly answered. Test scores are sometimes taken as measures of an invariant scale of human capital that can be compared over time and people. We show that for a prototypical test, invariance is violated. We use an unusually rich data set from an early childhood intervention program that measures knowledge of narrowly defined skills on essentially equivalent subsets of tasks. We examine if conventional, broadly-defined measures of skill are the same across people who are comparable on detailed knowledge measures. We reject the hypothesis of aggregate scale invariance and call into question the uncritical use of test scores in research on education and on skill formation. We compare different measures of skill and ability and reject the hypothesis of valid aggregate measures of skill.

Suggested Citation

  • Heckman, James J. & Zhou, Jin, 2022. "Measuring Knowledge," IZA Discussion Papers 15252, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp15252
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp15252.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Steven G. Rivkin & Eric A. Hanushek & John F. Kain, 2005. "Teachers, Schools, and Academic Achievement," Econometrica, Econometric Society, vol. 73(2), pages 417-458, March.
    2. Petra E. Todd & Kenneth I. Wolpin, 2007. "The Production of Cognitive Achievement in Children: Home, School, and Racial Test Score Gaps," Journal of Human Capital, University of Chicago Press, vol. 1(1), pages 91-136.
    3. Flavio Cunha & James J. Heckman & Susanne M. Schennach, 2010. "Estimating the Technology of Cognitive and Noncognitive Skill Formation," Econometrica, Econometric Society, vol. 78(3), pages 883-931, May.
    4. Flavio Cunha & Eric Nielsen & Benjamin Williams, 2021. "The Econometrics of Early Childhood Human Capital and Investments," Annual Review of Economics, Annual Reviews, vol. 13(1), pages 487-513, August.
    5. Heather C. Hill, 2009. "Evaluating value-added models: A validity argument approach," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 28(4), pages 700-709.
    6. Francesco Agostinelli & Morteza Saharkhiz & Matthew Wiswall, 2019. "Home and School in the Development of Children," Working Papers 2019-044, Human Capital and Economic Opportunity Working Group.
    7. Marta Rubio-Codina & María Caridad Araujo & Orazio P. Attanasio & Sally Grantham-McGregor, 2016. "Concurrent Validity and Feasibility of Short Tests Currently Used to Measure Early Childhood Development in Large Scale Studies: Methodology and Results," IDB Publications (Working Papers) 95556, Inter-American Development Bank.
    8. Flavio Cunha & James J. Heckman, 2008. "Formulating, Identifying and Estimating the Technology of Cognitive and Noncognitive Skill Formation," Journal of Human Resources, University of Wisconsin Press, vol. 43(4).
    9. Francesco Agostinelli & Morteza Saharkhiz & Matthew J. Wiswall, 2019. "Home and School in the Development of Children," NBER Working Papers 26037, National Bureau of Economic Research, Inc.
    10. John Cawley & James Heckman & Edward Vytlacil, 1999. "On Policies To Reward The Value Added By Educators," The Review of Economics and Statistics, MIT Press, vol. 81(4), pages 720-727, November.
    11. Rubio-Codina, Marta & Araujo, María Caridad & Attanasio, Orazio P. & Grantham-McGregor, Sally, 2016. "Concurrent Validity and Feasibility of Short Tests Currently Used to Measure Early Childhood Development in Large Scale Studies: Methodology and Results," IDB Publications (Working Papers) 7823, Inter-American Development Bank.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. James J. Heckman & Jin Zhou, 2022. "Measuring Knowledge and Learning," NBER Working Papers 29990, National Bureau of Economic Research, Inc.
    2. Flavio Cunha & James J. Heckman & Susanne M. Schennach, 2010. "Estimating the Technology of Cognitive and Noncognitive Skill Formation," Econometrica, Econometric Society, vol. 78(3), pages 883-931, May.
    3. Daniela Del Boca & Enrica Maria Martino & Chiara Pronzato, 2022. "Non cognitive skills and childcare attendance," Review of Economics of the Household, Springer, vol. 20(4), pages 1059-1085, December.
    4. Borra, Cristina & Iacovou, Maria & Sevilla, Almudena, 2023. "Adolescent development and the math gender gap," European Economic Review, Elsevier, vol. 158(C).
    5. Hofmeyr, Heleen, 2021. "Perseverance, passion and poverty: Examining the association between grit and reading achievement in high-poverty schools in South Africa," International Journal of Educational Development, Elsevier, vol. 83(C).
    6. Elizabeth M. Caucutt & Lance Lochner & Youngmin Park, 2017. "Correlation, Consumption, Confusion, or Constraints: Why Do Poor Children Perform so Poorly?," Scandinavian Journal of Economics, Wiley Blackwell, vol. 119(1), pages 102-147, January.
    7. Marta Favara & Catherine Porter & Tassew Woldehanna, 2019. "Smarter through social protection? Evaluating the impact of Ethiopia’s safety-net on child cognitive abilities," Oxford Development Studies, Taylor & Francis Journals, vol. 47(1), pages 79-96, January.
    8. Flèche, Sarah & Lekfuangfu, Warn N. & Clark, Andrew E., 2021. "The long-lasting effects of family and childhood on adult wellbeing: Evidence from British cohort data," Journal of Economic Behavior & Organization, Elsevier, vol. 181(C), pages 290-311.
    9. Ronni Pavan, 2016. "On the Production of Skills and the Birth-Order Effect," Journal of Human Resources, University of Wisconsin Press, vol. 51(3), pages 699-726.
    10. Heleen Hofmeyr, 2020. "Perseverance, Passion, and Poverty: Examining the association between grit and reading achievement in high-poverty schools," Working Papers 06/2020, Stellenbosch University, Department of Economics.
    11. Deborah Cobb-Clark & Tue Gørgens, 2014. "Parents’ economic support of young-adult children: do socioeconomic circumstances matter?," Journal of Population Economics, Springer;European Society for Population Economics, vol. 27(2), pages 447-471, April.
    12. Haelermans, Carla & Ghysels, Joris, 2019. "Effectively involving low-SES parents in human capital development," ROA Research Memorandum 008, Maastricht University, Research Centre for Education and the Labour Market (ROA).
    13. Nguyen, Ha Trong & Brinkman, Sally & Le, Huong Thu & Zubrick, Stephen R. & Mitrou, Francis, 2022. "Gender differences in time allocation contribute to differences in developmental outcomes in children and adolescents," Economics of Education Review, Elsevier, vol. 89(C).
    14. Elizabeth M. Caucutt & Lance Lochner, 2020. "Early and Late Human Capital Investments, Borrowing Constraints, and the Family," Journal of Political Economy, University of Chicago Press, vol. 128(3), pages 1065-1147.
    15. Bernal, Pedro & Mittag, Nikolas & Qureshi, Javaeria A., 2016. "Estimating effects of school quality using multiple proxies," Labour Economics, Elsevier, vol. 39(C), pages 1-10.
    16. Meghir, Costas & Rivkin, Steven, 2011. "Econometric Methods for Research in Education," Handbook of the Economics of Education, in: Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.), Handbook of the Economics of Education, edition 1, volume 3, chapter 1, pages 1-87, Elsevier.
    17. Huong Thu Le & Ha Trong Nguyen, 2017. "Parental health and children's cognitive and noncognitive development: New evidence from the longitudinal survey of Australian children," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1767-1788, December.
    18. Jonathan Norris, 2019. "Identify economics: social influence and skill development," Working Papers 1908, University of Strathclyde Business School, Department of Economics.
    19. Francesconi, Marco & Heckman, James J, 2015. "Symposium on Child Development and Parental Investment: Introduction," Economics Discussion Papers 16868, University of Essex, Department of Economics.
    20. Borghans, L. & Golsteyn, B.H.H. & Heckman, James & Humphries, John Eric, 2011. "Identification problems in personality psychology," Research Memorandum 025, Maastricht University, Maastricht Research School of Economics of Technology and Organization (METEOR).

    More about this item

    Keywords

    testing child development; psychometrics; measurement of discrimination; human capital; demographic economics;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • J71 - Labor and Demographic Economics - - Labor Discrimination - - - Hiring and Firing

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp15252. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.