IDEAS home Printed from https://ideas.repec.org/p/cdl/cshedu/qt66n4x6g4.html
   My bibliography  Save this paper

Admissions Bias: A New Approach to Validity Estimation in Selected Samples

Author

Listed:
  • Rothstein, Jesse M

Abstract

Validity researchers typically work with nonrandom samples, membership in which depends in part on the exam score being investigated. In a study of the SAT's validity for freshman GPA at a particular college, for example, FGPA is not observed for the entire pool of potential applicants, but only for those who are admitted and enroll. This sample selection biases validity estimates. Corrections for restriction of range remedy the problem only when the exam score is the sole determinant of selection, and even then do not permit consistent estimation of the exam's incremental validity. Regression omitted variables results motivate a proposed validity estimator that is consistent whenever the determinants of selection are observed. An algorithm is suggested for calculation of these "robust" validity coefficients, and used to estimate the SAT's validity at the University of California. The usual validity estimates are shown to be substantially biased, in the hypothesized directions, by admissions-induced sample selection.

Suggested Citation

  • Rothstein, Jesse M, 2002. "Admissions Bias: A New Approach to Validity Estimation in Selected Samples," University of California at Berkeley, Center for Studies in Higher Education qt66n4x6g4, Center for Studies in Higher Education, UC Berkeley.
    • Handle: RePEc:cdl:cshedu:qt66n4x6g4
    as

    Download full text from publisher

    File URL: https://www.escholarship.org/uc/item/66n4x6g4.pdf;origin=repeccitec
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Rothstein, J.M.Jesse M., 2004. "College performance predictions and the SAT," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 297-317.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sandra E. Black & Jeffrey T. Denning & Jesse Rothstein, 2023. "Winners and Losers? The Effect of Gaining and Losing Access to Selective Colleges on Education and Labor Market Outcomes," American Economic Journal: Applied Economics, American Economic Association, vol. 15(1), pages 26-67, January.
    2. Guyonne Kalb & Sholeh A. Maani, 2007. "The Importance of Observing Early School Leaving and Usually Unobserved Background and Peer Characteristics in Analysing Academic Performance," Melbourne Institute Working Paper Series wp2007n05, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    3. Juan Esteban Saavedra & Carlos Medina, 2012. "Formación para el Trabajo en Colombia," Borradores de Economia 10059, Banco de la Republica.
    4. Joseph H. Paris & Catherine Pressimone Beckowski & Sara Fiorot, 2023. "Predicting Success: An Examination of the Predictive Validity of a Measure of Motivational-Developmental Dimensions in College Admissions," Research in Higher Education, Springer;Association for Institutional Research, vol. 64(8), pages 1191-1216, December.
    5. Sezgin Polat & Jean-Jacques Paul, 2016. "How to predict university performance: a case study from a prestigious Turkish university?," Investigaciones de Economía de la Educación volume 11, in: José Manuel Cordero Ferrera & Rosa Simancas Rodríguez (ed.), Investigaciones de Economía de la Educación 11, edition 1, volume 11, chapter 22, pages 423-434, Asociación de Economía de la Educación.
    6. Jesse Rothstein, 2019. "Inequality of Educational Opportunity? Schools as Mediators of the Intergenerational Transmission of Income," Journal of Labor Economics, University of Chicago Press, vol. 37(S1), pages 85-123.
    7. Mario I. Suárez & Alan R Dabney & Hersh C Waxman & Timothy P Scott & Adrienne O Bentz, 2021. "Exploring Factors that Predict STEM Persistence at a Large, Public Research University," International Journal of Higher Education, Sciedu Press, vol. 10(4), pages 161-161, August.
    8. Dur, Robert & Glazer, Amihai, 2008. "Subsidizing Enjoyable Education," Labour Economics, Elsevier, vol. 15(5), pages 1023-1039, October.
    9. Gandil, Mikkel Høst & Leuven, Edwin, 2022. "College Admission as a Screening and Sorting Device," IZA Discussion Papers 15557, Institute of Labor Economics (IZA).
    10. Raj Chetty & John N. Friedman & Emmanuel Saez & Nicholas Turner & Danny Yagan, 2020. "The Determinants of Income Segregation and Intergenerational Mobility: Using Test Scores to Measure Undermatching," NBER Working Papers 26748, National Bureau of Economic Research, Inc.
    11. Rothstein, Jesse, 2022. "Qualitative information in undergraduate admissions: A pilot study of letters of recommendation," Economics of Education Review, Elsevier, vol. 89(C).
    12. Eric P. Bettinger & Brent J. Evans & Devin G. Pope, 2013. "Improving College Performance and Retention the Easy Way: Unpacking the ACT Exam," American Economic Journal: Economic Policy, American Economic Association, vol. 5(2), pages 26-52, May.
    13. Bai, Chong-en & Chi, Wei & Qian, Xiaoye, 2014. "Do college entrance examination scores predict undergraduate GPAs? A tale of two universities," China Economic Review, Elsevier, vol. 30(C), pages 632-647.
    14. Judith Scott-Clayton & Peter M. Crosta & Clive R. Belfield, 2012. "Improving the Targeting of Treatment: Evidence from College Remediation," NBER Working Papers 18457, National Bureau of Economic Research, Inc.
    15. Вербецкий Алексей Дмитриевич & Фридман Алла Александровна, 2016. "Политика Приема В Вузы И Конкуренция Абитуриентов," Economic policy Экономическая политика, CyberLeninka;Автономная некоммерческая организация «Редакция журнала “Экономическая политика”», vol. 11(5), pages 68-91.
    16. Jensen, Elizabeth J. & Wu, Stephen, 2010. "Early decision and college performance," Economics of Education Review, Elsevier, vol. 29(4), pages 517-525, August.
    17. Jake Anders, 2014. "Does an aptitude test affect socioeconomic and gender gaps in attendance at an elite university?," DoQSS Working Papers 14-07, Quantitative Social Science - UCL Social Research Institute, University College London.
    18. Wu, Binzhen & Zhong, Xiaohan, 2014. "Matching mechanisms and matching quality: Evidence from a top university in China," Games and Economic Behavior, Elsevier, vol. 84(C), pages 196-215.
    19. Bergman, Peter & Kopko, Elizabeth & Rodriguez, Julio, 2021. "Using Predictive Analytics to Track Students: Evidence from a Seven-College Experiment," IZA Discussion Papers 14500, Institute of Labor Economics (IZA).
    20. Attali, Yigal & Neeman, Zvika & Schlosser, Analia, 2011. "Rise to the Challenge or Not Give a Damn: Differential Performance in High vs. Low Stakes Tests," IZA Discussion Papers 5693, Institute of Labor Economics (IZA).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cdl:cshedu:qt66n4x6g4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Lisa Schiff (email available below). General contact details of provider: https://escholarship.org/uc/cshe/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.