IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v42y2017i1p3-45.html
   My bibliography  Save this article

Using Heteroskedastic Ordered Probit Models to Recover Moments of Continuous Test Score Distributions From Coarsened Data

Author

Listed:
  • Sean F. Reardon
  • Benjamin R. Shear

    (Stanford University)

  • Katherine E. Castellano

    (Educational Testing Service)

  • Andrew D. Ho

    (Harvard Graduate School of Education)

Abstract

Test score distributions of schools or demographic groups are often summarized by frequencies of students scoring in a small number of ordered proficiency categories. We show that heteroskedastic ordered probit (HETOP) models can be used to estimate means and standard deviations of multiple groups’ test score distributions from such data. Because the scale of HETOP estimates is indeterminate up to a linear transformation, we develop formulas for converting the HETOP parameter estimates and their standard errors to a scale in which the population distribution of scores is standardized. We demonstrate and evaluate this novel application of the HETOP model with a simulation study and using real test score data from two sources. We find that the HETOP model produces unbiased estimates of group means and standard deviations, except when group sample sizes are small. In such cases, we demonstrate that a “partially heteroskedastic†ordered probit (PHOP) model can produce estimates with a smaller root mean squared error than the fully heteroskedastic model.

Suggested Citation

  • Sean F. Reardon & Benjamin R. Shear & Katherine E. Castellano & Andrew D. Ho, 2017. "Using Heteroskedastic Ordered Probit Models to Recover Moments of Continuous Test Score Distributions From Coarsened Data," Journal of Educational and Behavioral Statistics, , vol. 42(1), pages 3-45, February.
  • Handle: RePEc:sae:jedbes:v:42:y:2017:i:1:p:3-45
    DOI: 10.3102/1076998616666279
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998616666279
    Download Restriction: no

    File URL: https://libkey.io/10.3102/1076998616666279?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Yuanyuan Gu & Denzil G. Fiebig & Edward Cripps & Robert Kohn, 2009. "Bayesian estimation of a random effects heteroscedastic probit model," Econometrics Journal, Royal Economic Society, vol. 12(2), pages 324-339, July.
    2. Anna N. Angelos Tosteson & Colin B. Begg, 1988. "A General Regression Methodology for ROC Curve Estimation," Medical Decision Making, , vol. 8(3), pages 204-215, August.
    3. Joel L. Horowitz & Jürg M. Sparmann & Carlos F. Daganzo, 1982. "An Investigation of the Accuracy of the Clark Approximation for the Multinomial Probit Model," Transportation Science, INFORMS, vol. 16(3), pages 382-401, August.
    4. Richard Williams, 2010. "Fitting heterogeneous choice models with oglm," Stata Journal, StataCorp LP, vol. 10(4), pages 540-567, December.
    5. Alexander Shapiro & Jos Berge, 2002. "Statistical inference of minimum rank factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 67(1), pages 79-94, March.
    6. Greene,William H. & Hensher,David A., 2010. "Modeling Ordered Choices," Cambridge Books, Cambridge University Press, number 9780521142373, October.
    7. Keane, Michael P, 1992. "A Note on Identification in the Multinomial Probit Model," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(2), pages 193-200, April.
    8. Greene,William H. & Hensher,David A., 2010. "Modeling Ordered Choices," Cambridge Books, Cambridge University Press, number 9780521194204, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Charlie Tchinda & Marcus Dejardin, 2021. "Are Business Policy Measures in Response to the COVID-19 Pandemic to Be Equally Valued? An Exploration According to SMEs Owners’ Business Expectations," Sustainability, MDPI, vol. 13(21), pages 1-42, October.
    2. Chen, Le-Yu & Oparina, Ekaterina & Powdthavee, Nattavudh & Srisuma, Sorawoot, 2022. "Robust Ranking of Happiness Outcomes: A Median Regression Perspective," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 672-686.
    3. Andrew S. Fullerton & Jun Xu, 2018. "Constrained and Unconstrained Partial Adjacent Category Logit Models for Ordinal Response Variables," Sociological Methods & Research, , vol. 47(2), pages 169-206, March.
    4. Juan Carlos Martín & Concepción Román, 2021. "COVID-19 Is Examining the EU and the Member States: The Role of Attitudes and Sociodemographic Factors on Citizens’ Support towards National Policies," Social Sciences, MDPI, vol. 10(2), pages 1-27, January.
    5. Omar Paccagnella, 2011. "Anchoring vignettes with sample selection due to non‐response," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 665-687, July.
    6. Chen, Le-Yu & Oparina, Ekaterina & Powdthavee, Nattavudh & Srisuma, Sorawoot, 2019. "Have Econometric Analyses of Happiness Data Been Futile? A Simple Truth about Happiness Scales," IZA Discussion Papers 12152, Institute of Labor Economics (IZA).
    7. William H. Greene & Mark N. Harris & Rachel J. Knott & Nigel Rice, 2021. "Specification and testing of hierarchical ordered response models with anchoring vignettes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 31-64, January.
    8. Hanna Dudek & Joanna Landmesser, 2012. "Income satisfaction and relative deprivation," Statistics in Transition new series, Główny Urząd Statystyczny (Polska), vol. 13(2), pages 321-334, June.
    9. Andrew Powell & Pilar Tavella, 2012. "Capital Inflow Surges in Emerging Economies: How Worried Should LAC Be?," Research Department Publications 4782, Inter-American Development Bank, Research Department.
    10. Chappell, Henry W. & McGregor, Rob Roy, 2018. "Committee decision-making at Sweden's Riksbank," European Journal of Political Economy, Elsevier, vol. 53(C), pages 120-133.
    11. Patricia Cubí‐Mollá & Mireia Jofre‐Bonet & Victoria Serra‐Sastre, 2017. "Adaptation to health states: Sick yet better off?," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1826-1843, December.
    12. Ahmad Adeel & Bruno Notteboom & Ansar Yasar & Kris Scheerlinck & Jeroen Stevens, 2021. "Sustainable Streetscape and Built Environment Designs around BRT Stations: A Stated Choice Experiment Using 3D Visualizations," Sustainability, MDPI, vol. 13(12), pages 1-21, June.
    13. Jeffrey A. Edwards & Tara R. Wade & Mark L. Burkey & R. Gary Pumphrey, 2014. "Forecasting the Public's Acceptability of Municipal Water Regulation and Price Rationing for Communities on the Ogallala Aquifer," Journal of Economic Insight, Missouri Valley Economic Association, vol. 40(1), pages 1-30.
    14. Simona Iammarino & Elisabetta Marinelli & Elisabetta Marinelli, 2011. "Is the Grass Greener on the other Side of the Fence? Graduate Mobility and Job Satisfaction in Italy," Environment and Planning A, , vol. 43(11), pages 2761-2777, November.
    15. Ermagun, Alireza & Stathopoulos, Amanda, 2018. "To bid or not to bid: An empirical study of the supply determinants of crowd-shipping," Transportation Research Part A: Policy and Practice, Elsevier, vol. 116(C), pages 468-483.
    16. Mohit Batham & Soudeh Mirghasemi & Mohammad Arshad Rahman & Manini Ojha, 2021. "Modeling and Analysis of Discrete Response Data: Applications to Public Opinion on Marijuana Legalization in the United States," Papers 2109.10122, arXiv.org, revised May 2023.
    17. Mahdi Rezapour & F. Richard Ferraro, 2021. "The impact of commuters’ psychological feelings due to delay on perceived quality of a rail transport," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-8, December.
    18. Migliardo, Carlo, 2012. "Heterogeneity in price setting behavior, spatial disparities and sectoral diversity: Evidence from a panel of Italian firms," Economic Modelling, Elsevier, vol. 29(4), pages 1106-1118.
    19. Jose Apesteguia & Miguel A. Ballester, 2023. "The rationalizability of survey responses," Economics Working Papers 1863, Department of Economics and Business, Universitat Pompeu Fabra.
    20. Evans, Keith S. & Teisl, Mario F. & Lando, Amy. M. & Liu, Sherry T., 2020. "Risk perceptions and food-handling practices in the home," Food Policy, Elsevier, vol. 95(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:42:y:2017:i:1:p:3-45. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.