IDEAS home Printed from https://ideas.repec.org/a/taf/amstat/v69y2015i3p174-181.html
   My bibliography  Save this article

Discreteness Causes Bias in Percentage-Based Comparisons: A Case Study From Educational Testing

Author

Listed:
  • Darrick Yee
  • Andrew Ho

Abstract

Discretizing continuous distributions can lead to bias in parameter estimates. We present a case study from educational testing that illustrates dramatic consequences of discreteness when discretizing partitions differ across distributions. The percentage of test takers who score above a certain cutoff score (percent above cutoff, or "PAC") often describes overall performance on a test. Year-over-year changes in PAC, or ΔPAC, have gained prominence under recent U.S. education policies, with public schools facing sanctions if they fail to meet PAC targets. In this article, we describe how test score distributions act as continuous distributions that are discretized inconsistently over time. We show that this can propagate considerable bias to PAC trends, where positive ΔPACs appear negative, and vice versa, for a substantial number of actual tests. A simple model shows that this bias applies to any comparison of PAC statistics in which values for one distribution are discretized differently from values for the other.

Suggested Citation

  • Darrick Yee & Andrew Ho, 2015. "Discreteness Causes Bias in Percentage-Based Comparisons: A Case Study From Educational Testing," The American Statistician, Taylor & Francis Journals, vol. 69(3), pages 174-181, August.
    • Handle: RePEc:taf:amstat:v:69:y:2015:i:3:p:174-181
    DOI: 10.1080/00031305.2015.1031828
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/00031305.2015.1031828
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/00031305.2015.1031828?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Horton N.J. & Lipsitz S.R. & Parzen M., 2003. "A Potential for Bias When Rounding in Multiple Imputation," The American Statistician, American Statistical Association, vol. 57, pages 229-232, November.
    2. H. Schneeweiss & J. Komlos & A. Ahmad, 2010. "Symmetric and asymmetric rounding: a review and some new results," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 94(3), pages 247-271, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rubio, F.J. & Steel, M.F.J., 2011. "Inference for grouped data with a truncated skew-Laplace distribution," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3218-3231, December.
    2. Paul T. von Hippel, 2013. "Should a Normal Imputation Model be Modified to Impute Skewed Variables?," Sociological Methods & Research, , vol. 42(1), pages 105-138, February.
    3. repec:zbw:bofrdp:2021_010 is not listed on IDEAS
    4. Tatjana Miljkovic & Ying-Ju Chen, 2021. "A new computational approach for estimation of the Gini index based on grouped data," Computational Statistics, Springer, vol. 36(3), pages 2289-2311, September.
    5. Matthew Desmond & Tracey Shollenberger, 2015. "Forced Displacement From Rental Housing: Prevalence and Neighborhood Consequences," Demography, Springer;Population Association of America (PAA), vol. 52(5), pages 1751-1772, October.
    6. Yajuan Si & Jerome P. Reiter, 2013. "Nonparametric Bayesian Multiple Imputation for Incomplete Categorical Variables in Large-Scale Assessment Surveys," Journal of Educational and Behavioral Statistics, , vol. 38(5), pages 499-521, October.
    7. Davide Vidotto & Jeroen K. Vermunt & Katrijn van Deun, 2018. "Bayesian Multilevel Latent Class Models for the Multiple Imputation of Nested Categorical Data," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 511-539, October.
    8. Stuart R. Lipsitz & Garrett M. Fitzmaurice & Roger D. Weiss, 2020. "Using Multiple Imputation with GEE with Non-monotone Missing Longitudinal Binary Outcomes," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 890-904, December.
    9. Kajal Lahiri & Zulkarnain Pulungan, 2006. "Health Inequality and Its Determinants in New York," Discussion Papers 06-03, University at Albany, SUNY, Department of Economics.
    10. Chae, David H. & Lincoln, Karen D. & Adler, Nancy E. & Syme, S. Leonard, 2010. "Do experiences of racial discrimination predict cardiovascular disease among African American men? The moderating role of internalized negative racial group attitudes," Social Science & Medicine, Elsevier, vol. 71(6), pages 1182-1188, September.
    11. Stanisławska, Ewa & Paloviita, Maritta, 2021. "Medium- vs. short-term consumer inflation expectations: Evidence from a new euro area survey," Bank of Finland Research Discussion Papers 10/2021, Bank of Finland.
    12. Lahiri, Kajal & Pulungan, Zulkarnain, 2007. "Income-related health disparity and its determinants in New York state: racial/ethnic and geographical comparisons," MPRA Paper 21694, University Library of Munich, Germany.
    13. Humera Razzak & Christian Heumann, 2019. "Hybrid Multiple Imputation In A Large Scale Complex Survey," Statistics in Transition New Series, Polish Statistical Association, vol. 20(4), pages 33-58, December.
    14. Drechsler, Jörg & Kiesl, Hans, 2014. "Beat the heap - an imputation strategy for valid inferences from rounded income data," IAB-Discussion Paper 201402, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    15. Razzak Humera & Heumann Christian, 2019. "Hybrid Multiple Imputation In A Large Scale Complex Survey," Statistics in Transition New Series, Polish Statistical Association, vol. 20(4), pages 33-58, December.
    16. Christopher Withers & Saralees Nadarajah, 2015. "Cumulants of a random variable distributed uniformly on the first $$n$$ n integers," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 99(2), pages 229-236, April.
    17. Schwekendiek, Daniel & Baten, Joerg, 2019. "Height development of men and women from China, South Korea, and Taiwan during the rapid economic transformation period of the 1960s–1980s," Economics & Human Biology, Elsevier, vol. 34(C), pages 169-180.
    18. Jean-Pascal Bassino & Marion Dovis & John Komlos, 2018. "Biological well-being in late nineteenth-century Philippines," Cliometrica, Journal of Historical Economics and Econometric History, Association Française de Cliométrie (AFC), vol. 12(1), pages 33-60, January.
    19. Jan Schulz & Mishael Milaković, 2023. "How Wealthy are the Rich?," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 69(1), pages 100-123, March.
    20. Kristian Kleinke & Jost Reinecke & Cornelia Weins, 2021. "The development of delinquency during adolescence: a comparison of missing data techniques revisited," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(3), pages 877-895, June.
    21. Xu, Wan & Khachatryan, Hayk, 2014. "Multiple Imputation in the Complex National Nursery Survey Data by Fully Conditional Specification," 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota 170208, Agricultural and Applied Economics Association.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:amstat:v:69:y:2015:i:3:p:174-181. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UTAS20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.