IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v47y2022i4p406-437.html

A Critical View on the NEAT Equating Design: Statistical Modeling and Identifiability Problems

Author

Listed:
  • Ernesto San Martín

    (Millennium Nucleus on Intergenerational Mobility: From Modelling to Policy (MOVI), Chile
    Faculty of Mathematics, Pontificia Universidad Católica de Chile, Chile
    Interdisciplinary Laboratory of Social Statistics, Chile
    The Economics School of Louvain, Université Catholique de Louvain, Belgium)

  • Jorge González

    (Millennium Nucleus on Intergenerational Mobility: From Modelling to Policy (MOVI), Chile
    Faculty of Mathematics, Pontificia Universidad Católica de Chile, Chile
    Interdisciplinary Laboratory of Social Statistics, Chile)

Abstract

The nonequivalent groups with anchor test (NEAT) design is widely used in test equating. Under this design, two groups of examinees are administered different test forms with each test form containing a subset of common items. Because test takers from different groups are assigned only one test form, missing score data emerge by design rendering some of the score distributions unavailable. The partially observed score data formally lead to an identifiability problem, which has not been recognized as such in the equating literature and has been considered from different perspectives, all of them making different assumptions in order to estimate the unidentified score distributions. In this article, we formally specify the statistical model underlying the NEAT design and unveil the lack of identifiability of the parameters of interest that compose the equating transformation. We use the theory of partial identification to show alternatives to traditional practices that have been proposed to identify the score distributions when conducting equating under the NEAT design.

Suggested Citation

  • Ernesto San Martín & Jorge González, 2022. "A Critical View on the NEAT Equating Design: Statistical Modeling and Identifiability Problems," Journal of Educational and Behavioral Statistics, , vol. 47(4), pages 406-437, August.
    • Handle: RePEc:sae:jedbes:v:47:y:2022:i:4:p:406-437
    DOI: 10.3102/10769986221090609
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986221090609
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986221090609?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gundersen, Craig & Kreider, Brent, 2009. "Bounding the effects of food insecurity on children's health outcomes," Journal of Health Economics, Elsevier, vol. 28(5), pages 971-983, September.
    2. Paul Embrechts & Marius Hofert, 2013. "A note on generalized inverses," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 77(3), pages 423-432, June.
    3. Florens, J.P. & Mouchart, M., 1982. "A note on noncausality," LIDAM Reprints CORE 479, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    4. Tamer, Elie, 2010. "Partial Identification in Econometrics," Scholarly Articles 34728615, Harvard University Department of Economics.
    5. Wim J. van der Linden, 2019. "Lord’s Equity Theorem Revisited," Journal of Educational and Behavioral Statistics, , vol. 44(4), pages 415-430, August.
    6. Michelle Liou & Philip Cheng, 1995. "Equipercentile equating via data-imputation techniques," Psychometrika, Springer;The Psychometric Society, vol. 60(1), pages 119-136, March.
    7. Wunsch, Guillaume & Mouchart, Michel & Russo, Federica, 2014. "Functions and Mechanisms in Structural-Modelling Explanations," LIDAM Reprints ISBA 2014011, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    8. Jörg Stoye, 2010. "Partial identification of spread parameters," Quantitative Economics, Econometric Society, vol. 1(2), pages 323-357, November.
    9. Richard Blundell & Amanda Gosling & Hidehiko Ichimura & Costas Meghir, 2007. "Changes in the Distribution of Male and Female Wages Accounting for Employment Composition Using Bounds," Econometrica, Econometric Society, vol. 75(2), pages 323-363, March.
    10. Kei Miyazaki & Takahiro Hoshino & Shin-ichi Mayekawa & Kazuo Shigemasu, 2009. "A New Concurrent Calibration Method for Nonequivalent Group Design under Nonrandom Assignment," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 1-19, March.
    11. John V. Pepper, 2000. "The Intergenerational Transmission Of Welfare Receipt: A Nonparametric Bounds Analysis," The Review of Economics and Statistics, MIT Press, vol. 82(3), pages 472-488, August.
    12. Sandip Sinharay & Paul Holland, 2010. "The Missing Data Assumptions of the NEAT Design and their Implications for Test Equating," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 309-327, June.
    13. Wim J. Linden & Michelle D. Barrett, 2016. "Linking Item Response Model Parameters," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 650-673, September.
    14. Florens, J P & Mouchart, M, 1982. "A Note on Noncausality," Econometrica, Econometric Society, vol. 50(3), pages 583-591, May.
    15. Elie Tamer, 2010. "Partial Identification in Econometrics," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 167-195, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wooyoung Kim & Koohyun Kwon & Soonwoo Kwon & Sokbae Lee, 2018. "The identification power of smoothness assumptions in models with counterfactual outcomes," Quantitative Economics, Econometric Society, vol. 9(2), pages 617-642, July.
    2. Molinari, Francesca, 2020. "Microeconometrics with partial identification," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 355-486, Elsevier.
    3. Ismaël Mourifié & Marc Henry & Romuald Méango, 2020. "Sharp Bounds and Testability of a Roy Model of STEM Major Choices," Journal of Political Economy, University of Chicago Press, vol. 128(8), pages 3220-3283.
    4. Monique de Haan, 2011. "The Effect of Parents' Schooling on Child's Schooling: A Nonparametric Bounds Analysis," Journal of Labor Economics, University of Chicago Press, vol. 29(4), pages 859-892.
    5. Manuel Arellano & Stéphane Bonhomme, 2017. "Sample Selection in Quantile Regression: A Survey," Working Papers wp2018_1702, CEMFI.
    6. Manuel Arellano & Stéphane Bonhomme, 2017. "Sample Selection in Quantile Regression: A Survey," Working Papers wp2017_1702, CEMFI.
    7. Benjamin C. Hamilton, 2024. "Identification Problems in Probabilistic Measures of Perceived Arrest Risk: Estimating a Partially-Identified Certainty Effect," Journal of Quantitative Criminology, Springer, vol. 40(2), pages 285-310, June.
    8. Daniel Millimet & Manan Roy, 2015. "Partial identification of the long-run causal effect of food security on child health," Empirical Economics, Springer, vol. 48(1), pages 83-141, February.
    9. Punarjit Roychowdhury & Gaurav Dhamija, 2024. "Educational hypogamy and female employment in rural India," Empirical Economics, Springer, vol. 67(6), pages 2893-2931, December.
    10. Oliver R. Cutbill & Rami V. Tabri, 2022. "The Impossibility of Testing for Dependence Using Kendall’s Ƭ Under Missing Data of Unknown Form," Working Papers 2022-03, University of Sydney, School of Economics.
    11. Angela Blanco-Fernández & Peter Winker, 2016. "Data generation processes and statistical management of interval data," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 100(4), pages 475-494, October.
    12. Punarjit Roychowdhury & Gaurav Dhamija, 2022. "Don't cross the line: Bounding the causal effect of hypergamy violation on domestic violence in India," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(4), pages 1952-1978, October.
    13. Juan Carlos Escanciano & Lin Zhu, 2013. "Set inferences and sensitivity analysis in semiparametric conditionally identified models," CeMMAP working papers CWP55/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    14. Fakih, Ali & Makdissi, Paul & Marrouch, Walid & Tabri, Rami V. & Yazbeck, Myra, 2022. "A stochastic dominance test under survey nonresponse with an application to comparing trust levels in Lebanese public institutions," Journal of Econometrics, Elsevier, vol. 228(2), pages 342-358.
    15. Sung Jae Jun & Sokbae Lee, 2024. "Causal Inference Under Outcome-Based Sampling with Monotonicity Assumptions," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 42(3), pages 998-1009, July.
    16. Vikesh Amin & Jere R. Behrman & Jason M. Fletcher & Carlos A. Flores & Alfonso Flores-Lagunes & Hans-Peter Kohler, 2022. "Does Schooling Improve Cognitive Abilities at Older Ages: Causal Evidence from Nonparametric Bounds," PIER Working Paper Archive 22-016, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    17. Vishal Kamat, 2017. "Identifying the Effects of a Program Offer with an Application to Head Start," Papers 1711.02048, arXiv.org, revised Aug 2023.
    18. Millimet, Daniel L. & Roy, Jayjit, 2015. "Multilateral environmental agreements and the WTO," Economics Letters, Elsevier, vol. 134(C), pages 20-23.
    19. Breunig, Christoph & Mammen, Enno & Simoni, Anna, 2018. "Nonparametric estimation in case of endogenous selection," Journal of Econometrics, Elsevier, vol. 202(2), pages 268-285.
    20. Donald S. Poskitt & Xueyan Zhao, 2023. "Bootstrap Hausdorff Confidence Regions for Average Treatment Effect Identified Sets," Monash Econometrics and Business Statistics Working Papers 9/23, Monash University, Department of Econometrics and Business Statistics.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:47:y:2022:i:4:p:406-437. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.