IDEAS home Printed from https://ideas.repec.org/a/spr/soinre/v146y2019i1d10.1007_s11205-018-1893-4.html
   My bibliography  Save this article

Testing Equality of Functions Across Multiple Experimental Conditions for Different Ability Levels in the IRT Context: The Case of the IPRASE TLT 2016 Survey

Author

Listed:
  • Fabrizio Maturo

    (“G. d’ Annunzio” University)

  • Francesca Fortuna

    (“G. d’ Annunzio” University)

  • Tonio Di Battista

    (“G. d’ Annunzio” University)

Abstract

In the educational field, it is common to analyze test data through item response theory models. In this context, a key role is played by item characteristic curves (ICCs) and item information curves (IICs). In many real cases, practitioners are interested in understanding if some factors have a significant influence on the probability of correctly answering items. In the literature, this problem has been addressed by applying the standard analysis of variance model, which is based on the total scores or the proportion of correct responses. However, this method needs to meet some strong assumptions and may present some limitations because it does not consider useful information typical of the IRT, such as the shapes of the ICCs and IICs, which provide interesting insights for different ability levels. To overcome these issues, this research suggests the use of the functional analysis of variance approach and a novel functional tool in the IRT context. The main advantages of this approach are that it is distribution-free and allows us to check the degree of consistency with the hypothesis of equality among mean curves for different ability levels. Specifically, the proposed method is applied on ICCs and IICs for improving the existing techniques in the educational studies. A real dataset drawn from the IPRASE Trentino Language Testing Survey 2016 is considered. The final purpose of this study is to provide additional tools for scholars and practitioners in defining specific educational plans.

Suggested Citation

  • Fabrizio Maturo & Francesca Fortuna & Tonio Di Battista, 2019. "Testing Equality of Functions Across Multiple Experimental Conditions for Different Ability Levels in the IRT Context: The Case of the IPRASE TLT 2016 Survey," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 146(1), pages 19-39, November.
    • Handle: RePEc:spr:soinre:v:146:y:2019:i:1:d:10.1007_s11205-018-1893-4
    DOI: 10.1007/s11205-018-1893-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11205-018-1893-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11205-018-1893-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ssu-Kuang Chen & Fang-Ming Hwang & Sunny Lin, 2013. "Satisfaction Ratings of QOLPAV: Psychometric Properties Based on the Graded Response Model," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 110(1), pages 367-383, January.
    2. Natasha Rossi & Xiaohui Wang & James O. Ramsay, 2002. "Nonparametric Item Response Function Estimates with the EM Algorithm," Journal of Educational and Behavioral Statistics, , vol. 27(3), pages 291-317, September.
    3. Brian O’Connor & Maxine Crawford & Mark Holder, 2015. "An Item Response Theory Analysis of the Subjective Happiness Scale," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 124(1), pages 249-258, October.
    4. R. Bock & Murray Aitkin, 1981. "Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm," Psychometrika, Springer;The Psychometric Society, vol. 46(4), pages 443-459, December.
    5. J. Ramsay, 1991. "Kernel smoothing approaches to nonparametric item characteristic curve estimation," Psychometrika, Springer;The Psychometric Society, vol. 56(4), pages 611-630, December.
    6. Rizopoulos, Dimitris, 2006. "ltm: An R Package for Latent Variable Modeling and Item Response Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 17(i05).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Javier Revuelta, 2004. "Analysis of distractor difficulty in multiple-choice items," Psychometrika, Springer;The Psychometric Society, vol. 69(2), pages 217-234, June.
    2. Chanjin Zheng & Shaoyang Guo & Justin L Kern, 2021. "Fast Bayesian Estimation for the Four-Parameter Logistic Model (4PLM)," SAGE Open, , vol. 11(4), pages 21582440211, October.
    3. Christian Genest & Johanna G. Nešlehová, 2014. "A Conversation with James O. Ramsay," International Statistical Review, International Statistical Institute, vol. 82(2), pages 161-183, August.
    4. Marie Wiberg & James O. Ramsay & Juan Li, 2019. "Optimal Scores: An Alternative to Parametric Item Response Theory and Sum Scores," Psychometrika, Springer;The Psychometric Society, vol. 84(1), pages 310-322, March.
    5. Yang Liu & Ji Seung Yang, 2018. "Bootstrap-Calibrated Interval Estimates for Latent Variable Scores in Item Response Theory," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 333-354, June.
    6. Yang Liu & Weimeng Wang, 2022. "Semiparametric Factor Analysis for Item-Level Response Time Data," Psychometrika, Springer;The Psychometric Society, vol. 87(2), pages 666-692, June.
    7. Longjuan Liang & Michael W. Browne, 2015. "A Quasi-Parametric Method for Fitting Flexible Item Response Functions," Journal of Educational and Behavioral Statistics, , vol. 40(1), pages 5-34, February.
    8. Michela Battauz, 2015. "Factors affecting the variability of IRT equating coefficients," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 69(2), pages 85-101, May.
    9. Chalmers, R. Philip, 2012. "mirt: A Multidimensional Item Response Theory Package for the R Environment," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i06).
    10. John Patrick Lalor & Pedro Rodriguez, 2023. "py-irt : A Scalable Item Response Theory Library for Python," INFORMS Journal on Computing, INFORMS, vol. 35(1), pages 5-13, January.
    11. Michael Peress, 2012. "Identification of a Semiparametric Item Response Model," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 223-243, April.
    12. Michela Battauz, 2019. "On Wald tests for differential item functioning detection," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 28(1), pages 103-118, March.
    13. Ying Cheng & Ke-Hai Yuan, 2010. "The Impact of Fallible Item Parameter Estimates on Latent Trait Recovery," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 280-291, June.
    14. Alberto Maydeu-Olivares & Rosa Montaño, 2013. "How Should We Assess the Fit of Rasch-Type Models? Approximating the Power of Goodness-of-Fit Statistics in Categorical Data Analysis," Psychometrika, Springer;The Psychometric Society, vol. 78(1), pages 116-133, January.
    15. Carolina Navarro & Luis Ayala & José Labeaga, 2010. "Housing deprivation and health status: evidence from Spain," Empirical Economics, Springer, vol. 38(3), pages 555-582, June.
    16. Henry Santa-Cruz-Espinoza & Gina Chávez-Ventura & Julio Domínguez-Vergara & César Merino-Soto, 2023. "Internal Structure of the Work–Family Conflict Questionnaire (WFCQ) in Teacher Teleworking," IJERPH, MDPI, vol. 20(2), pages 1-16, January.
    17. Joel A. Martínez-Regalado & Cinthia Leonora Murillo-Avalos & Purificación Vicente-Galindo & Mónica Jiménez-Hernández & José Luis Vicente-Villardón, 2021. "Using HJ-Biplot and External Logistic Biplot as Machine Learning Methods for Corporate Social Responsibility Practices for Sustainable Development," Mathematics, MDPI, vol. 9(20), pages 1-16, October.
    18. Nick Bailey, 2020. "Measuring Poverty Efficiently Using Adaptive Deprivation Scales," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 149(3), pages 891-910, June.
    19. W. W. Koczkodaj & T. Kakiashvili & A. Szymańska & J. Montero-Marin & R. Araya & J. Garcia-Campayo & K. Rutkowski & D. Strzałka, 2017. "How to reduce the number of rating scale items without predictability loss?," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(2), pages 581-593, May.
    20. Cafarelli Ryan & Rigdon Christopher J. & Rigdon Steven E., 2012. "Models for Third Down Conversion in the National Football League," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 8(3), pages 1-26, October.

    More about this item

    Keywords

    IRT; ICC; IIC; FANOVA; P-Statistic;
    All these keywords.

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:soinre:v:146:y:2019:i:1:d:10.1007_s11205-018-1893-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.