IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v42y2017i6p678-705.html
   My bibliography  Save this article

Absolute and Relative Measures of Instructional Sensitivity

Author

Listed:
  • Alexander Naumann
  • Johannes Hartig

    (German Institute for International Educational Research (DIPF))

  • Jan Hochweber

    (University of Teacher Education St. Gallen (PHSG))

Abstract

Valid inferences on teaching drawn from students’ test scores require that tests are sensitive to the instruction students received in class. Accordingly, measures of the test items’ instructional sensitivity provide empirical support for validity claims about inferences on instruction. In the present study, we first introduce the concepts of absolute and relative measures of instructional sensitivity. Absolute measures summarize a single item’s total capacity of capturing effects of instruction, which is independent of the test’s sensitivity. In contrast, relative measures summarize a single item’s capacity of capturing effects of instruction relative to test sensitivity. Then, we propose a longitudinal multilevel item response theory model that allows estimating both types of measures depending on the identification constraints.

Suggested Citation

  • Alexander Naumann & Johannes Hartig & Jan Hochweber, 2017. "Absolute and Relative Measures of Instructional Sensitivity," Journal of Educational and Behavioral Statistics, , vol. 42(6), pages 678-705, December.
    • Handle: RePEc:sae:jedbes:v:42:y:2017:i:6:p:678-705
    DOI: 10.3102/1076998617703649
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998617703649
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998617703649?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Paul Boeck, 2008. "Random Item IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 73(4), pages 533-559, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Quinn N. Lathrop & Ying Cheng, 2017. "Item Cloning Variation and the Impact on the Parameters of Response Models," Psychometrika, Springer;The Psychometric Society, vol. 82(1), pages 245-263, March.
    2. Antonio Caronni & Marina Ramella & Pietro Arcuri & Claudia Salatino & Lucia Pigini & Maurizio Saruggia & Chiara Folini & Stefano Scarano & Rosa Maria Converti, 2023. "The Rasch Analysis Shows Poor Construct Validity and Low Reliability of the Quebec User Evaluation of Satisfaction with Assistive Technology 2.0 (QUEST 2.0) Questionnaire," IJERPH, MDPI, vol. 20(2), pages 1-19, January.
    3. Sun-Joo Cho & Allan S. Cohen, 2010. "A Multilevel Mixture IRT Model With an Application to DIF," Journal of Educational and Behavioral Statistics, , vol. 35(3), pages 336-370, June.
    4. Sun-Joo Cho & Sarah Brown-Schmidt & Woo-yeol Lee, 2018. "Autoregressive Generalized Linear Mixed Effect Models with Crossed Random Effects: An Application to Intensive Binary Time Series Eye-Tracking Data," Psychometrika, Springer;The Psychometric Society, vol. 83(3), pages 751-771, September.
    5. Mariagiulia Matteucci & Bernard Veldkamp, 2015. "The approach of power priors for ability estimation in IRT models," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 917-926, May.
    6. Hanneke Geerlings & Cees Glas & Wim Linden, 2011. "Modeling Rule-Based Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 337-359, April.
    7. Sun-Joo Cho & Paul Boeck & Susan Embretson & Sophia Rabe-Hesketh, 2014. "Additive Multilevel Item Structure Models with Random Residuals: Item Modeling for Explanation and Item Generation," Psychometrika, Springer;The Psychometric Society, vol. 79(1), pages 84-104, January.
    8. Royce Anders & William Batchelder, 2015. "Cultural Consensus Theory for the Ordinal Data Case," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 151-181, March.
    9. Murray Aitkin & Duy Vu & Brian Francis, 2017. "Statistical modelling of a terrorist network," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(3), pages 751-768, June.
    10. Joshua B. Gilbert & James S. Kim & Luke W. Miratrix, 2023. "Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions," Journal of Educational and Behavioral Statistics, , vol. 48(6), pages 889-913, December.
    11. repec:jss:jstsof:39:i12 is not listed on IDEAS
    12. Sarra, Annalina & Di Zio, Simone & Cappucci, Marianna, 2015. "A quantitative valuation of tourist experience in Lisbon," Annals of Tourism Research, Elsevier, vol. 53(C), pages 1-16.
    13. Timo Bechger & Gunter Maris, 2015. "A Statistical Test for Differential Item Pair Functioning," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 317-340, June.
    14. Alexander Robitzsch, 2021. "A Comprehensive Simulation Study of Estimation Methods for the Rasch Model," Stats, MDPI, vol. 4(4), pages 1-23, October.
    15. Jean-Paul Fox & Joris Mulder & Sandip Sinharay, 2017. "Bayes Factor Covariance Testing in Item Response Models," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 979-1006, December.
    16. Alexander Robitzsch, 2020. "L p Loss Functions in Invariance Alignment and Haberman Linking with Few or Many Groups," Stats, MDPI, vol. 3(3), pages 1-38, August.
    17. Dora Matzke & Conor Dolan & William Batchelder & Eric-Jan Wagenmakers, 2015. "Bayesian Estimation of Multinomial Processing Tree Models with Heterogeneity in Participants and Items," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 205-235, March.
    18. Zita Oravecz & Royce Anders & William Batchelder, 2015. "Hierarchical Bayesian Modeling for Test Theory Without an Answer Key," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 341-364, June.
    19. Edgar C. Merkle & Daniel Furr & Sophia Rabe-Hesketh, 2019. "Bayesian Comparison of Latent Variable Models: Conditional Versus Marginal Likelihoods," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 802-829, September.
    20. Paul Boeck & Sun-Joo Cho, 2021. "Not all DIF is shaped similarly," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 712-716, September.
    21. Norman Rose & Matthias Davier & Benjamin Nagengast, 2017. "Modeling Omitted and Not-Reached Items in IRT Models," Psychometrika, Springer;The Psychometric Society, vol. 82(3), pages 795-819, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:42:y:2017:i:6:p:678-705. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.