IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v45y2020i3p316-338.html
   My bibliography  Save this article

Adaptive Pairwise Comparison for Educational Measurement

Author

Listed:
  • Elise A. V. Crompvoets

    (Tilburg University
    Cito)

  • Anton A. Béguin

    (Cito)

  • Klaas Sijtsma

    (Tilburg University)

Abstract

Pairwise comparison is becoming increasingly popular as a holistic measurement method in education. Unfortunately, many comparisons are required for reliable measurement. To reduce the number of required comparisons, we developed an adaptive selection algorithm (ASA) that selects the most informative comparisons while taking the uncertainty of the object parameters into account. The results of the simulation study showed that, given the number of comparisons, the ASA resulted in smaller standard errors of object parameter estimates than a random selection algorithm that served as a benchmark. Rank order accuracy and reliability were similar for the two algorithms. Because the scale separation reliability (SSR) may overestimate the benchmark reliability when the ASA is used, caution is required when interpreting the SSR.

Suggested Citation

  • Elise A. V. Crompvoets & Anton A. Béguin & Klaas Sijtsma, 2020. "Adaptive Pairwise Comparison for Educational Measurement," Journal of Educational and Behavioral Statistics, , vol. 45(3), pages 316-338, June.
    • Handle: RePEc:sae:jedbes:v:45:y:2020:i:3:p:316-338
    DOI: 10.3102/1076998619890589
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998619890589
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998619890589?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Klaas Sijtsma, 2009. "On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha," Psychometrika, Springer;The Psychometric Society, vol. 74(1), pages 107-120, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chang, Long & Ma, Chen & Zhang, Chenghui & Duan, Bin & Cui, Naxin & Li, Changlong, 2023. "Correlations of lithium-ion battery parameter variations and connected configurations on pack statistics," Applied Energy, Elsevier, vol. 329(C).
    2. Hartman, Dana T. & Goodman, Gail S. & Grandchamp, Jaxon & Vidales, Daisy & Shaver, Phillip R. & Lawler, Michael J. & Hobbs, Sue D., 2023. "Outcomes for foster youth placed at a special residential high school: A quantitative case study," Children and Youth Services Review, Elsevier, vol. 151(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Conzo, Pierluigi & Aassve, Arnstein & Fuochi, Giulia & Mencarini, Letizia, 2017. "The cultural foundations of happiness," Journal of Economic Psychology, Elsevier, vol. 62(C), pages 268-283.
    2. Xiaochuan Song, 2022. "Investigating Employees’ Responses to Abusive Supervision," Merits, MDPI, vol. 2(4), pages 1-20, November.
    3. Carmen León-Mantero & José Carlos Casas-Rosal & Alexander Maz-Machado & Miguel E Villarraga Rico, 2020. "Analysis of attitudinal components towards statistics among students from different academic degrees," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-13, January.
    4. Danni Liu & Anouk Dijk & Shanyan Lin & Zhenhong Wang & Maja Deković & Judith Semon Dubas, 2023. "Psychometric Properties of the Chinese Version of the Highly Sensitive Child Scale Across Age Groups, Gender, and Informants," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 16(4), pages 1755-1780, August.
    5. Brian K Miller & Kay M Nicols & Silvia Clark & Alison Daniels & Whitney Grant, 2018. "Meta-analysis of coefficient alpha for scores on the Narcissistic Personality Inventory," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-16, December.
    6. Adam Pawlewicz & Wojciech Gotkiewicz & Katarzyna Brodzińska & Katarzyna Pawlewicz & Bartosz Mickiewicz & Paweł Kluczek, 2022. "Organic Farming as an Alternative Maintenance Strategy in the Opinion of Farmers from Natura 2000 Areas," IJERPH, MDPI, vol. 19(7), pages 1-22, March.
    7. Rue, Lisa A. & Estrada, Samantha & Floren, Michael & MacKinnon, Krista, 2016. "Formative evaluation: Developing measures for online family mental health recovery education," Evaluation and Program Planning, Elsevier, vol. 55(C), pages 27-34.
    8. Isabel Gallego‐Álvarez & María Consuelo Pucheta‐Martínez, 2020. "How cultural dimensions, legal systems, and industry affect environmental reporting? Empirical evidence from an international perspective," Business Strategy and the Environment, Wiley Blackwell, vol. 29(5), pages 2037-2057, July.
    9. Sommerland, Nina & Masquillier, Caroline & Rau, Asta & Engelbrecht, Michelle & Kigozi, Gladys & Pliakas, Triantafyllos & Janse van Rensburg, Andre & Wouters, Edwin, 2020. "Reducing HIV- and TB-Stigma among healthcare co-workers in South Africa: Results of a cluster randomised trial," Social Science & Medicine, Elsevier, vol. 266(C).
    10. Michael Hennessy & Amy Bleakley & Martin Fishbein, 2012. "Measurement Models for Reasoned Action Theory," The ANNALS of the American Academy of Political and Social Science, , vol. 640(1), pages 42-57, March.
    11. Roberta Fida & Carlo Tramontano & Marinella Paciello & Valerio Ghezzi & Claudio Barbaranelli, 2018. "Understanding the Interplay Among Regulatory Self-Efficacy, Moral Disengagement, and Academic Cheating Behaviour During Vocational Education: A Three-Wave Study," Journal of Business Ethics, Springer, vol. 153(3), pages 725-740, December.
    12. José Miguel Morales‐Asencio & Ana María Porcel‐Gálvez & Rosa Oliveros‐Valenzuela & Susana Rodríguez‐Gómez & Lucrecia Sánchez‐Extremera & Francisco Andrés Serrano‐López & Marta Aranda‐Gallardo & José C, 2015. "Design and validation of the INICIARE instrument, for the assessment of dependency level in acutely ill hospitalised patients," Journal of Clinical Nursing, John Wiley & Sons, vol. 24(5-6), pages 761-777, March.
    13. Rauter, Romana & Globocnik, Dietfried & Baumgartner, Rupert J., 2023. "The role of organizational controls to advance sustainability innovation performance," Technovation, Elsevier, vol. 128(C).
    14. Salim Moussa, 2016. "A Comment on the Estimation of the Reliability of Multidimensional Marketing Constructs: A Store Personality Scale Application," Global Business Review, International Management Institute, vol. 17(5), pages 1125-1144, October.
    15. Hye Bin Rim & Brandon M. Turner & Nancy E. Betz & Thomas E. Nygren, 2011. "Studies of the dimensionality, correlates, and meaning of measures of the maximizing tendency," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 6(6), pages 565-579, August.
    16. Jeanne A. Teresi & Katja Ocepek-Welikson & John A. Toner & Marjorie Kleinman & Mildred Ramirez & Joseph P. Eimicke & Barry J. Gurland & Albert Siu, 2017. "Methodological Issues in Measuring Subjective Well-Being and Quality-of-Life: Applications to Assessment of Affect in Older, Chronically and Cognitively Impaired, Ethnically Diverse Groups Using the F," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 12(2), pages 251-288, June.
    17. Tingey-Holyoak, Joanne, 2014. "Sustainable water storage by agricultural businesses: Strategic responses to institutional pressures," Journal of Business Research, Elsevier, vol. 67(12), pages 2590-2602.
    18. de Boer, Joop & Aiking, Harry, 2021. "Climate change and species decline: Distinct sources of European consumer concern supporting more sustainable diets," Ecological Economics, Elsevier, vol. 188(C).
    19. Lena Busch & Till Utesch & Bernd Strauss, 2019. "Validation of the vignette-based German Exercise Causality Orientation Scale (G-ECOS)," PLOS ONE, Public Library of Science, vol. 14(10), pages 1-20, October.
    20. Eunseong Cho, 2021. "Neither Cronbach’s Alpha nor McDonald’s Omega: A Commentary on Sijtsma and Pfadt," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 877-886, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:45:y:2020:i:3:p:316-338. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.