IDEAS home Printed from https://ideas.repec.org/a/spr/metron/v81y2023i2d10.1007_s40300-022-00237-w.html
   My bibliography  Save this article

Theoretical evaluation of partial credit scoring of the multiple-choice test item

Author

Listed:
  • Rasmus A. X. Persson

    (University of Gothenburg)

Abstract

In multiple-choice tests, guessing is a source of test error which can be suppressed if its expected score is made negative by either penalizing wrong answers or rewarding expressions of partial knowledge. Starting from the most general formulation of the necessary and sufficient scoring conditions for guessing to lead to an expected loss beyond the test-taker’s knowledge, we formulate a class of optimal scoring functions, including the proposal by Zapechelnyuk (Econ. Lett. 132, 24–27 (2015)) as a special case. We then consider an arbitrary multiple-choice test taken by a rational test-taker whose knowledge of a test item is defined by the fraction of the answer options which can be ruled out. For this model, we study the statistical properties of the obtained score for both standard marking (where guessing is not penalized), and marking where guessing is suppressed either by expensive score penalties for incorrect answers or by different marking schemes that reward partial knowledge.

Suggested Citation

  • Rasmus A. X. Persson, 2023. "Theoretical evaluation of partial credit scoring of the multiple-choice test item," METRON, Springer;Sapienza Università di Roma, vol. 81(2), pages 143-161, August.
    • Handle: RePEc:spr:metron:v:81:y:2023:i:2:d:10.1007_s40300-022-00237-w
    DOI: 10.1007/s40300-022-00237-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40300-022-00237-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40300-022-00237-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jean Gibbons & Ingram Olkin & Milton Sobel, 1979. "A subset selection technique for scoring items on a multiple choice test," Psychometrika, Springer;The Psychometric Society, vol. 44(3), pages 259-270, September.
    2. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    3. Zapechelnyuk, Andriy, 2015. "An axiomatization of multiple-choice test scoring," Economics Letters, Elsevier, vol. 132(C), pages 24-27.
    4. Jef Vanderoost & Rianne Janssen & Jan Eggermont & Riet Callens & Tinne De Laet, 2018. "Elimination testing with adapted scoring reduces guessing and anxiety in multiple-choice assessments, but does not increase grade average in comparison with negative marking," PLOS ONE, Public Library of Science, vol. 13(10), pages 1-27, October.
    5. David Andrich, 1978. "A rating formulation for ordered response categories," Psychometrika, Springer;The Psychometric Society, vol. 43(4), pages 561-573, December.
    6. James Ramsay & Marie Wiberg & Juan Li, 2020. "Full Information Optimal Scoring," Journal of Educational and Behavioral Statistics, , vol. 45(3), pages 297-315, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. P. A. Ferrari & S. Salini, 2008. "Measuring Service Quality: The Opinion of Europeans about Utilities," Working Papers 2008.36, Fondazione Eni Enrico Mattei.
    2. Chang, Hsin-Li & Yang, Cheng-Hua, 2008. "Explore airlines’ brand niches through measuring passengers’ repurchase motivation—an application of Rasch measurement," Journal of Air Transport Management, Elsevier, vol. 14(3), pages 105-112.
    3. Ivana Bassi & Matteo Carzedda & Enrico Gori & Luca Iseppi, 2022. "Rasch analysis of consumer attitudes towards the mountain product label," Agricultural and Food Economics, Springer;Italian Society of Agricultural Economics (SIDEA), vol. 10(1), pages 1-25, December.
    4. Antonio Caronni & Marina Ramella & Pietro Arcuri & Claudia Salatino & Lucia Pigini & Maurizio Saruggia & Chiara Folini & Stefano Scarano & Rosa Maria Converti, 2023. "The Rasch Analysis Shows Poor Construct Validity and Low Reliability of the Quebec User Evaluation of Satisfaction with Assistive Technology 2.0 (QUEST 2.0) Questionnaire," IJERPH, MDPI, vol. 20(2), pages 1-19, January.
    5. Hua-Hua Chang, 1996. "The asymptotic posterior normality of the latent trait for polytomous IRT models," Psychometrika, Springer;The Psychometric Society, vol. 61(3), pages 445-463, September.
    6. Curt Hagquist & Raili Välimaa & Nina Simonsen & Sakari Suominen, 2017. "Differential Item Functioning in Trend Analyses of Adolescent Mental Health – Illustrative Examples Using HBSC-Data from Finland," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 10(3), pages 673-691, September.
    7. Salzberger, Thomas & Newton, Fiona J. & Ewing, Michael T., 2014. "Detecting gender item bias and differential manifest response behavior: A Rasch-based solution," Journal of Business Research, Elsevier, vol. 67(4), pages 598-607.
    8. Chang, Hsin-Li & Wu, Shun-Cheng, 2008. "Exploring the vehicle dependence behind mode choice: Evidence of motorcycle dependence in Taipei," Transportation Research Part A: Policy and Practice, Elsevier, vol. 42(2), pages 307-320, February.
    9. Genge, Ewa & Bartolucci, Francesco, 2019. "Are attitudes towards immigration changing in Europe? An analysis based on bidimensional latent class IRT models," MPRA Paper 94672, University Library of Munich, Germany.
    10. Jesper Tijmstra & Maria Bolsinova, 2019. "Bayes Factors for Evaluating Latent Monotonicity in Polytomous Item Response Theory Models," Psychometrika, Springer;The Psychometric Society, vol. 84(3), pages 846-869, September.
    11. Salzberger, Thomas & Koller, Monika, 2013. "Towards a new paradigm of measurement in marketing," Journal of Business Research, Elsevier, vol. 66(9), pages 1307-1317.
    12. Richard N McNeely & Salissou Moutari & Samuel Arba-Mosquera & Shwetabh Verma & Jonathan E Moore, 2018. "An alternative application of Rasch analysis to assess data from ophthalmic patient-reported outcome instruments," PLOS ONE, Public Library of Science, vol. 13(6), pages 1-32, June.
    13. Francesca DE BATTISTI & Giovanna NICOLINI & Silvia SALINI, 2008. "Methodological overview of Rasch model and application in customer satisfaction survey data," Departmental Working Papers 2008-04, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    14. Kuan-Yu Jin & Yi-Jhen Wu & Hui-Fang Chen, 2022. "A New Multiprocess IRT Model With Ideal Points for Likert-Type Items," Journal of Educational and Behavioral Statistics, , vol. 47(3), pages 297-321, June.
    15. van der Ark, L. Andries, 2012. "New Developments in Mokken Scale Analysis in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i05).
    16. Piotr Tarka, 2013. "Model of latent profile factor analysis for ordered categorical data," Statistics in Transition new series, Główny Urząd Statystyczny (Polska), vol. 14(1), pages 171-182, March.
    17. Xiaohui Zheng & Sophia Rabe-Hesketh, 2007. "Estimating parameters of dichotomous and ordinal item response models with gllamm," Stata Journal, StataCorp LP, vol. 7(3), pages 313-333, September.
    18. Lai-Fa Hung & Wen-Chung Wang, 2012. "The Generalized Multilevel Facets Model for Longitudinal Data," Journal of Educational and Behavioral Statistics, , vol. 37(2), pages 231-255, April.
    19. Cheng, Yung-Hsiang & Liu, Kuo-Chu, 2012. "Evaluating bicycle-transit users’ perceptions of intermodal inconvenience," Transportation Research Part A: Policy and Practice, Elsevier, vol. 46(10), pages 1690-1706.
    20. Ghady El Khoury & Olivier Barbier & Xavier Libouton & Jean-Louis Thonnard & Philippe Lefèvre & Massimo Penta, 2020. "Manual ability in hand surgery patients: Validation of the ABILHAND scale in four diagnostic groups," PLOS ONE, Public Library of Science, vol. 15(12), pages 1-17, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:metron:v:81:y:2023:i:2:d:10.1007_s40300-022-00237-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.