IDEAS home Printed from https://ideas.repec.org/a/gam/jjopen/v2y2019i1p4-49d198984.html

Concept and Implementation of a Two-Stage Coding Scheme for the Development of Computer-Based Testing (CBT)-Items in Traditional Test Software

Author

Listed:
  • Thilo J. Ketschau

    (Department of Educational Philosophy and Vocational Education, TU Dortmund University, 44227 Dortmund, Germany)

  • Janne Kleinhans

    (TOPSIM GmbH, 72070 Tübingen, Germany)

Abstract

Computer-based testing (CBT) is gaining importance for studies addressing the diagnosis of competencies, because it is possible to simulate authentic action situations and may reduce the effort of analyzing the data. This benefit is most important for the phase of item design. In this phase of assessment construction, the pattern of answers of a sample is used to draw conclusions on the functionality of the items. Currently, there are no standards for the encodement of items which consider the specifications of CBT-instruments. These specifications are, for example, the a posteriori non-variability of the coding, a lack of information when using conventional test scores and the need of standardization of different formats of items. Taking these specifications into consideration, this paper proposes and discusses a two-stage coding systematization for CBT-items. For this, a distinction between item-coding and answer-coding was done. The coding is discussed for single-section and multi-section formats as well as dichotomous and polytomous answer modes. Therefore, this paper is for users of CBT-instruments who want to achieve the optimal information value of their test results with efficient coding.

Suggested Citation

  • Thilo J. Ketschau & Janne Kleinhans, 2019. "Concept and Implementation of a Two-Stage Coding Scheme for the Development of Computer-Based Testing (CBT)-Items in Traditional Test Software," J, MDPI, vol. 2(1), pages 1-9, January.
    • Handle: RePEc:gam:jjopen:v:2:y:2019:i:1:p:4-49:d:198984
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2571-8800/2/1/4/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2571-8800/2/1/4/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. César Merino-Soto & Gina Chávez-Ventura & Verónica López-Fernández & Guillermo M. Chans & Filiberto Toledano-Toledano, 2022. "Learning Self-Regulation Questionnaire (SRQ-L): Psychometric and Measurement Invariance Evidence in Peruvian Undergraduate Students," Sustainability, MDPI, vol. 14(18), pages 1-17, September.
    2. Fumiko Samejima, 1997. "Departure from normal assumptions: A promise for future psychometrics with substantive mathematical modeling," Psychometrika, Springer;The Psychometric Society, vol. 62(4), pages 471-493, December.
    3. Geofferey Masters, 1985. "A comparison of latent trait and latent class analyses of Likert-type data," Psychometrika, Springer;The Psychometric Society, vol. 50(1), pages 69-82, March.
    4. Anja C. Rohenkohl & Monika Bullinger & Andreas M. Pleil & Levente Kriston & Julia H. Quitmann, 2016. "A Brief Version of the Quality of Life in Short Stature Youth Questionnaire - the QoLISSY-Brief," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 9(4), pages 971-984, December.
    5. Roberto Burro & Riccardo Sartori & Giulio Vidotto, 2011. "The method of constant stimuli with three rating categories and the use of Rasch models," Quality & Quantity: International Journal of Methodology, Springer, vol. 45(1), pages 43-58, January.
    6. P. A. Ferrari & S. Salini, 2008. "Measuring Service Quality: The Opinion of Europeans about Utilities," Working Papers 2008.36, Fondazione Eni Enrico Mattei.
    7. Daphna Harel & Russell J. Steele, 2018. "An Information Matrix Test for the Collapsing of Categories Under the Partial Credit Model," Journal of Educational and Behavioral Statistics, , vol. 43(6), pages 721-750, December.
    8. Clelia Cascella & Maria Pampaka & Julian Williams, 2024. "Gender Attitudes Within and Between European Countries: Regional Variations Matter," SAGE Open, , vol. 14(3), pages 21582440241, August.
    9. Chang, Hsin-Li & Yang, Cheng-Hua, 2008. "Explore airlines’ brand niches through measuring passengers’ repurchase motivation—an application of Rasch measurement," Journal of Air Transport Management, Elsevier, vol. 14(3), pages 105-112.
    10. Laura Maldonado-Murciano & Halley M. Pontes & Mark D. Griffiths & Maite Barrios & Juana Gómez-Benito & Georgina Guilera, 2020. "The Spanish Version of the Internet Gaming Disorder Scale-Short Form (IGDS9-SF): Further Examination Using Item Response Theory," IJERPH, MDPI, vol. 17(19), pages 1-14, September.
    11. Galeotti, Marzio & Rubashkina, Yana & Salini, Silvia & Verdolini, Elena, 2018. "Environmental policy performance and its determinants: Application of a three-level random intercept model," Energy Policy, Elsevier, vol. 114(C), pages 134-144.
    12. Luo, Nanyu & Ji, Feng & Han, Yuting & He, Jinbo & Zhang, Xiaoya, 2024. "Fitting item response theory models using deep learning computational frameworks," OSF Preprints tjxab, Center for Open Science.
    13. Ivana Bassi & Matteo Carzedda & Enrico Gori & Luca Iseppi, 2022. "Rasch analysis of consumer attitudes towards the mountain product label," Agricultural and Food Economics, Springer;Italian Society of Agricultural Economics (SIDEA), vol. 10(1), pages 1-25, December.
    14. Matthias Davier & Xueli Xu & Claus Carstensen, 2011. "Measuring Growth in a Longitudinal Large-Scale Assessment with a General Latent Variable Model," Psychometrika, Springer;The Psychometric Society, vol. 76(2), pages 318-336, April.
    15. Antonio Caronni & Marina Ramella & Pietro Arcuri & Claudia Salatino & Lucia Pigini & Maurizio Saruggia & Chiara Folini & Stefano Scarano & Rosa Maria Converti, 2023. "The Rasch Analysis Shows Poor Construct Validity and Low Reliability of the Quebec User Evaluation of Satisfaction with Assistive Technology 2.0 (QUEST 2.0) Questionnaire," IJERPH, MDPI, vol. 20(2), pages 1-19, January.
    16. Singh, Jagdip, 2004. "Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example," Journal of Business Research, Elsevier, vol. 57(2), pages 184-208, February.
    17. Hua-Hua Chang, 1996. "The asymptotic posterior normality of the latent trait for polytomous IRT models," Psychometrika, Springer;The Psychometric Society, vol. 61(3), pages 445-463, September.
    18. repec:osf:osfxxx:tjxab_v1 is not listed on IDEAS
    19. Ann Yang, 2013. "Decision Making for Individual Investors: A Measurement of Latent Difficulties," Journal of Financial Services Research, Springer;Western Finance Association, vol. 44(3), pages 303-329, December.
    20. Curt Hagquist & Raili Välimaa & Nina Simonsen & Sakari Suominen, 2017. "Differential Item Functioning in Trend Analyses of Adolescent Mental Health – Illustrative Examples Using HBSC-Data from Finland," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 10(3), pages 673-691, September.
    21. Edward Ip & Yuchung Wang & Paul Boeck & Michel Meulders, 2004. "Locally dependent latent trait model for polytomous responses with application to inventory of hostility," Psychometrika, Springer;The Psychometric Society, vol. 69(2), pages 191-216, June.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jjopen:v:2:y:2019:i:1:p:4-49:d:198984. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager The email address of this maintainer does not seem to be valid anymore. Please ask MDPI Indexing Manager to update the entry or send us the correct address (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.