IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v111y2017i2d10.1007_s11192-017-2283-4.html
   My bibliography  Save this article

How to reduce the number of rating scale items without predictability loss?

Author

Listed:
  • W. W. Koczkodaj

    (Laurentian University)

  • T. Kakiashvili

    (Sudbury Therapy)

  • A. Szymańska

    (UKSW University)

  • J. Montero-Marin

    (University of Zaragoza)

  • R. Araya

    (London School of Hygiene and Tropical Medicine)

  • J. Garcia-Campayo

    (University of Zaragoza)

  • K. Rutkowski

    (Jagiellonian University)

  • D. Strzałka

    (Rzeszów University of Technology)

Abstract

Rating scales are used to elicit data about qualitative entities (e.g., research collaboration). This study presents an innovative method for reducing the number of rating scale items without the predictability loss. The “area under the receiver operator curve method” (AUC ROC) is used. The presented method has reduced the number of rating scale items (variables) to 28.57% (from 21 to 6) making over 70% of collected data unnecessary. Results have been verified by two methods of analysis: Graded Response Model (GRM) and Confirmatory Factor Analysis (CFA). GRM revealed that the new method differentiates observations of high and middle scores. CFA proved that the reliability of the rating scale has not deteriorated by the scale item reduction. Both statistical analysis evidenced usefulness of the AUC ROC reduction method.

Suggested Citation

  • W. W. Koczkodaj & T. Kakiashvili & A. Szymańska & J. Montero-Marin & R. Araya & J. Garcia-Campayo & K. Rutkowski & D. Strzałka, 2017. "How to reduce the number of rating scale items without predictability loss?," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(2), pages 581-593, May.
    • Handle: RePEc:spr:scient:v:111:y:2017:i:2:d:10.1007_s11192-017-2283-4
    DOI: 10.1007/s11192-017-2283-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-017-2283-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-017-2283-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rizopoulos, Dimitris, 2006. "ltm: An R Package for Latent Variable Modeling and Item Response Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 17(i05).
    2. Gualberto Buela-Casal & Izabela Zych, 2012. "What do the scientists think about the impact factor?," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 281-292, August.
    3. Philippe Moigne & Pascal Ragouet, 2012. "Science as instrumentation. The case for psychiatric rating scales," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 329-349, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Waldemar W. Koczkodaj & Mirosław Mazurek & Dominik Strzałka & Alicja Wolny-Dominiak & Marc Woodbury-Smith, 2019. "Electronic Health Record Breaches as Social Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(2), pages 861-871, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ramón A. Feenstra & Emilio Delgado López-Cózar, 2022. "Philosophers’ appraisals of bibliometric indicators and their use in evaluation: from recognition to knee-jerk rejection," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2085-2103, April.
    2. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    3. Nick Bailey, 2020. "Measuring Poverty Efficiently Using Adaptive Deprivation Scales," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 149(3), pages 891-910, June.
    4. Yang Yixin & Lü Xin & Ma Jian & Qiao Han, 2014. "A Robust Factor Analysis Model for Dichotomous Data," Journal of Systems Science and Information, De Gruyter, vol. 2(5), pages 437-450, October.
    5. Arulmani Thiyagarajan & Tyler G. James & Roy Rillera Marzo, 2022. "Psychometric properties of the 21-item Depression, Anxiety, and Stress Scale (DASS-21) among Malaysians during COVID-19: a methodological study," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-8, December.
    6. Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.
    7. Frick, Hannah & Strobl, Carolin & Leisch, Friedrich & Zeileis, Achim, 2012. "Flexible Rasch Mixture Models with Package psychomix," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i07).
    8. Daniel L. Oberski, 2016. "A Review of Latent Variable Modeling With R," Journal of Educational and Behavioral Statistics, , vol. 41(2), pages 226-233, April.
    9. Cervantes, Víctor H., 2017. "DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i05).
    10. Andrew Karl & Randy Eubank & Jelena Milovanovic & Mark Reiser & Dennis Young, 2014. "Using RngStreams for parallel random number generation in C++ and R," Computational Statistics, Springer, vol. 29(5), pages 1301-1320, October.
    11. Sora Lee & Daniel M. Bolt, 2018. "Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 453-475, June.
    12. Gazni, Ali, 2020. "The growing number of patent citations to scientific papers: Changes in the world, nations, and fields," Technology in Society, Elsevier, vol. 62(C).
    13. Jochen Ranger & Kay Brauer, 2022. "On the Generalized S − X 2 –Test of Item Fit: Some Variants, Residuals, and a Graphical Visualization," Journal of Educational and Behavioral Statistics, , vol. 47(2), pages 202-230, April.
    14. Nick Bailey & Anne-Catherine Guio, 2022. "Adaptive Deprivation Scales in a Multi-National Context: The European Child Deprivation Indicators," Child Indicators Research, Springer;The International Society of Child Indicators (ISCI), vol. 15(6), pages 2335-2362, December.
    15. Andrei V. Grinëv & Daria S. Bylieva & Victoria V. Lobatyuk, 2021. "Russian University Teachers’ Perceptions of Scientometrics," Publications, MDPI, vol. 9(2), pages 1-16, May.
    16. Fuchs, Lisa Elena & Orero, Levi & Apondi, Victoria Atieno & Kipkorir, Lang'at, 2021. "How to stop wasting money in international development: Using a structured group selection approach to counter procedural inefficiency," World Development Perspectives, Elsevier, vol. 24(C).
    17. Arnond Sakworawich & Howard Wainer, 2020. "Scoring Tests With Contaminated Response Vectors," Journal of Educational and Behavioral Statistics, , vol. 45(2), pages 209-226, April.
    18. Gustavo Dambiski Gomes de Carvalho & Luis Mauricio Martins de Resende & Joseane Pontes & Hélio Gomes de Carvalho & Leozenir Mendes Betim, 2021. "Innovation and Management in MSMEs: A Literature Review of Highly Cited Papers," SAGE Open, , vol. 11(4), pages 21582440211, October.
    19. Tormod Bøe & Mari Hysing & Jens Christoffer Skogen & Kyrre Breivik, 2016. "The Strengths and Difficulties Questionnaire (SDQ): Factor Structure and Gender Equivalence in Norwegian Adolescents," PLOS ONE, Public Library of Science, vol. 11(5), pages 1-15, May.
    20. Dawit G. Ayele & Temesgen Zewotir & Henry Mwambi, 2014. "Using Rasch Modeling to Re-Evaluate Rapid Malaria Diagnosis Test Analyses," IJERPH, MDPI, vol. 11(7), pages 1-11, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:111:y:2017:i:2:d:10.1007_s11192-017-2283-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.