IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v11y2021i3p21582440211040121.html
   My bibliography  Save this article

Student Perceptions of Teaching Quality in Five Countries: A Partial Credit Model Approach to Assess Measurement Invariance

Author

Listed:
  • Rikkert M. van der Lans
  • Ridwan Maulana
  • Michelle Helms-Lorenz
  • Carmen-María Fernández-García
  • Seyeoung Chun
  • Thelma de Jager
  • Yulia Irnidayanti
  • Mercedes Inda-Caro
  • Okhwa Lee
  • Thys Coetzee
  • Nurul Fadhilah
  • Meae Jeon
  • Peter Moorer

Abstract

This study examines measurement invariance of student perceptions of teaching quality collected in five countries: Indonesia (n students = 6,331), the Netherlands (n students = 6,738), South Africa (n students = 3,422), South Korea (n students = 6,997) and Spain (n students = 4,676). The administered questionnaire was the My Teacher Questionnaire (MTQ). Student perceived teachers’ teaching quality was estimated using the partial credit model (PCM). Tests for differential item functioning (DIF) were used to assess measurement invariance. Furthermore, if DIF was found, it was explored whether an application of a quasi-international calibration, which estimates country-unique parameters for DIF items, can provide more valid estimates for between-country comparisons. Results indicate the absence of non-uniform DIF, but presence of uniform DIF among most items. This suggests that direct comparisons of raw mean or sum scores between countries is not advisable. Details of the set of invariant items are provided. Furthermore, results suggest that the quasi-international calibration is promising, but also that this approach needs further exploration in the context of student perceptions of teaching quality.

Suggested Citation

  • Rikkert M. van der Lans & Ridwan Maulana & Michelle Helms-Lorenz & Carmen-María Fernández-García & Seyeoung Chun & Thelma de Jager & Yulia Irnidayanti & Mercedes Inda-Caro & Okhwa Lee & Thys Coetze, 2021. "Student Perceptions of Teaching Quality in Five Countries: A Partial Credit Model Approach to Assess Measurement Invariance," SAGE Open, , vol. 11(3), pages 21582440211, August.
    • Handle: RePEc:sae:sagope:v:11:y:2021:i:3:p:21582440211040121
    DOI: 10.1177/21582440211040121
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/21582440211040121
    Download Restriction: no
    File URL: https://libkey.io/10.1177/21582440211040121?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ralph Hippe & Maciej Jakubowski & Luisa De Sousa Lobo Borges de Araujo, 2018. "Regional inequalities in PISA: the case of Italy and Spain," JRC Research Reports JRC109057, Joint Research Centre.
    2. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    3. John Horn, 1965. "A rationale and test for the number of factors in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 30(2), pages 179-185, June.
    4. Mair, Patrick & Hatzinger, Reinhold, 2007. "Extended Rasch Modeling: The eRm Package for the Application of IRT Models in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 20(i09).
    5. Chalmers, R. Philip, 2012. "mirt: A Multidimensional Item Response Theory Package for the R Environment," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i06).
    6. Silvia Bacci & Valeria Caviezel, 2011. "Multilevel IRT models for the university teaching evaluation," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(12), pages 2775-2791, February.
    7. Michael Browne, 1992. "Circumplex models for correlation matrices," Psychometrika, Springer;The Psychometric Society, vol. 57(4), pages 469-497, December.
    8. Tazeen Fasih & Rythia Afkar & Heather Tomlinson, 2018. "Learning for All," World Bank Publications - Reports 29379, The World Bank Group.
    9. Isaac M. Mbiti, 2016. "The Need for Accountability in Education in Developing Countries," Journal of Economic Perspectives, American Economic Association, vol. 30(3), pages 109-132, Summer.
    10. Choi, Seung W. & Gibbons, Laura E. & Crane, Paul K., 2011. "lordif: An R Package for Detecting Differential Item Functioning Using Iterative Hybrid Ordinal Logistic Regression/Item Response Theory and Monte Carlo Simulations," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 39(i08).
    11. Weeks, Jonathan P., 2010. "plink: An R Package for Linking Mixed-Format Tests Using IRT-Based Methods," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 35(i12).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cervantes, Víctor H., 2017. "DFIT: An R Package for Raju's Differential Functioning of Items and Tests Framework," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i05).
    2. Kai Liu & Longfei Zhang & Dongbo Tu & Yan Cai, 2022. "Developing an Item Bank of Computerized Adaptive Testing for Eating Disorders in Chinese University Students," SAGE Open, , vol. 12(4), pages 21582440221, December.
    3. Carla Sabariego & Cornelia Oberhauser & Aleksandra Posarac & Jerome Bickenbach & Nenad Kostanjsek & Somnath Chatterji & Alana Officer & Michaela Coenen & Lay Chhan & Alarcos Cieza, 2015. "Measuring Disability: Comparing the Impact of Two Data Collection Approaches on Disability Rates," IJERPH, MDPI, vol. 12(9), pages 1-23, August.
    4. Brzezińska Justyna, 2018. "Item Response Theory Models in the Measurement Theory with the Use of ltm Package in R," Econometrics. Advances in Applied Data Analysis, Sciendo, vol. 22(1), pages 11-25, March.
    5. César Merino-Soto & Gina Chávez-Ventura & Verónica López-Fernández & Guillermo M. Chans & Filiberto Toledano-Toledano, 2022. "Learning Self-Regulation Questionnaire (SRQ-L): Psychometric and Measurement Invariance Evidence in Peruvian Undergraduate Students," Sustainability, MDPI, vol. 14(18), pages 1-17, September.
    6. Salzberger, Thomas & Koller, Monika, 2013. "Towards a new paradigm of measurement in marketing," Journal of Business Research, Elsevier, vol. 66(9), pages 1307-1317.
    7. Wenling Liu & Ziping Xu & Tianan Yang, 2018. "Health Effects of Air Pollution in China," IJERPH, MDPI, vol. 15(7), pages 1-15, July.
    8. repec:jss:jstsof:36:c01 is not listed on IDEAS
    9. Jeanne A. Teresi & Katja Ocepek-Welikson & John A. Toner & Marjorie Kleinman & Mildred Ramirez & Joseph P. Eimicke & Barry J. Gurland & Albert Siu, 2017. "Methodological Issues in Measuring Subjective Well-Being and Quality-of-Life: Applications to Assessment of Affect in Older, Chronically and Cognitively Impaired, Ethnically Diverse Groups Using the F," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 12(2), pages 251-288, June.
    10. Gerhard Tutz & Gunther Schauberger, 2015. "A Penalty Approach to Differential Item Functioning in Rasch Models," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 21-43, March.
    11. Silvia Bacci & Michela Gnaldi, 2015. "A classification of university courses based on students’ satisfaction: an application of a two-level mixture item response model," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 927-940, May.
    12. Galeotti, Marzio & Rubashkina, Yana & Salini, Silvia & Verdolini, Elena, 2018. "Environmental policy performance and its determinants: Application of a three-level random intercept model," Energy Policy, Elsevier, vol. 114(C), pages 134-144.
    13. Ke-Hai Yuan & Hongyun Liu & Yuting Han, 2021. "Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 345-377, June.
    14. repec:jss:jstsof:35:i12 is not listed on IDEAS
    15. Bacci, Silvia & Fabbricatore, Rosa & Iannario, Maria, 2023. "Multilevel IRT models for the analysis of satisfaction for distance learning during the Covid-19 pandemic," Socio-Economic Planning Sciences, Elsevier, vol. 86(C).
    16. Gerhard Tutz, 2022. "Item Response Thresholds Models: A General Class of Models for Varying Types of Items," Psychometrika, Springer;The Psychometric Society, vol. 87(4), pages 1238-1269, December.
    17. Daphna Harel & Russell J. Steele, 2018. "An Information Matrix Test for the Collapsing of Categories Under the Partial Credit Model," Journal of Educational and Behavioral Statistics, , vol. 43(6), pages 721-750, December.
    18. Laura Maldonado-Murciano & Halley M. Pontes & Mark D. Griffiths & Maite Barrios & Juana Gómez-Benito & Georgina Guilera, 2020. "The Spanish Version of the Internet Gaming Disorder Scale-Short Form (IGDS9-SF): Further Examination Using Item Response Theory," IJERPH, MDPI, vol. 17(19), pages 1-14, September.
    19. Singh, Jagdip, 2004. "Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example," Journal of Business Research, Elsevier, vol. 57(2), pages 184-208, February.
    20. Benjamin D. Schalet & Sangdon Lim & David Cella & Seung W. Choi, 2021. "Linking Scores with Patient-Reported Health Outcome Instruments:A VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS," Psychometrika, Springer;The Psychometric Society, vol. 86(3), pages 717-746, September.
    21. Sati Bozkurt & Gizem B. Ekitli & Christopher L. Thomas & Jerrell C. Cassady, 2017. "Validation of the Turkish Version of the Cognitive Test Anxiety Scale–Revised," SAGE Open, , vol. 7(1), pages 21582440166, January.
    22. Wenjia Wang & Mickaël Guedj & Viviane Bertrand & Julie Foucquier & Elisabeth Jouve & Daniel Commenges & Cécile Proust-Lima & Niall P Murphy & Olivier Blin & Laurent Magy & Daniel Cohen & Shahram Attar, 2017. "A Rasch Analysis of the Charcot-Marie-Tooth Neuropathy Score (CMTNS) in a Cohort of Charcot-Marie-Tooth Type 1A Patients," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-14, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:11:y:2021:i:3:p:21582440211040121. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.