IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v51y2017i2d10.1007_s11135-016-0432-0.html
   My bibliography  Save this article

An integrated strategy for the analysis of student evaluation of teaching: from descriptive measures to explanatory models

Author

Listed:
  • Michele La Rocca

    (University of Salerno)

  • Maria Lucia Parrella

    (University of Salerno)

  • Ilaria Primerano

    (University of Salerno)

  • Isabella Sulis

    (University of Cagliari)

  • Maria Prosperina Vitale

    (University of Salerno)

Abstract

Over the last decade, the assessment of university teaching quality has assumed a prominent role in the university system with the main purpose of improving the quality of courses offered to students. As a result of this process, a host of studies on the evaluation of university teaching was devoted to the Italian system, covering different topics and considering case studies and methodological issues. Based upon this debate, the contribution aims to present an integrated strategy of analysis which combines both descriptive and model-based methods for the treatment of student evaluation of teaching data. More specifically, the joint use of item response theory and multilevel models allows, on the one hand, to compare courses’ ranking based on different indicators and, on the other hand, to define a model-based approach for building up indicators of overall students’ satisfaction, while adjusting for their characteristics and differences in the compositional variables across courses. The usefulness and the relative merits of the proposed procedure are discussed within a real data set.

Suggested Citation

  • Michele La Rocca & Maria Lucia Parrella & Ilaria Primerano & Isabella Sulis & Maria Prosperina Vitale, 2017. "An integrated strategy for the analysis of student evaluation of teaching: from descriptive measures to explanatory models," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(2), pages 675-691, March.
    • Handle: RePEc:spr:qualqt:v:51:y:2017:i:2:d:10.1007_s11135-016-0432-0
    DOI: 10.1007/s11135-016-0432-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-016-0432-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-016-0432-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Isabella Sulis & Vincenza Capursi, 2013. "Building up adjusted indicators of students’ evaluation of university courses using generalized item response models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 40(1), pages 88-102, January.
    2. Isabella Sulis & Mariano Porcu, 2015. "Assessing Divergences in Mathematics and Reading Achievement in Italian Primary Schools: A Proposal of Adjusted Indicators of School Effectiveness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(2), pages 607-634, June.
    3. Zhang, Zhengzheng & Parker, Richard M. A. & Charlton, Christopher M. J. & Leckie, George & Browne, William J., 2016. "R2MLwiN: A Package to Run MLwiN from within R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 72(i10).
    4. Silvia Bacci & Valeria Caviezel, 2011. "Multilevel IRT models for the university teaching evaluation," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(12), pages 2775-2791, February.
    5. Carla Rampichini & Leonardo Grilli & Alessandra Petrucci, 2004. "Analysis of university course evaluations: from descriptive measures to multilevel models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 13(3), pages 357-373, December.
    6. George Leckie & Harvey Goldstein, 2009. "The limitations of using school league tables to inform school choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 835-851, October.
    7. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marco Guerra & Francesca Bassi & José G. Dias, 2020. "A Multiple-Indicator Latent Growth Mixture Model to Track Courses with Low-Quality Teaching," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 147(2), pages 361-381, January.
    2. Isabella Sulis & Mariano Porcu & Vincenza Capursi, 2019. "On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 142(3), pages 1305-1331, April.
    3. Marta Retamosa & Ángel Millán & Miguel Moital, 2020. "Does the Type of Degree Predict Different Levels of Satisfaction and Loyalty? A Brand Equity Perspective," Corporate Reputation Review, Palgrave Macmillan, vol. 23(2), pages 57-77, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Isabella Sulis & Mariano Porcu & Vincenza Capursi, 2019. "On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 142(3), pages 1305-1331, April.
    2. Sulis, Isabella & Giambona, Francesca & Porcu, Mariano, 2020. "Adjusted indicators of quality and equity for monitoring the education systems over time. Insights on EU15 countries from PISA surveys," Socio-Economic Planning Sciences, Elsevier, vol. 69(C).
    3. Arpino, Bruno & Varriale, Roberta, 2009. "Assessing the quality of institutions’ rankings obtained through multilevel linear regression models," MPRA Paper 19873, University Library of Munich, Germany.
    4. Francesca Giambona & Mariano Porcu & Isabella Sulis, 2017. "Students Mobility: Assessing the Determinants of Attractiveness Across Competing Territorial Areas," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 133(3), pages 1105-1132, September.
    5. Bruno ARPINO & Roberta VARRIALE, 2010. "Assessing The Quality Of Institutions’ Rankings Obtained Through Multilevel Linear Regression Models," Journal of Applied Economic Sciences, Spiru Haret University, Faculty of Financial Management and Accounting Craiova, vol. 5(1(11)_Spr), pages 7-22.
    6. Columbu, Silvia & Porcu, Mariano & Sulis, Isabella, 2021. "University choice and the attractiveness of the study area: Insights on the differences amongst degree programmes in Italy based on generalised mixed-effect models," Socio-Economic Planning Sciences, Elsevier, vol. 74(C).
    7. Isabella Sulis & Mariano Porcu, 2012. "Comparing degree programs from students’ assessments: A LCRA-based adjusted composite indicator," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 21(2), pages 193-209, June.
    8. Pier Ferrari & Laura Pagani & Carlo Fiorio, 2011. "A Two-Step Approach to Analyze Satisfaction Data," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 104(3), pages 545-554, December.
    9. Nicholas T. Longford, 2020. "Performance assessment as an application of causal inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(4), pages 1363-1385, October.
    10. Isabella Sulis & Mariano Porcu, 2015. "Assessing Divergences in Mathematics and Reading Achievement in Italian Primary Schools: A Proposal of Adjusted Indicators of School Effectiveness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(2), pages 607-634, June.
    11. George Leckie, 2022. "A celebration of Harvey Goldstein’s lifetime contributions: Memories of working with Harvey Goldstein on educational research and statistics," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 758-762, July.
    12. Nicholas Tibor Longford, 2016. "Decision Theory Applied to Selecting the Winners, Ranking, and Classification," Journal of Educational and Behavioral Statistics, , vol. 41(4), pages 420-442, August.
    13. Lorraine Dearden, 2010. "Administrative Data and Economic Policy Evaluation," The Economic Record, The Economic Society of Australia, vol. 86(s1), pages 18-21, September.
    14. Marco Guerra & Francesca Bassi & José G. Dias, 2020. "A Multiple-Indicator Latent Growth Mixture Model to Track Courses with Low-Quality Teaching," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 147(2), pages 361-381, January.
    15. Paul Hewson & Keming Yu, 2008. "Quantile regression for binary performance indicators," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 24(5), pages 401-418, September.
    16. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    17. Mutz, Rüdiger & Daniel, Hans-Dieter, 2018. "The bibliometric quotient (BQ), or how to measure a researcher’s performance capacity: A Bayesian Poisson Rasch model," Journal of Informetrics, Elsevier, vol. 12(4), pages 1282-1295.
    18. Magne Mogstad & Joseph P. Romano & Azeem Shaikh & Daniel Wilhelm, 2020. "Inference for Ranks with Applications to Mobility across Neighborhoods and Academic Achievement across Countries," NBER Working Papers 26883, National Bureau of Economic Research, Inc.
    19. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    20. repec:lan:wpaper:991 is not listed on IDEAS
    21. Claudia Herresthal, 2015. "Inferring School Quality from Rankings: The Impact of School Choice," Economics Series Working Papers 747, University of Oxford, Department of Economics.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:51:y:2017:i:2:d:10.1007_s11135-016-0432-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.