Systematic Influences on Teaching Evaluations : The Case for Caution
AbstractThe evaluation of teaching and learning has become an important activity in tertiary education institutions. Student surveys provide information about student perceptions and judgments of a particular subject. However, as is widely recognised, the appropriate interpretation of this data is problematic. There is a large literature, mainly for the US, on the use and usefulness of student subject evaluations. This literature has highlighted a number of ‘mitigating factors’ such as subject difficulty, discipline area, etc., that should be taken into account in interpreting the results of these questionnaires. In this paper we examine 8 years of QOT responses from an Economics Department in an Australian University which accounted for more than 79,000 student subject enrolments in 565 subjects. The purpose of this analysis is to establish how the information contained in these data can be used to interpret the responses. In particular, we determine to what extent other factors besides the instructor in charge of the subject have an impact on the raw average student evaluation scores. We find that the following characteristics of the students in these classes had an influence on the average QOT score: year level, enrolment size, the quantitative nature of the subject, the country of origin of the students, the proportion that are female, Honours status of the student, the differential in their mark from previous marks, quality of workbook, quality of textbook and the relative QOT score versus other subjects taught at the same time. However, a number of other factors proposed in the literature to be important influences were found not to be. These include the student’s fee paying status, whether they attended a public, private or catholic secondary school, which other faculty within the University they came from, and if the subject was taught in multiple sessions.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by The University of Melbourne in its series Department of Economics - Working Papers Series with number 953.
Length: 33 pages
Date of creation: 2005
Date of revision:
Contact details of provider:
Postal: Department of Economics, The University of Melbourne, 5th Floor, Economics and Commerce Building, Victoria, 3010, Australia
Phone: +61 3 8344 5289
Fax: +61 3 8344 6899
Web page: http://www.economics.unimelb.edu.au
More information through EDIRC
Other versions of this item:
- Martin Davies & Joe Hirschberg & Jenny Lye & Carol Johnston & Ian Mcdonald, 2007. "Systematic Influences On Teaching Evaluations: The Case For Caution ," Australian Economic Papers, Wiley Blackwell, vol. 46(1), pages 18-38, 03.
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Mason, Paul M. & Steagall, Jeffrey W. & Fabritius, Michael M., 1995. "Student evaluations of faculty: A new procedure for using aggregate measures of performance," Economics of Education Review, Elsevier, vol. 14(4), pages 403-416, December.
- White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-38, May.
- Omer Gokcekus, 2000. "How do university students value economics courses? A hedonic approach," Applied Economics Letters, Taylor & Francis Journals, vol. 7(8), pages 493-496.
- William Bosshardt & Michael Watts, 2001. "Comparing Student and Instructor Evaluations of Teaching," The Journal of Economic Education, Taylor & Francis Journals, vol. 32(1), pages 3-17, January.
- L. F. Jameson Boex, 2000. "Attributes of Effective Economics Instructors: An Analysis of Student Evaluations," The Journal of Economic Education, Taylor & Francis Journals, vol. 31(3), pages 211-227, January.
- De Witte, Kristof & Rogge, Nicky, 2009.
"Accounting for exogenous influences in a benevolent performance evaluation of teachers,"
2009/15, Hogeschool-Universiteit Brussel, Faculteit Economie en Management.
- De Witte, K. & Rogge, N., 2009. "Accounting for exogenous influences in a benevolent performance evaluation of teachers," Working Papers 15, Top Institute for Evidence Based Education Research.
- Kristof DE WITTE & Nicky ROGGE, 2009. "Accounting for exogenous influences in a benevolent performance evaluation of teachers," Center for Economic Studies - Discussion papers ces09.13, Katholieke Universiteit Leuven, Centrum voor Economische Studiën.
- Marco p. Tucci & Sandra Fontani & Silvia Ferrini, 2008. "L’ “R-Factor”: un nuovo modo di valutare la ricerca scientifica," Department of Economics University of Siena 527, Department of Economics, University of Siena.
- Bredtmann, Julia & Crede, Carsten J. & Otten, Sebastian, 2013.
"Methods for evaluating educational programs: Does Writing Center Participation affect student achievement?,"
Evaluation and Program Planning,
Elsevier, vol. 36(1), pages 115-123.
- Julia Bredtmann & Carsten J. Crede & Sebastian Otten, 2011. "Methods for Evaluating Educational Programs – Does Writing Center Participation Affect Student Achievement?," Ruhr Economic Papers 0275, Rheinisch-Westfälisches Institut für Wirtschaftsforschung, Ruhr-Universität Bochum, Universität Dortmund, Universität Duisburg-Essen.
- Joe Hirschberg & Jenny Lye & Martin Davies & Carol Johnston, 2011. "Measuring Student Experience: Relationships between Teaching Quality Instruments (TQI) and Course Experience Questionnaire (CEQ)," Department of Economics - Working Papers Series 1134, The University of Melbourne.
- Silvia Ferrini & Marco P. Tucci, 2011. "Evaluating Research Activity:Impact Factor vs. Research Factor," Department of Economics University of Siena 614, Department of Economics, University of Siena.
- De Witte, Kristof & Rogge, Nicky, 2011. "Accounting for exogenous influences in performance evaluations of teachers," Economics of Education Review, Elsevier, vol. 30(4), pages 641-653, August.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Marisa Cerantola).
If references are entirely missing, you can add them using this form.