IDEAS home Printed from https://ideas.repec.org/p/use/tkiwps/1416.html
   My bibliography  Save this paper

Measuring Teaching Quality in Higher Education : Assessing the Problem of Selection Bias in Course Evaluations

Author

Listed:
  • Anna Salomons
  • Maarten Goos

Abstract

Student evaluations of teaching are widely used to measure teaching quality and compare it across different courses, teachers, departments and institutions: as such, they are of increasing importance for teacher promotion decisions as well as student course selection. However, the response on course evaluations is rarely perfect, rendering such uses unwarranted if students who participate in the evaluation are not randomly selected: this paper is the first to investigate this issue. We quantify the direction and size of selection on both observable and unobservable characteristics for a large European university where course evaluation response rates differ across courses. Our results suggest course evaluations are upward biased, and that this bias mostly derives from selection on characteristics unlikely to be observed by the typical university administrator. Correcting for selection bias has sizable effects on both scores in any given course and the evaluation-based ranking of different courses.

Suggested Citation

  • Anna Salomons & Maarten Goos, 2014. "Measuring Teaching Quality in Higher Education : Assessing the Problem of Selection Bias in Course Evaluations," Working Papers 14-16, Utrecht School of Economics.
  • Handle: RePEc:use:tkiwps:1416
    as

    Download full text from publisher

    File URL: https://dspace.library.uu.nl/bitstream/handle/1874/306265/16_2.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. repec:ags:stataj:117582 is not listed on IDEAS
    2. Braga, Michela & Paccagnella, Marco & Pellizzari, Michele, 2014. "Evaluating students’ evaluations of professors," Economics of Education Review, Elsevier, vol. 41(C), pages 71-88.
    3. Giuseppe De Luca & Valeria Perotti, 2011. "Estimation of ordered response models with sample selection," Stata Journal, StataCorp LP, vol. 11(2), pages 213-239, June.
    4. Michael A. McPherson, 2006. "Determinants of How Students Evaluate Teachers," The Journal of Economic Education, Taylor & Francis Journals, vol. 37(1), pages 3-20, January.
    5. Heckman, James J, 1978. "Dummy Endogenous Variables in a Simultaneous Equation System," Econometrica, Econometric Society, vol. 46(4), pages 931-959, July.
    6. Florian Hoffmann & Philip Oreopoulos, 2009. "Professor Qualities and Student Achievement," The Review of Economics and Statistics, MIT Press, vol. 91(1), pages 83-92, February.
    7. Krautmann, Anthony C. & Sander, William, 1999. "Grades and student evaluations of teachers," Economics of Education Review, Elsevier, vol. 18(1), pages 59-63, February.
    8. Rosemary J. Avery & W. Keith Bryant & Alan Mathios & Hyojin Kang & Duncan Bell, 2006. "Electronic Course Evaluations: Does an Online Delivery System Influence Student Evaluations?," The Journal of Economic Education, Taylor & Francis Journals, vol. 37(1), pages 21-37, January.
    9. Scott E. Carrell & James E. West, 2010. "Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors," Journal of Political Economy, University of Chicago Press, vol. 118(3), pages 409-432, June.
    10. Imberman, Scott A. & Lovenheim, Michael F., 2016. "Does the market value value-added? Evidence from housing prices after a public release of school and teacher value-added," Journal of Urban Economics, Elsevier, vol. 91(C), pages 104-121.
    11. Sartori, Anne E., 2003. "An Estimator for Some Binary-Outcome Selection Models Without Exclusion Restrictions," Political Analysis, Cambridge University Press, vol. 11(02), pages 111-138, March.
    12. Alfonso Miranda & Sophia Rabe-Hesketh, 2006. "Maximum likelihood estimation of endogenous switching and sample selection models for binary, ordinal, and count variables," Stata Journal, StataCorp LP, vol. 6(3), pages 285-308, September.
    13. Van de Ven, Wynand P. M. M. & Van Praag, Bernard M. S., 1981. "The demand for deductibles in private health insurance : A probit model with sample selection," Journal of Econometrics, Elsevier, vol. 17(2), pages 229-252, November.
    14. William E. Becker & William Bosshardt & Michael Watts, 2012. "How Departments of Economics Evaluate Teaching," The Journal of Economic Education, Taylor & Francis Journals, vol. 43(3), pages 325-333, July.
    15. Beleche, Trinidad & Fairris, David & Marks, Mindy, 2012. "Do course evaluations truly reflect student learning? Evidence from an objectively graded post-test," Economics of Education Review, Elsevier, vol. 31(5), pages 709-719.
    16. Cunha, Jesse M. & Miller, Trey, 2014. "Measuring value-added in higher education: Possibilities and limitations in the use of administrative data," Economics of Education Review, Elsevier, vol. 42(C), pages 64-77.
    17. Michael A McPherson & R Todd Jewell & Myungsup Kim, 2009. "What Determines Student Evaluation Scores? A Random Effects Analysis of Undergraduate Economics Classes," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 35(1), pages 37-51.
    18. Heckman, James J, 1974. "Shadow Prices, Market Wages, and Labor Supply," Econometrica, Econometric Society, vol. 42(4), pages 679-694, July.
    19. Paul Isely & Harinder Singh, 2005. "Do Higher Grades Lead to Favorable Student Evaluations?," The Journal of Economic Education, Taylor & Francis Journals, vol. 36(1), pages 29-42, January.
    20. Ewing, Andrew M., 2012. "Estimating the impact of relative expected grade on student evaluations of teachers," Economics of Education Review, Elsevier, vol. 31(1), pages 141-154.
    21. Bruce A. Weinberg & Masanori Hashimoto & Belton M. Fleisher, 2009. "Evaluating Teaching in Higher Education," The Journal of Economic Education, Taylor & Francis Journals, vol. 40(3), pages 227-261, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:spr:reihed:v:58:y:2017:i:4:d:10.1007_s11162-016-9429-8 is not listed on IDEAS

    More about this item

    Keywords

    Educational economics; Student evaluations of teaching; Education quality; Sample selection;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:use:tkiwps:1416. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Marina Muilwijk). General contact details of provider: http://edirc.repec.org/data/eiruunl.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.