IDEAS home Printed from https://ideas.repec.org/a/wly/apecpp/v44y2022i1p110-128.html
   My bibliography  Save this article

Mitigating implicit bias in student evaluations: A randomized intervention

Author

Listed:
  • Brandon Genetin
  • Joyce Chen
  • Vladimir Kogan
  • Alan Kalish

Abstract

We conduct a randomized control trial to assess the efficacy of utilizing modified introductory language in student evaluations of instruction to mitigate implicit bias. Students are randomly assigned within courses to three treatment arms and shown so‐called “cheap talk” scripts referencing implicit bias, the high stakes associated with student evaluations, and the combination of the two. We analyze both the impact assignment of the treatment has on completion rates as well as the effect on average instructor rating. Our analysis indicates assignment has statistically significant effects on the likelihood of response for those assigned the combined treatment, though the effects are heterogeneous with respect to both instructor and student race/ethnicity and gender. We further find the high‐stakes treatment leads to higher average scores for racial/ethnic minority instructors with no significant effects from the implicit bias and combined scripts.

Suggested Citation

  • Brandon Genetin & Joyce Chen & Vladimir Kogan & Alan Kalish, 2022. "Mitigating implicit bias in student evaluations: A randomized intervention," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 44(1), pages 110-128, March.
    • Handle: RePEc:wly:apecpp:v:44:y:2022:i:1:p:110-128
    DOI: 10.1002/aepp.13217
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/aepp.13217
    Download Restriction: no
    File URL: https://libkey.io/10.1002/aepp.13217?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Firpo, Sergio & Foguel, Miguel N. & Jales, Hugo, 2020. "Balancing tests in stratified randomized controlled trials: A cautionary note," Economics Letters, Elsevier, vol. 186(C).
    2. Laura O. Taylor & Ronald G. Cummings, 1999. "Unbiased Value Estimates for Environmental Goods: A Cheap Talk Design for the Contingent Valuation Method," American Economic Review, American Economic Association, vol. 89(3), pages 649-665, June.
    3. Friederike Mengel & Jan Sauermann & Ulf Zölitz, 2019. "Gender Bias in Teaching Evaluations," Journal of the European Economic Association, European Economic Association, vol. 17(2), pages 535-566.
    4. Michael A McPherson & R Todd Jewell & Myungsup Kim, 2009. "What Determines Student Evaluation Scores? A Random Effects Analysis of Undergraduate Economics Classes," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 35(1), pages 37-51.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert L. Moore & Hanna Song & James D. Whitney, 2021. "Do Students Discriminate? Exploring Differentials by Race and Sex in Class Enrollments and Student Ratings of Instructors," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 47(1), pages 135-162, January.
    2. Franziska Voelckner, 2006. "An empirical comparison of methods for measuring consumers’ willingness to pay," Marketing Letters, Springer, vol. 17(2), pages 137-149, April.
    3. Fuchsman, Dillon & McGee, Josh B. & Zamarro, Gema, 2023. "Teachers’ willingness to pay for retirement benefits: A national stated preferences experiment," Economics of Education Review, Elsevier, vol. 92(C).
    4. Giles Atkinson & Sian Morse-Jones & Susana Mourato & Allan Provins, 2012. "‘When to Take “No” for an Answer’? Using Entreaties to Reduce Protests in Contingent Valuation Studies," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 51(4), pages 497-523, April.
    5. Melanie Lefevre, 2011. "Willingness-to-pay for Local Milk-based Dairy Product in Senegal," CREPP Working Papers 1108, Centre de Recherche en Economie Publique et de la Population (CREPP) (Research Center on Public and Population Economics) HEC-Management School, University of Liège.
    6. Lorenzo Ductor & Sanjeev Goyal & Anja Prummer, 2018. "Gender & Collaboration," Working Papers 856, Queen Mary University of London, School of Economics and Finance.
    7. John A. List, 2001. "Do Explicit Warnings Eliminate the Hypothetical Bias in Elicitation Procedures? Evidence from Field Auctions for Sportscards," American Economic Review, American Economic Association, vol. 91(5), pages 1498-1507, December.
    8. Ewa Zawojska & Pierre-Alexandre Mahieu & Romain Crastes & Jordan Louviere, 2016. "On a way to overcome strategic overbidding in open-ended stated preference surveys: A recoding approach," Working Papers 2016-34, Faculty of Economic Sciences, University of Warsaw.
    9. Liu, Ruifeng & ,, 2021. "What We Can Learn from the Interactions of Food Traceable Attributes? a Case Study of Fuji Apple in China," 2021 Conference, August 17-31, 2021, Virtual 315916, International Association of Agricultural Economists.
    10. Frode Alfnes & Chengyan Yue & Helen H. Jensen, 2010. "Cognitive dissonance as a means of reducing hypothetical bias," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 37(2), pages 147-163, June.
    11. John A. List & Robert P. Berrens & Alok K. Bohara & Joe Kerkvliet, 2004. "Examining the Role of Social Isolation on Stated Preferences," American Economic Review, American Economic Association, vol. 94(3), pages 741-752, June.
    12. Akinwehinmi, Oluwagbenga & Ogundari, Kolawole & Amos, Taiwo, 2021. "Consumers' Food Control Risk Perception and Preference for Government-Controlled Safety Certification in Emerging Food Markets," 2021 Conference, August 17-31, 2021, Virtual 315312, International Association of Agricultural Economists.
    13. Lehmann, Nico & Sloot, Daniel & Schüle, Christopher & Ardone, Armin & Fichtner, Wolf, 2023. "The motivational drivers behind consumer preferences for regional electricity – Results of a choice experiment in Southern Germany," Energy Economics, Elsevier, vol. 120(C).
    14. Printezis, Iryna & Grebitus, Carola, 2018. "Marketing Channels for Local Food," Ecological Economics, Elsevier, vol. 152(C), pages 161-171.
    15. Sarah Shandera & Jes L Matsick & David R Hunter & Louis Leblond, 2021. "RASE: Modeling cumulative disadvantage due to marginalized group status in academia," PLOS ONE, Public Library of Science, vol. 16(12), pages 1-20, December.
    16. Kevin N. Griffith & Lawrence M. Scheier, 2013. "Did We Get Our Money’s Worth? Bridging Economic and Behavioral Measures of Program Success in Adolescent Drug Prevention," IJERPH, MDPI, vol. 10(11), pages 1-28, November.
    17. Kanchanaroek, Yingluk & Termansen, Mette & Quinn, Claire, 2013. "Property rights regimes in complex fishery management systems: A choice experiment application," Ecological Economics, Elsevier, vol. 93(C), pages 363-373.
    18. Catherine M.H. Keske & Greta Lohman & John B. Loomis, 2013. "Do Respondents Report Willingness-to-Pay on a per Person or per Group Basis? A High Mountain Recreation Example," Tourism Economics, , vol. 19(1), pages 133-145, February.
    19. List John A. & Sinha Paramita & Taylor Michael H., 2006. "Using Choice Experiments to Value Non-Market Goods and Services: Evidence from Field Experiments," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(2), pages 1-39, January.
    20. Blomquist, Glenn C. & Coomes, Paul A. & Jepsen, Christopher & Koford, Brandon C. & Troske, Kenneth R., 2014. "Estimating the social value of higher education: willingness to pay for community and technical colleges," Journal of Benefit-Cost Analysis, Cambridge University Press, vol. 5(1), pages 3-41, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:apecpp:v:44:y:2022:i:1:p:110-128. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)2040-5804 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.