IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v9y2019i1p2158244018822377.html
   My bibliography  Save this article

For a Greater Good: Bias Analysis in Writing Assessment

Author

Listed:
  • Masoumeh Ahmadi Shirazi

Abstract

Threats to construct validity should be reduced to a minimum. If true, sources of bias, namely raters, items, tests as well as gender, age, race, language background, culture, and socio-economic status need to be spotted and removed. This study investigates raters’ experience, language background, and the choice of essay prompt as potential sources of biases. Eight raters, four native English speakers and four Persian L1 speakers of English as a Foreign Language (EFL), scored 40 essays on one general and one field-specific topic. The raters assessed these essays based on Test of English as a Foreign Language (TOEFL) holistic and International English Language Testing System (IELTS) analytic band scores. Multifaceted Rasch Measurement (MFRM) was run to find extant biases. In spite of not finding statistically significant biases, several interesting results emerged illustrating the influence of construct-irrelevant factors such as raters’ experience, L1, and educational background. Further research is warranted to investigate these factors as potential sources of rater bias.

Suggested Citation

  • Masoumeh Ahmadi Shirazi, 2019. "For a Greater Good: Bias Analysis in Writing Assessment," SAGE Open, , vol. 9(1), pages 21582440188, January.
    • Handle: RePEc:sae:sagope:v:9:y:2019:i:1:p:2158244018822377
    DOI: 10.1177/2158244018822377
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2158244018822377
    Download Restriction: no
    File URL: https://libkey.io/10.1177/2158244018822377?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Zwick, Rebecca & Brown, Terran & Sklar, Jeffrey C., 2004. "California and the SAT: A Reanalysis of University of California Admissions Data," University of California at Berkeley, Center for Studies in Higher Education qt9hf3t044, Center for Studies in Higher Education, UC Berkeley.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dur, Robert & Glazer, Amihai, 2008. "Subsidizing Enjoyable Education," Labour Economics, Elsevier, vol. 15(5), pages 1023-1039, October.
    2. Chatman, Steve, 2007. "Institutional Versus Academic Discipline Measures of Student Experience: A Matter of Relative Validity," University of California at Berkeley, Center for Studies in Higher Education qt81f4h1nn, Center for Studies in Higher Education, UC Berkeley.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:9:y:2019:i:1:p:2158244018822377. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.