IDEAS home Printed from https://ideas.repec.org/a/ibn/eltjnl/v18y2025i7p48.html
   My bibliography  Save this article

Evaluating GPT-4 Turbo`s Ability to Design English Reading Test Items for Language Learners

Author

Listed:
  • Sharifah Mofareh Alshehri
  • Mohammed S Alharbi

Abstract

This study evaluates the Generative Pre-trained Transformer (GPT-4 turbo) to design English reading multiple-choice questions (MCQs) for intermediate learners by addressing a deficiency in studies that examine GPT-4 turbo`s capabilities to generate MCQs using various prompt engineering techniques and evaluate their psychometric properties. Utilizing a descriptive quantitative method, a cohort of eight-item writers and 150 preparatory students participated in the study. Both the questionnaire and the generated online test were used to collect data. The findings reveal that zero-shot prompting demonstrates the highest level of agreement compared to few-shot prompting across three of the six aspects- text coherence, question stem quality, and answer options quality. The study finds that although all generated MCQs using few-shot prompting exhibit significantly higher discrimination values than those generated using zero-shot prompting, all MCQs displayed a low level of difficulty across all three prompt engineering techniques (zero-shot prompting, few-shot prompting (two-shot & four-shot). Taken together, this study proposes some profound implications for language assessment developers by illustrating how prompt design influences both the perceived and measured quality of AI-generated items. It also contributes to the literature by providing meaningful insights into using large language models, especially GPT-4 turbo, for AIG.

Suggested Citation

  • Sharifah Mofareh Alshehri & Mohammed S Alharbi, 2025. "Evaluating GPT-4 Turbo`s Ability to Design English Reading Test Items for Language Learners," English Language Teaching, Canadian Center of Science and Education, vol. 18(7), pages 1-48, July.
  • Handle: RePEc:ibn:eltjnl:v:18:y:2025:i:7:p:48
    as

    Download full text from publisher

    File URL: https://ccsenet.org/journal/index.php/elt/article/download/0/0/51869/56435
    Download Restriction: no

    File URL: https://ccsenet.org/journal/index.php/elt/article/view/0/51869
    Download Restriction: no
    ---><---

    More about this item

    JEL classification:

    • R00 - Urban, Rural, Regional, Real Estate, and Transportation Economics - - General - - - General
    • Z0 - Other Special Topics - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ibn:eltjnl:v:18:y:2025:i:7:p:48. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Canadian Center of Science and Education (email available below). General contact details of provider: https://edirc.repec.org/data/cepflch.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.