IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v29y2004i2p177-199.html
   My bibliography  Save this article

Evaluation of the CATSIB DIF Procedure in a Pretest Setting

Author

Listed:
  • Ratna Nandakumar
  • Louis Roussos

Abstract

A new procedure, CATSIB, for assessing differential item functioning (DIF) on computerized adaptive tests (CATs) is proposed. CATSIB, a modified SIBTEST procedure, matches test takers on estimated ability and controls for impact-induced Type 1 error inflation by employing a CAT version of the SIBTEST “regression correction.†The performance of CATSIB in terms of detection of DIF in pretest items was evaluated in a simulation study. Simulated test takers were adaptively administered 25 operational items from a pool of 1,000 and were linearly administered 16 pretest items that were evaluated for DIF. Sample size varied from 250 to 500 in each group. Simulated impact levels ranged from a 0- to 1-standard-deviation difference in mean ability levels. The results showed that CATSIB with the regression correction displayed good control over Type 1 error, whereas CATSIB without the regression correction displayed impact-induced Type 1 error inflation. With 500 test takers in each group, power rates were exceptionally high (84% to 99%) for values of DIF at the boundary between moderate and large DIF. For smaller samples of 250 test takers in each group, the corresponding power rates ranged from 47% to 95%. In addition, in all cases, CATSIB was very accurate in estimating the true values of DIF, displaying at most only minor estimation bias.

Suggested Citation

  • Ratna Nandakumar & Louis Roussos, 2004. "Evaluation of the CATSIB DIF Procedure in a Pretest Setting," Journal of Educational and Behavioral Statistics, , vol. 29(2), pages 177-199, June.
    • Handle: RePEc:sae:jedbes:v:29:y:2004:i:2:p:177-199
    DOI: 10.3102/10769986029002177
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986029002177
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986029002177?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    More about this item

    Keywords

    CAT; DIF; item response theory; LSAT;
    All these keywords.

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:29:y:2004:i:2:p:177-199. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.