IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v206y2013i1p595-61010.1007-s10479-012-1218-y.html
   My bibliography  Save this article

Application of robust optimization to automated test assembly

Author

Listed:
  • Bernard Veldkamp

Abstract

In automated test assembly (ATA), 0-1 linear programming (0-1 LP) methods are applied to select questions (items) from an item bank to assemble an optimal test. The objective in this 0-1 LP optimization problem is to assemble a test that measures, in as precise a way as possible, the ability of candidates. Item response theory (IRT) is commonly applied to model the relationship between the responses of candidates and their ability level. Parameters that describe the characteristics of each item, such as difficulty level and the extent to which an item differentiates between more and less able test takers (discrimination) are estimated in the application of the IRT model. Unfortunately, since all parameters in IRT models have to be estimated, they do have a level of uncertainty to them. Some of the other parameters in the test assembly model, such as average response times, have been estimated with uncertainty as well. General 0-1 LP methods do not take this uncertainty into account, and overestimate the predicted level of measurement precision. In this paper, alternative robust optimization methods are applied. It is demonstrated how the Bertsimas and Sim method can be applied to take this uncertainty into account in ATA. The impact of applying this method is illustrated in two numerical examples. Implications are discussed, and some directions for future research are presented. Copyright Springer Science+Business Media, LLC 2013

Suggested Citation

  • Bernard Veldkamp, 2013. "Application of robust optimization to automated test assembly," Annals of Operations Research, Springer, vol. 206(1), pages 595-610, July.
    • Handle: RePEc:spr:annopr:v:206:y:2013:i:1:p:595-610:10.1007/s10479-012-1218-y
    DOI: 10.1007/s10479-012-1218-y
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10479-012-1218-y
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10479-012-1218-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Adema, Jos J. & Boekkooi-Timminga, Ellen & van der Linden, Wim J., 1991. "Achievement test construction using 0-1 linear programming," European Journal of Operational Research, Elsevier, vol. 55(1), pages 103-111, November.
    2. Ronald Armstrong & Dmitry Belov & Alexander Weissman, 2005. "Developing and Assembling the Law School Admission Test," Interfaces, INFORMS, vol. 35(2), pages 140-151, April.
    3. Wim Linden & Ellen Boekkooi-Timminga, 1989. "A maximin model for IRT-based test design with practical constraints," Psychometrika, Springer;The Psychometric Society, vol. 54(2), pages 237-247, June.
    4. T. Theunissen, 1985. "Binary programming and test design," Psychometrika, Springer;The Psychometric Society, vol. 50(4), pages 411-420, December.
    5. A. L. Soyster, 1973. "Technical Note—Convex Programming with Set-Inclusive Constraints and Applications to Inexact Linear Programming," Operations Research, INFORMS, vol. 21(5), pages 1154-1157, October.
    6. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Giada Spaccapanico Proietti & Mariagiulia Matteucci & Stefania Mignani & Bernard P. Veldkamp, 2024. "Chance-Constrained Automated Test Assembly," Journal of Educational and Behavioral Statistics, , vol. 49(1), pages 92-120, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Armstrong, Ronald D. & Kung, Mabel T. & Roussos, Louis A., 2010. "Determining targets for multi-stage adaptive tests using integer programming," European Journal of Operational Research, Elsevier, vol. 205(3), pages 709-718, September.
    2. Edmonds, Jennifer & Armstrong, Ronald, 2009. "A mixed integer programming model for multiple stage adaptive testing," European Journal of Operational Research, Elsevier, vol. 193(2), pages 342-350, March.
    3. Wenqing Chen & Melvyn Sim & Jie Sun & Chung-Piaw Teo, 2010. "From CVaR to Uncertainty Set: Implications in Joint Chance-Constrained Optimization," Operations Research, INFORMS, vol. 58(2), pages 470-485, April.
    4. Zhi Chen & Melvyn Sim & Huan Xu, 2019. "Distributionally Robust Optimization with Infinitely Constrained Ambiguity Sets," Operations Research, INFORMS, vol. 67(5), pages 1328-1344, September.
    5. Stefan Mišković, 2017. "A VNS-LP algorithm for the robust dynamic maximal covering location problem," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 39(4), pages 1011-1033, October.
    6. Minjiao Zhang & Simge Küçükyavuz & Saumya Goel, 2014. "A Branch-and-Cut Method for Dynamic Decision Making Under Joint Chance Constraints," Management Science, INFORMS, vol. 60(5), pages 1317-1333, May.
    7. M. J. Naderi & M. S. Pishvaee, 2017. "Robust bi-objective macroscopic municipal water supply network redesign and rehabilitation," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 31(9), pages 2689-2711, July.
    8. Csilla Horváth & Feray Adigüzel & Hester van Herk, 2013. "Cultural Aspects Of Compulsive Buying In Emerging And Developed Economies: A Cross Cultural Study In Compulsive Buying," Organizations and Markets in Emerging Economies, Faculty of Economics, Vilnius University, vol. 4(2).
    9. Xuejie Bai & Yankui Liu, 2016. "Robust optimization of supply chain network design in fuzzy decision system," Journal of Intelligent Manufacturing, Springer, vol. 27(6), pages 1131-1149, December.
    10. Zhang, Wei & (Ato) Xu, Wangtu, 2017. "Simulation-based robust optimization for the schedule of single-direction bus transit route: The design of experiment," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 106(C), pages 203-230.
    11. Beck, Yasmine & Ljubić, Ivana & Schmidt, Martin, 2023. "A survey on bilevel optimization under uncertainty," European Journal of Operational Research, Elsevier, vol. 311(2), pages 401-426.
    12. Sun, Hao & Yang, Jun & Yang, Chao, 2019. "A robust optimization approach to multi-interval location-inventory and recharging planning for electric vehicles," Omega, Elsevier, vol. 86(C), pages 59-75.
    13. Antonio G. Martín & Manuel Díaz-Madroñero & Josefa Mula, 2020. "Master production schedule using robust optimization approaches in an automobile second-tier supplier," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 28(1), pages 143-166, March.
    14. Wang, Luming & Finn, Adam, 2014. "A psychometric theory that measures up to marketing reality: An adapted Many Faceted IRT model," Australasian marketing journal, Elsevier, vol. 22(2), pages 93-102.
    15. Martijn G. de Jong & Jan-Benedict E. M. Steenkamp & Bernard P. Veldkamp, 2009. "A Model for the Construction of Country-Specific Yet Internationally Comparable Short-Form Marketing Scales," Marketing Science, INFORMS, vol. 28(4), pages 674-689, 07-08.
    16. Mehdi Ansari & Juan S. Borrero & Leonardo Lozano, 2023. "Robust Minimum-Cost Flow Problems Under Multiple Ripple Effect Disruptions," INFORMS Journal on Computing, INFORMS, vol. 35(1), pages 83-103, January.
    17. Rashed Khanjani-Shiraz & Ali Babapour-Azar & Zohreh Hosseini-Noudeh & Panos M. Pardalos, 2022. "Distributionally robust maximum probability shortest path problem," Journal of Combinatorial Optimization, Springer, vol. 43(1), pages 140-167, January.
    18. Khoirunnisa Rohadatul Aisy Muslihin & Endang Rusyaman & Diah Chaerani, 2022. "Conic Duality for Multi-Objective Robust Optimization Problem," Mathematics, MDPI, vol. 10(21), pages 1-22, October.
    19. Walid Ben-Ameur & Adam Ouorou & Guanglei Wang & Mateusz Żotkiewicz, 2018. "Multipolar robust optimization," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 6(4), pages 395-434, December.
    20. Hanks, Robert W. & Weir, Jeffery D. & Lunday, Brian J., 2017. "Robust goal programming using different robustness echelons via norm-based and ellipsoidal uncertainty sets," European Journal of Operational Research, Elsevier, vol. 262(2), pages 636-646.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:206:y:2013:i:1:p:595-610:10.1007/s10479-012-1218-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.