IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v49y2024i1p92-120.html
   My bibliography  Save this article

Chance-Constrained Automated Test Assembly

Author

Listed:
  • Giada Spaccapanico Proietti
  • Mariagiulia Matteucci
  • Stefania Mignani

    (University of Bologna)

  • Bernard P. Veldkamp

    (University of Twente)

Abstract

Classical automated test assembly (ATA) methods assume fixed and known coefficients for the constraints and the objective function. This hypothesis is not true for the estimates of item response theory parameters, which are crucial elements in test assembly classical models. To account for uncertainty in ATA, we propose a chance-constrained version of the maximin ATA model, which allows maximizing the α -quantile of the sampling distribution of the test information function obtained by applying the bootstrap on the item parameter estimation. A heuristic inspired by the simulated annealing optimization technique is implemented to solve the ATA model. The validity of the proposed approach is empirically demonstrated by a simulation study. The applicability is proven by using the real responses to the Trends in International Mathematics and Science Study (TIMSS) 2015 science test.

Suggested Citation

  • Giada Spaccapanico Proietti & Mariagiulia Matteucci & Stefania Mignani & Bernard P. Veldkamp, 2024. "Chance-Constrained Automated Test Assembly," Journal of Educational and Behavioral Statistics, , vol. 49(1), pages 92-120, February.
    • Handle: RePEc:sae:jedbes:v:49:y:2024:i:1:p:92-120
    DOI: 10.3102/10769986231169039
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/10769986231169039
    Download Restriction: no
    File URL: https://libkey.io/10.3102/10769986231169039?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Robert Tsutakawa & Jane Johnson, 1990. "The effect of uncertainty of item parameter estimation on ability estimates," Psychometrika, Springer;The Psychometric Society, vol. 55(2), pages 371-390, June.
    2. Joyce T. Chen, 1973. "Quadratic Programming for Least-Cost Feed Formulations under Probabilistic Protein Constraints," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 55(2), pages 175-183.
    3. Bernard Veldkamp, 2013. "Application of robust optimization to automated test assembly," Annals of Operations Research, Springer, vol. 206(1), pages 595-610, July.
    4. John T. Scott & Chester B. Baker, 1972. "A Practical Way to Select an Optimum Farm Plan Under Risk," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 54(4_Part_1), pages 657-660.
    5. A. Charnes & W. W. Cooper & G. H. Symonds, 1958. "Cost Horizons and Certainty Equivalents: An Approach to Stochastic Programming of Heating Oil," Management Science, INFORMS, vol. 4(3), pages 235-263, April.
    6. A. Charnes & W. W. Cooper, 1963. "Deterministic Equivalents for Optimizing and Satisficing under Chance Constraints," Operations Research, INFORMS, vol. 11(1), pages 18-39, February.
    7. Yongjia Song & James R. Luedtke & Simge Küçükyavuz, 2014. "Chance-Constrained Binary Packing Problems," INFORMS Journal on Computing, INFORMS, vol. 26(4), pages 735-747, November.
    8. A. Charnes & W. W. Cooper, 1959. "Chance-Constrained Programming," Management Science, INFORMS, vol. 6(1), pages 73-79, October.
    9. Kim, C.S. & Schaible, Glenn D. & Segarra, Eduardo, 1990. "The Deterministic Equivalents of Chance-Constrained Programming," Journal of Agricultural Economics Research, United States Department of Agriculture, Economic Research Service, vol. 42(2), pages 1-9.
    10. Jinming Zhang & Minge Xie & Xiaolan Song & Ting Lu, 2011. "Investigating the Impact of Uncertainty About Item Parameters on Ability Estimation," Psychometrika, Springer;The Psychometric Society, vol. 76(1), pages 97-118, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Minjiao Zhang & Simge Küçükyavuz & Saumya Goel, 2014. "A Branch-and-Cut Method for Dynamic Decision Making Under Joint Chance Constraints," Management Science, INFORMS, vol. 60(5), pages 1317-1333, May.
    2. Wu, Desheng (Dash) & Lee, Chi-Guhn, 2010. "Stochastic DEA with ordinal data applied to a multi-attribute pricing problem," European Journal of Operational Research, Elsevier, vol. 207(3), pages 1679-1688, December.
    3. Glover, Fred & Sueyoshi, Toshiyuki, 2009. "Contributions of Professor William W. Cooper in Operations Research and Management Science," European Journal of Operational Research, Elsevier, vol. 197(1), pages 1-16, August.
    4. Xiao Liu & Simge Küçükyavuz, 2018. "A polyhedral study of the static probabilistic lot-sizing problem," Annals of Operations Research, Springer, vol. 261(1), pages 233-254, February.
    5. Maji, Chandi Charan, 1975. "Intertemporal allocation of irrigation water in the Mayurakshi Project (India): an application of deterministic and chance-constrained linear programming," ISU General Staff Papers 197501010800006381, Iowa State University, Department of Economics.
    6. Yongjia Song & Minjiao Zhang, 2015. "Chance‐constrained multi‐terminal network design problems," Naval Research Logistics (NRL), John Wiley & Sons, vol. 62(4), pages 321-334, June.
    7. Xiaodi Bai & Jie Sun & Xiaojin Zheng, 2021. "An Augmented Lagrangian Decomposition Method for Chance-Constrained Optimization Problems," INFORMS Journal on Computing, INFORMS, vol. 33(3), pages 1056-1069, July.
    8. Marla, Lavanya & Rikun, Alexander & Stauffer, Gautier & Pratsini, Eleni, 2020. "Robust modeling and planning: Insights from three industrial applications," Operations Research Perspectives, Elsevier, vol. 7(C).
    9. Aouam, Tarik & Brahimi, Nadjib, 2013. "Integrated production planning and order acceptance under uncertainty: A robust optimization approach," European Journal of Operational Research, Elsevier, vol. 228(3), pages 504-515.
    10. Kim, C.S. & Schaible, Glenn D. & Segarra, Eduardo, 1990. "The Deterministic Equivalents of Chance-Constrained Programming," Journal of Agricultural Economics Research, United States Department of Agriculture, Economic Research Service, vol. 42(2), pages 1-9.
    11. Hang Li & Zhe Zhang & Xianggen Yin & Buhan Zhang, 2020. "Preventive Security-Constrained Optimal Power Flow with Probabilistic Guarantees," Energies, MDPI, vol. 13(9), pages 1-13, May.
    12. Guigues, Vincent & Juditsky, Anatoli & Nemirovski, Arkadi, 2021. "Constant Depth Decision Rules for multistage optimization under uncertainty," European Journal of Operational Research, Elsevier, vol. 295(1), pages 223-232.
    13. Bilsel, R. Ufuk & Ravindran, A., 2011. "A multiobjective chance constrained programming model for supplier selection under uncertainty," Transportation Research Part B: Methodological, Elsevier, vol. 45(8), pages 1284-1300, September.
    14. Yanikoglu, I. & den Hertog, D., 2011. "Safe Approximations of Chance Constraints Using Historical Data," Other publications TiSEM ab77f6f2-248a-42f1-bde1-0, Tilburg University, School of Economics and Management.
    15. L. Jeff Hong & Zhiyuan Huang & Henry Lam, 2021. "Learning-Based Robust Optimization: Procedures and Statistical Guarantees," Management Science, INFORMS, vol. 67(6), pages 3447-3467, June.
    16. Jianqiang Cheng & Richard Li-Yang Chen & Habib N. Najm & Ali Pinar & Cosmin Safta & Jean-Paul Watson, 2018. "Chance-constrained economic dispatch with renewable energy and storage," Computational Optimization and Applications, Springer, vol. 70(2), pages 479-502, June.
    17. Ali Salmasnia & Mostafa Khatami & Reza Kazemzadeh & Seyed Zegordi, 2015. "Bi-objective single machine scheduling problem with stochastic processing times," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(1), pages 275-297, April.
    18. Kliebenstein, James B. & Farthing, Douglas C., 1979. "Impacts of Fuel Pricing and Non-Price Allocation Scenarios On High and Low Risk Crop Producers," 1979 Annual Meeting, July 29-August 1, Pullman, Washington 278181, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    19. Wenqing Chen & Melvyn Sim, 2009. "Goal-Driven Optimization," Operations Research, INFORMS, vol. 57(2), pages 342-357, April.
    20. Yanikoglu, I. & den Hertog, D., 2011. "Safe Approximations of Chance Constraints Using Historical Data," Discussion Paper 2011-137, Tilburg University, Center for Economic Research.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:49:y:2024:i:1:p:92-120. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.