IDEAS home Printed from https://ideas.repec.org/a/kap/hcarem/v18y2015i3p363-375.html
   My bibliography  Save this article

Applying reinforcement learning techniques to detect hepatocellular carcinoma under limited screening capacity

Author

Listed:
  • Elliot Lee
  • Mariel Lavieri
  • Michael Volk
  • Yongcai Xu

Abstract

We investigate the problem faced by a healthcare system wishing to allocate its constrained screening resources across a population at risk for developing a disease. A patient’s risk of developing the disease depends on his/her biomedical dynamics. However, knowledge of these dynamics must be learned by the system over time. Three classes of reinforcement learning policies are designed to address this problem of simultaneously gathering and utilizing information across multiple patients. We investigate a case study based upon the screening for Hepatocellular Carcinoma (HCC), and optimize each of the three classes of policies using the indifference zone method. A simulation is built to gauge the performance of these policies, and their performance is compared to current practice. We then demonstrate how the benefits of learning-based screening policies differ across various levels of resource scarcity and provide metrics of policy performance. Copyright Springer Science+Business Media New York 2015

Suggested Citation

  • Elliot Lee & Mariel Lavieri & Michael Volk & Yongcai Xu, 2015. "Applying reinforcement learning techniques to detect hepatocellular carcinoma under limited screening capacity," Health Care Management Science, Springer, vol. 18(3), pages 363-375, September.
    • Handle: RePEc:kap:hcarem:v:18:y:2015:i:3:p:363-375
    DOI: 10.1007/s10729-014-9304-0
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10729-014-9304-0
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10729-014-9304-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jagpreet Chhatwal & Oguzhan Alagoz & Elizabeth S. Burnside, 2010. "Optimal Breast Biopsy Decision-Making Based on Mammographic Features and Demographic Factors," Operations Research, INFORMS, vol. 58(6), pages 1577-1591, December.
    2. Turgay Ayer & Oguzhan Alagoz & Natasha K. Stout, 2012. "OR Forum---A POMDP Approach to Personalize Mammography Screening Decisions," Operations Research, INFORMS, vol. 60(5), pages 1019-1034, October.
    3. Aswin Dhamodharan & Ruben Proano, 2012. "Determining the optimal vaccine vial size in developing countries: a Monte Carlo simulation approach," Health Care Management Science, Springer, vol. 15(3), pages 188-196, September.
    4. Lisa M. Maillart & Julie Simmons Ivy & Scott Ransom & Kathleen Diehl, 2008. "Assessing Dynamic Breast Cancer Screening Policies," Operations Research, INFORMS, vol. 56(6), pages 1411-1427, December.
    5. Fatih Safa Erenay & Oguzhan Alagoz & Adnan Said, 2014. "Optimizing Colonoscopy Screening for Colorectal Cancer Prevention and Surveillance," Manufacturing & Service Operations Management, INFORMS, vol. 16(3), pages 381-400, July.
    6. Marion S. Rauner & Walter J. Gutjahr & Kurt Heidenberger & Joachim Wagner & Joseph Pasia, 2010. "Dynamic Policy Modeling for Chronic Diseases: Metaheuristic-Based Identification of Pareto-Optimal Screening Strategies," Operations Research, INFORMS, vol. 58(5), pages 1269-1286, October.
    7. Jingyu Zhang & Brian T. Denton & Hari Balasubramanian & Nilay D. Shah & Brant A. Inman, 2012. "Optimization of Prostate Biopsy Referral Decisions," Manufacturing & Service Operations Management, INFORMS, vol. 14(4), pages 529-547, October.
    8. H. Romero & N. Dellaert & S. Geer & M. Frunt & M. Jansen-Vullers & G. Krekels, 2013. "Admission and capacity planning for the implementation of one-stop-shop in skin cancer treatment using simulation-based optimization," Health Care Management Science, Springer, vol. 16(1), pages 75-86, March.
    9. Mehmet U. S. Ayvaci & Oguzhan Alagoz & Elizabeth S. Burnside, 2012. "The Effect of Budgetary Restrictions on Breast Cancer Diagnostic Decisions," Manufacturing & Service Operations Management, INFORMS, vol. 14(4), pages 600-617, October.
    10. Giovanni Parmigiani & Steven Skates & Marvin Zelen, 2002. "Modeling and Optimization in Early Detection Programs with a Single Exam," Biometrics, The International Biometric Society, vol. 58(1), pages 30-36, March.
    11. Sandra J. Lee & Marvin Zelen, 2008. "Mortality Modeling of Early Detection Programs," Biometrics, The International Biometric Society, vol. 64(2), pages 386-395, June.
    12. Yue Zhang & Martin Puterman, 2013. "Developing an adaptive policy for long-term care capacity planning," Health Care Management Science, Springer, vol. 16(3), pages 271-279, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gong, Jue & Liu, Shan, 2023. "Partially observable collaborative model for optimizing personalized treatment selection," European Journal of Operational Research, Elsevier, vol. 309(3), pages 1409-1419.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert Kraig Helmeczi & Can Kavaklioglu & Mucahit Cevik & Davood Pirayesh Neghab, 2023. "A multi-objective constrained partially observable Markov decision process model for breast cancer screening," Operational Research, Springer, vol. 23(2), pages 1-42, June.
    2. Wang, Fan & Zhang, Shengfan & Henderson, Louise M., 2018. "Adaptive decision-making of breast cancer mammography screening: A heuristic-based regression model," Omega, Elsevier, vol. 76(C), pages 70-84.
    3. M. Reza Skandari & Steven M. Shechter & Nadia Zalunardo, 2015. "Optimal Vascular Access Choice for Patients on Hemodialysis," Manufacturing & Service Operations Management, INFORMS, vol. 17(4), pages 608-619, October.
    4. Mehmet A. Ergun & Ali Hajjar & Oguzhan Alagoz & Murtuza Rampurwala, 2022. "Optimal breast cancer risk reduction policies tailored to personal risk level," Health Care Management Science, Springer, vol. 25(3), pages 363-388, September.
    5. Zlatana Nenova & Jennifer Shang, 2022. "Personalized Chronic Disease Follow‐Up Appointments: Risk‐Stratified Care Through Big Data," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 583-606, February.
    6. Elliot Lee & Mariel S. Lavieri & Michael Volk, 2019. "Optimal Screening for Hepatocellular Carcinoma: A Restless Bandit Model," Service Science, INFORMS, vol. 21(1), pages 198-212, January.
    7. Jonathan E. Helm & Mariel S. Lavieri & Mark P. Van Oyen & Joshua D. Stein & David C. Musch, 2015. "Dynamic Forecasting and Control Algorithms of Glaucoma Progression for Clinician Decision Support," Operations Research, INFORMS, vol. 63(5), pages 979-999, October.
    8. Sait Tunç & Oguzhan Alagoz & Elizabeth S. Burnside, 2022. "A new perspective on breast cancer diagnostic guidelines to reduce overdiagnosis," Production and Operations Management, Production and Operations Management Society, vol. 31(5), pages 2361-2378, May.
    9. Hossein Kamalzadeh & Vishal Ahuja & Michael Hahsler & Michael E. Bowen, 2021. "An Analytics‐Driven Approach for Optimal Individualized Diabetes Screening," Production and Operations Management, Production and Operations Management Society, vol. 30(9), pages 3161-3191, September.
    10. Jing Li & Ming Dong & Yijiong Ren & Kaiqi Yin, 2015. "How patient compliance impacts the recommendations for colorectal cancer screening," Journal of Combinatorial Optimization, Springer, vol. 30(4), pages 920-937, November.
    11. Malek Ebadi & Raha Akhavan-Tabatabaei, 2021. "Personalized Cotesting Policies for Cervical Cancer Screening: A POMDP Approach," Mathematics, MDPI, vol. 9(6), pages 1-20, March.
    12. M. Reza Skandari & Steven M. Shechter, 2021. "Patient-Type Bayes-Adaptive Treatment Plans," Operations Research, INFORMS, vol. 69(2), pages 574-598, March.
    13. Anirudh Tomer & Daan Nieboer & Monique J. Roobol & Ewout W. Steyerberg & Dimitris Rizopoulos, 2019. "Personalized schedules for surveillance of low‐risk prostate cancer patients," Biometrics, The International Biometric Society, vol. 75(1), pages 153-162, March.
    14. Hessam Bavafa & Sergei Savin & Christian Terwiesch, 2021. "Customizing Primary Care Delivery Using E‐Visits," Production and Operations Management, Production and Operations Management Society, vol. 30(11), pages 4306-4327, November.
    15. Li, Y. & Zhu, M. & Klein, R. & Kong, N., 2014. "Using a partially observable Markov chain model to assess colonoscopy screening strategies – A cohort study," European Journal of Operational Research, Elsevier, vol. 238(1), pages 313-326.
    16. Mehmet U. S. Ayvaci & Oguzhan Alagoz & Elizabeth S. Burnside, 2012. "The Effect of Budgetary Restrictions on Breast Cancer Diagnostic Decisions," Manufacturing & Service Operations Management, INFORMS, vol. 14(4), pages 600-617, October.
    17. Li, Weiyu & Denton, Brian T. & Morgan, Todd M., 2023. "Optimizing active surveillance for prostate cancer using partially observable Markov decision processes," European Journal of Operational Research, Elsevier, vol. 305(1), pages 386-399.
    18. Wesley J. Marrero & Mariel S. Lavieri & Jeremy B. Sussman, 2021. "Optimal cholesterol treatment plans and genetic testing strategies for cardiovascular diseases," Health Care Management Science, Springer, vol. 24(1), pages 1-25, March.
    19. Boloori, Alireza & Saghafian, Soroush & Chakkera, Harini A. A. & Cook, Curtiss B., 2017. "Data-Driven Management of Post-transplant Medications: An APOMDP Approach," Working Paper Series rwp17-036, Harvard University, John F. Kennedy School of Government.
    20. Fatih Safa Erenay & Oguzhan Alagoz & Adnan Said, 2014. "Optimizing Colonoscopy Screening for Colorectal Cancer Prevention and Surveillance," Manufacturing & Service Operations Management, INFORMS, vol. 16(3), pages 381-400, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:hcarem:v:18:y:2015:i:3:p:363-375. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.