IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v134y2025ics0305048325000258.html
   My bibliography  Save this article

IoT-driven dynamic replenishment of fresh produce in the presence of seasonal variations: A deep reinforcement learning approach using reward shaping

Author

Listed:
  • Wang, Zihao
  • Wang, Wenlong
  • Liu, Tianjun
  • Chang, Jasmine
  • Shi, Jim

Abstract

Internet of things (IoT) has been transforming inventory management disruptively by linking and synchronizing inventory products together. It is one of the driving forces for the prevailing innovation of AgriTech. For fresh produce replenishment in the presence of its inherent seasonal variations, not only can IoT devices capture bidirectional seasonal information of lead time and demand, but also detect fresh produce loss and waste (FPLW) caused by deterioration. With the aid of the massive data collected by IoT, we propose a data-driven deep reinforcement learning (DRL) approach using reward shaping, called DQN-SV-RS, to optimize the dynamic replenishment policy for a fresh produce wholesaler, specifically addressing the challenge posed by seasonal variations. Experimental results show that our DQN-SV-SR approach yields significant improvements for fresh produce supply chain (FPSC) inventory management, especially achieving a remarkable reduction in FPLW. As a core innovation in our DQN-SV-SR approach, the introduced reward shaping can significantly mitigate lost sales and inventory holding, thereby lowering the total cost. Furthermore, with numerical experiments based on real business data, our proposed approach is demonstrated with plausible robustness and scalable applicability.

Suggested Citation

  • Wang, Zihao & Wang, Wenlong & Liu, Tianjun & Chang, Jasmine & Shi, Jim, 2025. "IoT-driven dynamic replenishment of fresh produce in the presence of seasonal variations: A deep reinforcement learning approach using reward shaping," Omega, Elsevier, vol. 134(C).
  • Handle: RePEc:eee:jomega:v:134:y:2025:i:c:s0305048325000258
    DOI: 10.1016/j.omega.2025.103299
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0305048325000258
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.omega.2025.103299?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhang, Li-Hao & Li, Tian & Fan, Ti-Jun, 2018. "Radio-frequency identification (RFID) adoption with inventory misplacement under retail competition," European Journal of Operational Research, Elsevier, vol. 270(3), pages 1028-1043.
    2. Qin, Yiyan & Wang, Jianjun & Wei, Caimin, 2014. "Joint pricing and inventory control for fresh produce and foods with quality and physical quantity deteriorating simultaneously," International Journal of Production Economics, Elsevier, vol. 152(C), pages 42-48.
    3. De Moor, Bram J. & Gijsbrechts, Joren & Boute, Robert N., 2022. "Reward shaping to improve the performance of deep reinforcement learning in perishable inventory management," European Journal of Operational Research, Elsevier, vol. 301(2), pages 535-545.
    4. Tufano, Alessandro & Zuidwijk, Rob & Van Dalen, Jan, 2023. "The development of data-driven logistic platforms for barge transportation network under incomplete data," Omega, Elsevier, vol. 114(C).
    5. Qiu, Yuzhuo & Qiao, Jun & Pardalos, Panos M., 2019. "Optimal production, replenishment, delivery, routing and inventory management policies for products with perishable inventory," Omega, Elsevier, vol. 82(C), pages 193-204.
    6. Adrián Macías-López & Leopoldo Eduardo Cárdenas-Barrón & Rodrigo E. Peimbert-García & Buddhadev Mandal, 2021. "An Inventory Model for Perishable Items with Price-, Stock-, and Time-Dependent Demand Rate considering Shelf-Life and Nonlinear Holding Costs," Mathematical Problems in Engineering, Hindawi, vol. 2021, pages 1-36, April.
    7. Cenying Yang & Yihao Feng & Andrew Whinston, 2022. "Dynamic Pricing and Information Disclosure for Fresh Produce: An Artificial Intelligence Approach," Production and Operations Management, Production and Operations Management Society, vol. 31(1), pages 155-171, January.
    8. Jing, Fuying & Chao, Xiangrui, 2021. "A dynamic lot size model with perishable inventory and stockout," Omega, Elsevier, vol. 103(C).
    9. Afshin Oroojlooyjadid & MohammadReza Nazari & Lawrence V. Snyder & Martin Takáč, 2022. "A Deep Q-Network for the Beer Game: Deep Reinforcement Learning for Inventory Optimization," Manufacturing & Service Operations Management, INFORMS, vol. 24(1), pages 285-304, January.
    10. Aichih (Jasmine) Chang & Nesreen El-Rayes & Jim Shi, 2022. "Blockchain Technology for Supply Chain Management: A Comprehensive Review," FinTech, MDPI, vol. 1(2), pages 1-15, June.
    11. Hansen, Ole & Transchel, Sandra & Friedrich, Hanno, 2023. "Replenishment strategies for lost sales inventory systems of perishables under demand and lead time uncertainty," European Journal of Operational Research, Elsevier, vol. 308(2), pages 661-675.
    12. Liu, Hengyu & Zhang, Juliang & Zhou, Chen & Ru, Yihong, 2018. "Optimal purchase and inventory retrieval policies for perishable seasonal agricultural products," Omega, Elsevier, vol. 79(C), pages 133-145.
    13. Lingxiu Dong, 2021. "Toward Resilient Agriculture Value Chains: Challenges and Opportunities," Production and Operations Management, Production and Operations Management Society, vol. 30(3), pages 666-675, March.
    14. Özbilge, Armağan & Hassini, Elkafi & Parlar, Mahmut, 2024. "Optimal pricing and donation policy for fresh goods," European Journal of Operational Research, Elsevier, vol. 312(1), pages 198-210.
    15. Chan, Chi Kin & Zhou, Yan & Wong, Kar Hung, 2018. "A dynamic equilibrium model of the oligopolistic closed-loop supply chain network under uncertain and time-dependent demands," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 118(C), pages 325-354.
    16. Jinzhi Bu & Xiting Gong & Xiuli Chao, 2023. "Asymptotic Optimality of Base-Stock Policies for Perishable Inventory Systems," Management Science, INFORMS, vol. 69(2), pages 846-864, February.
    17. Riezebos, Jan & Zhu, Stuart X., 2020. "Inventory control with seasonality of lead times," Omega, Elsevier, vol. 92(C).
    18. Ruibin Bai & Graham Kendall, 2008. "A Model for Fresh Produce Shelf-Space Allocation and Inventory Management with Freshness-Condition-Dependent Demand," INFORMS Journal on Computing, INFORMS, vol. 20(1), pages 78-85, February.
    19. Haijema, René & Minner, Stefan, 2019. "Improved ordering of perishables: The value of stock-age information," International Journal of Production Economics, Elsevier, vol. 209(C), pages 316-324.
    20. Alvarez, Aldair & Cordeau, Jean-François & Jans, Raf & Munari, Pedro & Morabito, Reinaldo, 2021. "Inventory routing under stochastic supply and demand," Omega, Elsevier, vol. 102(C).
    21. Paolo Priore & Borja Ponte & Rafael Rosillo & David de la Fuente, 2019. "Applying machine learning to the dynamic selection of replenishment policies in fast-changing supply chain environments," International Journal of Production Research, Taylor & Francis Journals, vol. 57(11), pages 3663-3677, June.
    22. Yue Xie & Allen H. Tai & Wai-Ki Ching & Yong-Hong Kuo & Na Song, 2021. "Joint inspection and inventory control for deteriorating items with time-dependent demand and deteriorating rate," Annals of Operations Research, Springer, vol. 300(1), pages 225-265, May.
    23. Boute, Robert N. & Gijsbrechts, Joren & van Jaarsveld, Willem & Vanvuchelen, Nathalie, 2022. "Deep reinforcement learning for inventory control: A roadmap," European Journal of Operational Research, Elsevier, vol. 298(2), pages 401-412.
    24. Carlos Arnade & Daniel Pick & Mark Gehlhar, 2005. "Testing and incorporating seasonal structures into demand models for fruit," Agricultural Economics, International Association of Agricultural Economists, vol. 33(s3), pages 527-532, November.
    25. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    26. Meng Qi & Yuanyuan Shi & Yongzhi Qi & Chenxin Ma & Rong Yuan & Di Wu & Zuo-Jun (Max) Shen, 2023. "A Practical End-to-End Inventory Management Model with Deep Learning," Management Science, INFORMS, vol. 69(2), pages 759-773, February.
    27. Joren Gijsbrechts & Robert N. Boute & Jan A. Van Mieghem & Dennis J. Zhang, 2022. "Can Deep Reinforcement Learning Improve Inventory Management? Performance on Lost Sales, Dual-Sourcing, and Multi-Echelon Problems," Manufacturing & Service Operations Management, INFORMS, vol. 24(3), pages 1349-1368, May.
    28. Disney, Stephen M. & Maltz, Arnold & Wang, Xun & Warburton, Roger D.H., 2016. "Inventory management for stochastic lead times with order crossovers," European Journal of Operational Research, Elsevier, vol. 248(2), pages 473-486.
    29. Alibabaei, Khadijeh & Gaspar, Pedro D. & Assunção, Eduardo & Alirezazadeh, Saeid & Lima, Tânia M., 2022. "Irrigation optimization with a deep reinforcement learning model: Case study on a site in Portugal," Agricultural Water Management, Elsevier, vol. 263(C).
    30. Chang, Jasmine (Aichih) & Katehakis, Michael N. & Shi, Jim (Junmin) & Yan, Zhipeng, 2021. "Blockchain-empowered Newsvendor optimization," International Journal of Production Economics, Elsevier, vol. 238(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Temizöz, Tarkan & Imdahl, Christina & Dijkman, Remco & Lamghari-Idrissi, Douniel & van Jaarsveld, Willem, 2025. "Deep Controlled Learning for Inventory Control," European Journal of Operational Research, Elsevier, vol. 324(1), pages 104-117.
    2. Ralfs, Jana & Pham, Dai T. & Kiesmüller, Gudrun P., 2025. "Optimal outbound shipment policy for an inventory system with advance demand information," European Journal of Operational Research, Elsevier, vol. 324(1), pages 92-103.
    3. Yen, Benjamin P.-C. & Luo, Yu, 2023. "Navigational guidance – A deep learning approach," European Journal of Operational Research, Elsevier, vol. 310(3), pages 1179-1191.
    4. Ding, Jingying & Peng, Zhenkang, 2024. "Heuristics for perishable inventory systems under mixture issuance policies," Omega, Elsevier, vol. 126(C).
    5. Gulecyuz, Suheyl & O’Sullivan, Barry & Armagan Tarim, S., 2025. "A heuristic method for perishable inventory management under non-stationary demand," Omega, Elsevier, vol. 133(C).
    6. Alfonso-Sánchez, Sherly & Solano, Jesús & Correa-Bahnsen, Alejandro & Sendova, Kristina P. & Bravo, Cristián, 2024. "Optimizing credit limit adjustments under adversarial goals using reinforcement learning," European Journal of Operational Research, Elsevier, vol. 315(2), pages 802-817.
    7. Hansen, Ole & Transchel, Sandra & Friedrich, Hanno, 2023. "Replenishment strategies for lost sales inventory systems of perishables under demand and lead time uncertainty," European Journal of Operational Research, Elsevier, vol. 308(2), pages 661-675.
    8. Chen, Yi & Lin, Meiwei & Yu, Zhuo & Sun, Weihong & Fu, Weiguo & He, Liang, 2025. "Enhancing cotton irrigation with distributional actor–critic reinforcement learning," Agricultural Water Management, Elsevier, vol. 307(C).
    9. Alvarez, Aldair & Miranda, Pedro & Rohmer, S.U.K., 2022. "Production routing for perishable products," Omega, Elsevier, vol. 111(C).
    10. Jing, Fuying & Chao, Xiangrui, 2021. "A dynamic lot size model with perishable inventory and stockout," Omega, Elsevier, vol. 103(C).
    11. Po-Han Ko & Yu-Ling Hsueh & Chih-Wen Hsueh, 2022. "A Low-Storage Blockchain Framework Based on Incentive Pricing Strategies," FinTech, MDPI, vol. 1(3), pages 1-26, September.
    12. Pahr, Alexander & Grunow, Martin & Amorim, Pedro, 2025. "Learning from the aggregated optimum: Managing port wine inventory in the face of climate risks," European Journal of Operational Research, Elsevier, vol. 323(2), pages 671-685.
    13. Riezebos, Jan & Zhu, Stuart X., 2020. "Inventory control with seasonality of lead times," Omega, Elsevier, vol. 92(C).
    14. Jian-Jun Wang & Zongli Dai & Wenxuan Zhang & Jim Junmin Shi, 2023. "Operating room scheduling for non-operating room anesthesia with emergency uncertainty," Annals of Operations Research, Springer, vol. 321(1), pages 565-588, February.
    15. Bootaki, Behrang & Zhang, Guoqing, 2024. "A location-production-routing problem for distributed manufacturing platforms: A neural genetic algorithm solution methodology," International Journal of Production Economics, Elsevier, vol. 275(C).
    16. Jasmine Chang, Aichih & Zhou, Fuqin & El-Rayes, Nesreen & Shi, Jim, 2024. "Food transportation and price impacted by diesel price and truck-driver shortage pre-, amid and post pandemic," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 192(C).
    17. Verleijsdonk, Peter & van Jaarsveld, Willem & Kapodistria, Stella, 2024. "Scalable policies for the dynamic traveling multi-maintainer problem with alerts," European Journal of Operational Research, Elsevier, vol. 319(1), pages 121-134.
    18. Ali Akbar Shaikh & Leopoldo Eduardo Cárdenas-Barrón & Amalesh Kumar Manna & Armando Céspedes-Mota & Gerardo Treviño-Garza, 2021. "Two Level Trade Credit Policy Approach in Inventory Model with Expiration Rate and Stock Dependent Demand under Nonzero Inventory and Partial Backlogged Shortages," Sustainability, MDPI, vol. 13(23), pages 1-19, December.
    19. Danyang Gao & Albert S. Chen & Fayyaz Ali Memon, 2024. "A Systematic Review of Methods for Investigating Climate Change Impacts on Water-Energy-Food Nexus," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 38(1), pages 1-43, January.
    20. Dehaybe, Henri & Catanzaro, Daniele & Chevalier, Philippe, 2024. "Deep Reinforcement Learning for inventory optimization with non-stationary uncertain demand," European Journal of Operational Research, Elsevier, vol. 314(2), pages 433-445.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:134:y:2025:i:c:s0305048325000258. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.