IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v12y2020i14p5706-d385054.html
   My bibliography  Save this article

Deep Reinforcement Learning for the Management of Software-Defined Networks and Network Function Virtualization in an Edge-IoT Architecture

Author

Listed:
  • Ricardo S. Alonso

    (BISITE Research Group, University of Salamanca, Edificio Multiusos I+D+i, Calle Espejo 2, 37007 Salamanca, Spain)

  • Inés Sittón-Candanedo

    (BISITE Research Group, University of Salamanca, Edificio Multiusos I+D+i, Calle Espejo 2, 37007 Salamanca, Spain)

  • Roberto Casado-Vara

    (BISITE Research Group, University of Salamanca, Edificio Multiusos I+D+i, Calle Espejo 2, 37007 Salamanca, Spain)

  • Javier Prieto

    (BISITE Research Group, University of Salamanca, Edificio Multiusos I+D+i, Calle Espejo 2, 37007 Salamanca, Spain
    AIR Institute, Edificio Parque Científico, Módulo 305, Paseo de Belén 11, Campus Miguel Delibes, 47011 Valladolid, Spain)

  • Juan M. Corchado

    (BISITE Research Group, University of Salamanca, Edificio Multiusos I+D+i, Calle Espejo 2, 37007 Salamanca, Spain
    AIR Institute, Edificio Parque Científico, Módulo 305, Paseo de Belén 11, Campus Miguel Delibes, 47011 Valladolid, Spain
    Department of Electronics, Information and Communication, Faculty of Engineering, Osaka Institute of Technology, 5-16-1 Omiya, Asahi-ku, Osaka 535-8585, Japan
    Pusat Komputeran dan Informatik, Universiti Malaysia Kelantan, Bachok 16300, Malaysia)

Abstract

The Internet of Things (IoT) paradigm allows the interconnection of millions of sensor devices gathering information and forwarding to the Cloud, where data is stored and processed to infer knowledge and perform analysis and predictions. Cloud service providers charge users based on the computing and storage resources used in the Cloud. In this regard, Edge Computing can be used to reduce these costs. In Edge Computing scenarios, data is pre-processed and filtered in network edge before being sent to the Cloud, resulting in shorter response times and providing a certain service level even if the link between IoT devices and Cloud is interrupted. Moreover, there is a growing trend to share physical network resources and costs through Network Function Virtualization (NFV) architectures. In this sense, and related to NFV, Software-Defined Networks (SDNs) are used to reconfigure the network dynamically according to the necessities during time. For this purpose, Machine Learning mechanisms, such as Deep Reinforcement Learning techniques, can be employed to manage virtual data flows in networks. In this work, we propose the evolution of an existing Edge-IoT architecture to a new improved version in which SDN/NFV are used over the Edge-IoT capabilities. The proposed new architecture contemplates the use of Deep Reinforcement Learning techniques for the implementation of the SDN controller.

Suggested Citation

  • Ricardo S. Alonso & Inés Sittón-Candanedo & Roberto Casado-Vara & Javier Prieto & Juan M. Corchado, 2020. "Deep Reinforcement Learning for the Management of Software-Defined Networks and Network Function Virtualization in an Edge-IoT Architecture," Sustainability, MDPI, vol. 12(14), pages 1-23, July.
    • Handle: RePEc:gam:jsusta:v:12:y:2020:i:14:p:5706-:d:385054
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/12/14/5706/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/12/14/5706/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David Silver & Julian Schrittwieser & Karen Simonyan & Ioannis Antonoglou & Aja Huang & Arthur Guez & Thomas Hubert & Lucas Baker & Matthew Lai & Adrian Bolton & Yutian Chen & Timothy Lillicrap & Fan , 2017. "Mastering the game of Go without human knowledge," Nature, Nature, vol. 550(7676), pages 354-359, October.
    2. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    3. Radanliev, Petar & De Roure, David & Nicolescu, Razvan & Huth, Michael & Mantilla Montalvo, Rafael & Cannady, Stacy & Burnap, Peter, 2018. "Future developments in cyber risk assessment for the internet of things," MPRA Paper 92567, University Library of Munich, Germany, revised Sep 2018.
    4. David Silver & Aja Huang & Chris J. Maddison & Arthur Guez & Laurent Sifre & George van den Driessche & Julian Schrittwieser & Ioannis Antonoglou & Veda Panneershelvam & Marc Lanctot & Sander Dieleman, 2016. "Mastering the game of Go with deep neural networks and tree search," Nature, Nature, vol. 529(7587), pages 484-489, January.
    5. Alfonso González-Briones & Fernando De La Prieta & Mohd Saberi Mohamad & Sigeru Omatu & Juan M. Corchado, 2018. "Multi-Agent Systems Applications in Energy Optimization Problems: A State-of-the-Art Review," Energies, MDPI, vol. 11(8), pages 1-28, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. María E. Pérez-Pons & Marta Plaza-Hernández & Ricardo S. Alonso & Javier Parra-Domínguez & Javier Prieto, 2020. "Increasing Profitability and Monitoring Environmental Performance: A Case Study in the Agri-Food Industry through an Edge-IoT Platform," Sustainability, MDPI, vol. 13(1), pages 1-16, December.
    2. Kun Jin & Wei Wang & Xuedong Hua & Wei Zhou, 2020. "Reinforcement Learning for Optimizing Driving Policies on Cruising Taxis Services," Sustainability, MDPI, vol. 12(21), pages 1-19, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Omar Al-Ani & Sanjoy Das, 2022. "Reinforcement Learning: Theory and Applications in HEMS," Energies, MDPI, vol. 15(17), pages 1-37, September.
    2. Zhang, Yihao & Chai, Zhaojie & Lykotrafitis, George, 2021. "Deep reinforcement learning with a particle dynamics environment applied to emergency evacuation of a room with obstacles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 571(C).
    3. Weifan Long & Taixian Hou & Xiaoyi Wei & Shichao Yan & Peng Zhai & Lihua Zhang, 2023. "A Survey on Population-Based Deep Reinforcement Learning," Mathematics, MDPI, vol. 11(10), pages 1-17, May.
    4. Yifeng Guo & Xingyu Fu & Yuyan Shi & Mingwen Liu, 2018. "Robust Log-Optimal Strategy with Reinforcement Learning," Papers 1805.00205, arXiv.org.
    5. Yuchao Dong, 2022. "Randomized Optimal Stopping Problem in Continuous time and Reinforcement Learning Algorithm," Papers 2208.02409, arXiv.org, revised Sep 2023.
    6. Shijun Wang & Baocheng Zhu & Chen Li & Mingzhe Wu & James Zhang & Wei Chu & Yuan Qi, 2020. "Riemannian Proximal Policy Optimization," Computer and Information Science, Canadian Center of Science and Education, vol. 13(3), pages 1-93, August.
    7. Iwao Maeda & David deGraw & Michiharu Kitano & Hiroyasu Matsushima & Hiroki Sakaji & Kiyoshi Izumi & Atsuo Kato, 2020. "Deep Reinforcement Learning in Agent Based Financial Market Simulation," JRFM, MDPI, vol. 13(4), pages 1-17, April.
    8. Li, Wenqing & Ni, Shaoquan, 2022. "Train timetabling with the general learning environment and multi-agent deep reinforcement learning," Transportation Research Part B: Methodological, Elsevier, vol. 157(C), pages 230-251.
    9. Bo Hu & Jiaxi Li & Shuang Li & Jie Yang, 2019. "A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR," Energies, MDPI, vol. 12(19), pages 1-15, September.
    10. Elsisi, Mahmoud & Amer, Mohammed & Dababat, Alya’ & Su, Chun-Lien, 2023. "A comprehensive review of machine learning and IoT solutions for demand side energy management, conservation, and resilient operation," Energy, Elsevier, vol. 281(C).
    11. Lai, Jianfa & Weng, Lin-Chen & Peng, Xiaoling & Fang, Kai-Tai, 2022. "Construction of symmetric orthogonal designs with deep Q-network and orthogonal complementary design," Computational Statistics & Data Analysis, Elsevier, vol. 171(C).
    12. Zechu Li & Xiao-Yang Liu & Jiahao Zheng & Zhaoran Wang & Anwar Walid & Jian Guo, 2021. "FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance," Papers 2111.05188, arXiv.org.
    13. De Moor, Bram J. & Gijsbrechts, Joren & Boute, Robert N., 2022. "Reward shaping to improve the performance of deep reinforcement learning in perishable inventory management," European Journal of Operational Research, Elsevier, vol. 301(2), pages 535-545.
    14. Christopher R. Madan, 2020. "Considerations for Comparing Video Game AI Agents with Humans," Challenges, MDPI, vol. 11(2), pages 1-12, August.
    15. Qu, Xiaobo & Yu, Yang & Zhou, Mofan & Lin, Chin-Teng & Wang, Xiangyu, 2020. "Jointly dampening traffic oscillations and improving energy consumption with electric, connected and automated vehicles: A reinforcement learning based approach," Applied Energy, Elsevier, vol. 257(C).
    16. Matt Taddy, 2018. "The Technological Elements of Artificial Intelligence," NBER Chapters, in: The Economics of Artificial Intelligence: An Agenda, pages 61-87, National Bureau of Economic Research, Inc.
    17. Guan, Xiaoshu & Sun, Huabin & Hou, Rongrong & Xu, Yang & Bao, Yuequan & Li, Hui, 2023. "A deep reinforcement learning method for structural dominant failure modes searching based on self-play strategy," Reliability Engineering and System Safety, Elsevier, vol. 233(C).
    18. Jermain C. Kaminski & Christian Hopp, 2020. "Predicting outcomes in crowdfunding campaigns with textual, visual, and linguistic signals," Small Business Economics, Springer, vol. 55(3), pages 627-649, October.
    19. Yuchen Zhang & Wei Yang, 2022. "Breakthrough invention and problem complexity: Evidence from a quasi‐experiment," Strategic Management Journal, Wiley Blackwell, vol. 43(12), pages 2510-2544, December.
    20. Yassine Chemingui & Adel Gastli & Omar Ellabban, 2020. "Reinforcement Learning-Based School Energy Management System," Energies, MDPI, vol. 13(23), pages 1-21, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:12:y:2020:i:14:p:5706-:d:385054. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.