IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2401.15108.html
   My bibliography  Save this paper

Multi-agent Deep Reinforcement Learning for Dynamic Pricing by Fast-charging Electric Vehicle Hubs in ccompetition

Author

Listed:
  • Diwas Paudel
  • Tapas K. Das

Abstract

Fast-charging hubs for electric vehicles will soon become part of the newly built infrastructure for transportation electrification across the world. These hubs are expected to host many DC fast-charging stations and will admit EVs only for charging. Like the gasoline refueling stations, fast-charging hubs in a neighborhood will dynamically vary their prices to compete for the same pool of EV owners. These hubs will interact with the electric power network by making purchase commitments for a significant part of their power needs in the day-ahead (DA) electricity market and meeting the difference from the real-time (RT) market. Hubs may have supplemental battery storage systems (BSS), which they will use for arbitrage. In this paper, we develop a two-step data-driven dynamic pricing methodology for hubs in price competition. We first obtain the DA commitment by solving a stochastic DA commitment model. Thereafter we obtain the hub pricing strategies by modeling the game as a competitive Markov decision process (CMDP) and solving it using a multi-agent deep reinforcement learning (MADRL) approach. We develop a numerical case study for a pricing game between two charging hubs. We solve the case study with our methodology by using combinations of two different DRL algorithms, DQN and SAC, and two different neural networks (NN) architectures, a feed-forward (FF) neural network, and a multi-head attention (MHA) neural network. We construct a measure of collusion (index) using the hub profits. A value of zero for this index indicates no collusion (perfect competition) and a value of one indicates full collusion (monopolistic behavior). Our results show that the collusion index varies approximately between 0.14 and 0.45 depending on the combinations of the algorithms and the architectures chosen by the hubs.

Suggested Citation

  • Diwas Paudel & Tapas K. Das, 2024. "Multi-agent Deep Reinforcement Learning for Dynamic Pricing by Fast-charging Electric Vehicle Hubs in ccompetition," Papers 2401.15108, arXiv.org.
    • Handle: RePEc:arx:papers:2401.15108
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2401.15108
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Arnoud V. den Boer & Janusz M. Meylahn & Maarten Pieter Schinkel, 2022. "Artificial Collusion: Examining Supracompetitive Pricing by Q-learning Algorithms," Tinbergen Institute Discussion Papers 22-067/VII, Tinbergen Institute.
    2. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2020. "Artificial Intelligence, Algorithmic Pricing, and Collusion," American Economic Review, American Economic Association, vol. 110(10), pages 3267-3297, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thomas Loots & Arnoud V. den Boer, 2023. "Data‐driven collusion and competition in a pricing duopoly with multinomial logit demand," Production and Operations Management, Production and Operations Management Society, vol. 32(4), pages 1169-1186, April.
    2. Inkoo Cho & Noah Williams, 2024. "Collusive Outcomes Without Collusion," Papers 2403.07177, arXiv.org.
    3. Ding, Shasha & Sun, Hao & Sun, Panfei & Han, Weibin, 2022. "Dynamic outcome of coopetition duopoly with implicit collusion," Chaos, Solitons & Fractals, Elsevier, vol. 160(C).
    4. Daníelsson, Jón & Macrae, Robert & Uthemann, Andreas, 2022. "Artificial intelligence and systemic risk," Journal of Banking & Finance, Elsevier, vol. 140(C).
    5. Soria, Jorge & Moya, Jorge & Mohazab, Amin, 2023. "Optimal mining in proof-of-work blockchain protocols," Finance Research Letters, Elsevier, vol. 53(C).
    6. Zhijun Chen & pch346 & Chongwoo Choe & Jiajia Cong & Noriaki Matsushima, 2020. "Data-Driven Mergers and Personalization," Monash Economics Working Papers 16-20, Monash University, Department of Economics.
    7. Böheim, René & Hackl, Franz & Hölzl-Leitner, Michael, 2021. "The impact of price adjustment costs on price dispersion in e-commerce," International Journal of Industrial Organization, Elsevier, vol. 77(C).
    8. Emilio Calvano & Giacomo Calzolari & Vincenzo Denicolò & Sergio Pastorello, 2019. "Algorithmic Pricing What Implications for Competition Policy?," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 55(1), pages 155-171, August.
    9. Bingyan Han, 2021. "Understanding algorithmic collusion with experience replay," Papers 2102.09139, arXiv.org, revised Mar 2021.
    10. Marcel Wieting & Geza Sapi, 2021. "Algorithms in the Marketplace: An Empirical Analysis of Automated Pricing in E-Commerce," Working Papers 21-06, NET Institute.
    11. Yiquan Gu & Leonardo Madio & Carlo Reggiani, 2019. "Exclusive Data, Price Manipulation and Market Leadership," CESifo Working Paper Series 7853, CESifo.
    12. Martino Banchio & Giacomo Mantegazza, 2022. "Artificial Intelligence and Spontaneous Collusion," Papers 2202.05946, arXiv.org, revised Sep 2023.
    13. Zengqing Wu & Run Peng & Xu Han & Shuyuan Zheng & Yixin Zhang & Chuan Xiao, 2023. "Smart Agent-Based Modeling: On the Use of Large Language Models in Computer Simulations," Papers 2311.06330, arXiv.org, revised Dec 2023.
    14. Stefano Colombo & Aldo Pignataro, 2022. "Information accuracy and collusion," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 31(3), pages 638-656, August.
    15. Martin, Ian W.R. & Nagel, Stefan, 2022. "Market efficiency in the age of big data," Journal of Financial Economics, Elsevier, vol. 145(1), pages 154-177.
    16. Jens Prüfer & Patricia Prüfer, 2020. "Data science for entrepreneurship research: studying demand dynamics for entrepreneurial skills in the Netherlands," Small Business Economics, Springer, vol. 55(3), pages 651-672, October.
    17. Martin, Simon & Rasch, Alexander, 2022. "Collusion by algorithm: The role of unobserved actions," DICE Discussion Papers 382, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    18. Davide Proserpio & John R. Hauser & Xiao Liu & Tomomichi Amano & Alex Burnap & Tong Guo & Dokyun (DK) Lee & Randall Lewis & Kanishka Misra & Eric Schwarz & Artem Timoshenko & Lilei Xu & Hema Yoganaras, 2020. "Soul and machine (learning)," Marketing Letters, Springer, vol. 31(4), pages 393-404, December.
    19. Calvano, Emilio & Calzolari, Giacomo & Denicoló, Vincenzo & Pastorello, Sergio, 2021. "Algorithmic collusion with imperfect monitoring," International Journal of Industrial Organization, Elsevier, vol. 79(C).
    20. Elliott, M. & Galeotti., A. & Koh., A. & Li, W., 2021. "Market Segmentation Through Information," Janeway Institute Working Papers 2114, Faculty of Economics, University of Cambridge.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2401.15108. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.