IDEAS home Printed from https://ideas.repec.org/a/eee/energy/v232y2021ics0360544221012615.html
   My bibliography  Save this article

Learning spatiotemporal dynamics in wholesale energy markets with dynamic mode decomposition

Author

Listed:
  • Elmore, Clay T.
  • Dowling, Alexander W.

Abstract

Energy markets balance electricity generation (supply) and demand while ensuring non-discriminatory access. Understanding energy market dynamics is essential to improving grid efficiency and optimizing the development of new energy conversion and storage technologies. Accurate energy price forecasts are important for many energy storage technologies to be profitable from price arbitrage. In this paper, we apply Dynamic Mode Decomposition (DMD), a popular spatial-temporal reduced-form modeling technique, to forecast 6587 locational marginal prices in the California Day-Ahead Market (DAM). DMD is a promising equation-free modeling technique in systems with inherent periodic tendencies in time such as financial markets and fluid dynamics. Yet we show, for the first time, that DMD cannot reliably forecast day-ahead energy prices due to the periodicity of the data. Instead, we show Augmented DMD (ADMD) overcomes these limitations and is a fast and accurate price forecaster. We benchmark DMD, ADMD, and backcasting forecasting methods for optimal price arbitrage with an energy storage system. Using ADMD, a market-connected energy storage system can capture up to 92% of allowable revenues in rolling horizon simulations. Moreover, ADMD is up to 1000-times faster than time-series forecasting methods and requires orders of magnitude less data than deep/machine learning techniques.

Suggested Citation

  • Elmore, Clay T. & Dowling, Alexander W., 2021. "Learning spatiotemporal dynamics in wholesale energy markets with dynamic mode decomposition," Energy, Elsevier, vol. 232(C).
    • Handle: RePEc:eee:energy:v:232:y:2021:i:c:s0360544221012615
    DOI: 10.1016/j.energy.2021.121013
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0360544221012615
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.energy.2021.121013?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Brusaferri, Alessandro & Matteucci, Matteo & Portolani, Pietro & Vitali, Andrea, 2019. "Bayesian deep learning based method for probabilistic forecast of day-ahead electricity prices," Applied Energy, Elsevier, vol. 250(C), pages 1158-1175.
    2. Zhang, Jinliang & Tan, Zhongfu & Wei, Yiming, 2020. "An adaptive hybrid model for short term electricity price forecasting," Applied Energy, Elsevier, vol. 258(C).
    3. Bhar, Ramaprasad & Colwell, David B. & Xiao, Yuewen, 2013. "A jump diffusion model for spot electricity prices and market price of risk," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(15), pages 3213-3222.
    4. Dowling, Alexander W. & Kumar, Ranjeet & Zavala, Victor M., 2017. "A multi-scale optimization framework for electricity market participation," Applied Energy, Elsevier, vol. 190(C), pages 147-164.
    5. Panapakidis, Ioannis P. & Dagoumas, Athanasios S., 2016. "Day-ahead electricity price forecasting via the application of artificial neural network based models," Applied Energy, Elsevier, vol. 172(C), pages 132-151.
    6. Marcjasz, Grzegorz & Uniejewski, Bartosz & Weron, Rafał, 2020. "Probabilistic electricity price forecasting with NARX networks: Combine point or probabilistic forecasts?," International Journal of Forecasting, Elsevier, vol. 36(2), pages 466-479.
    7. Afanasyev, Dmitriy O. & Fedorova, Elena A., 2019. "On the impact of outlier filtering on the electricity price forecasting accuracy," Applied Energy, Elsevier, vol. 236(C), pages 196-210.
    8. Alvaro Cartea & Marcelo Figueroa, 2005. "Pricing in Electricity Markets: A Mean Reverting Jump Diffusion Model with Seasonality," Applied Mathematical Finance, Taylor & Francis Journals, vol. 12(4), pages 313-335.
    9. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    10. Otashu, Joannah I. & Baldea, Michael, 2020. "Scheduling chemical processes for frequency regulation," Applied Energy, Elsevier, vol. 260(C).
    11. Tan, Zhongfu & Zhang, Jinliang & Wang, Jianhui & Xu, Jun, 2010. "Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models," Applied Energy, Elsevier, vol. 87(11), pages 3606-3610, November.
    12. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    13. Ziel, Florian & Steinert, Rick, 2018. "Probabilistic mid- and long-term electricity price forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 94(C), pages 251-266.
    14. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    15. Rafal Weron, 2014. "A review of electricity price forecasting: The past, the present and the future," HSC Research Reports HSC/14/02, Hugo Steinhaus Center, Wroclaw University of Technology.
    16. Conejo, Antonio J. & Contreras, Javier & Espinola, Rosa & Plazas, Miguel A., 2005. "Forecasting electricity prices for a day-ahead pool-based electric energy market," International Journal of Forecasting, Elsevier, vol. 21(3), pages 435-462.
    17. O׳Connell, Niamh & Pinson, Pierre & Madsen, Henrik & O׳Malley, Mark, 2014. "Benefits and challenges of electrical demand response: A critical review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 39(C), pages 686-699.
    18. Yang, Wendong & Wang, Jianzhou & Niu, Tong & Du, Pei, 2019. "A hybrid forecasting system based on a dual decomposition strategy and multi-objective optimization for electricity price forecasting," Applied Energy, Elsevier, vol. 235(C), pages 1205-1225.
    19. Keles, Dogan & Scelle, Jonathan & Paraschiv, Florentina & Fichtner, Wolf, 2016. "Extended forecast methods for day-ahead electricity spot prices applying artificial neural networks," Applied Energy, Elsevier, vol. 162(C), pages 218-230.
    20. Jordan Mann & J. Nathan Kutz, 2016. "Dynamic mode decomposition for financial trading strategies," Quantitative Finance, Taylor & Francis Journals, vol. 16(11), pages 1643-1655, November.
    21. Hua, Jia-Chen & Roy, Sukesh & McCauley, Joseph L. & Gunaratne, Gemunu H., 2016. "Using dynamic mode decomposition to extract cyclic behavior in the stock market," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 448(C), pages 172-180.
    22. Lizarraga-Garcia, Enrique & Ghobeity, Amin & Totten, Mark & Mitsos, Alexander, 2013. "Optimal operation of a solar-thermal power plant with energy storage and electricity buy-back from grid," Energy, Elsevier, vol. 51(C), pages 61-70.
    23. Cui, Ling-xiao & Long, Wen, 2016. "Trading strategy based on dynamic mode decomposition: Tested in Chinese stock market," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 461(C), pages 498-508.
    24. Lago, Jesus & De Ridder, Fjo & De Schutter, Bart, 2018. "Forecasting spot electricity prices: Deep learning approaches and empirical comparison of traditional algorithms," Applied Energy, Elsevier, vol. 221(C), pages 386-405.
    25. Kost, Christoph & Flath, Christoph M. & Möst, Dominik, 2013. "Concentrating solar power plant investment and operation decisions under different price and support mechanisms," Energy Policy, Elsevier, vol. 61(C), pages 238-248.
    26. Qiao, Weibiao & Yang, Zhe, 2020. "Forecast the electricity price of U.S. using a wavelet transform-based hybrid model," Energy, Elsevier, vol. 193(C).
    27. Aghaei, Jamshid & Alizadeh, Mohammad-Iman, 2013. "Demand response in smart electricity grids equipped with renewable energy sources: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 18(C), pages 64-72.
    28. Mitra, Sumit & Sun, Lige & Grossmann, Ignacio E., 2013. "Optimal scheduling of industrial combined heat and power plants under time-sensitive electricity prices," Energy, Elsevier, vol. 54(C), pages 194-211.
    29. Florian Ziel & Rick Steinert, 2017. "Probabilistic Mid- and Long-Term Electricity Price Forecasting," Papers 1703.10806, arXiv.org, revised May 2018.
    30. Zhang, Jinliang & Wei, Yiming & Tan, Zhongfu, 2020. "An adaptive hybrid model for short term wind speed forecasting," Energy, Elsevier, vol. 190(C).
    31. Walawalkar, Rahul & Apt, Jay & Mancini, Rick, 2007. "Economics of electric energy storage for energy arbitrage and regulation in New York," Energy Policy, Elsevier, vol. 35(4), pages 2558-2568, April.
    32. Fares, Robert L. & Meyers, Jeremy P. & Webber, Michael E., 2014. "A dynamic model-based estimate of the value of a vanadium redox flow battery for frequency regulation in Texas," Applied Energy, Elsevier, vol. 113(C), pages 189-198.
    33. Sioshansi, Ramteen & Denholm, Paul & Jenkin, Thomas & Weiss, Jurgen, 2009. "Estimating the value of electricity storage in PJM: Arbitrage and some welfare effects," Energy Economics, Elsevier, vol. 31(2), pages 269-277, March.
    34. Luo, Shuman & Weng, Yang, 2019. "A two-stage supervised learning approach for electricity price forecasting by leveraging different data sources," Applied Energy, Elsevier, vol. 242(C), pages 1497-1512.
    35. Dowling, Alexander W. & Zheng, Tian & Zavala, Victor M., 2017. "Economic assessment of concentrated solar power technologies: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 72(C), pages 1019-1032.
    36. Agrawal, Rahul Kumar & Muchahary, Frankle & Tripathi, Madan Mohan, 2019. "Ensemble of relevance vector machines and boosted trees for electricity price forecasting," Applied Energy, Elsevier, vol. 250(C), pages 540-548.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Koo, Bonchan & Chang, Seungjoon & Kwon, Hweeung, 2023. "Digital twin for natural gas infrastructure operation and management via streaming dynamic mode decomposition with control," Energy, Elsevier, vol. 274(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Qiao, Weibiao & Yang, Zhe, 2020. "Forecast the electricity price of U.S. using a wavelet transform-based hybrid model," Energy, Elsevier, vol. 193(C).
    2. Lago, Jesus & Marcjasz, Grzegorz & De Schutter, Bart & Weron, Rafał, 2021. "Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark," Applied Energy, Elsevier, vol. 293(C).
    3. Heydari, Azim & Majidi Nezhad, Meysam & Pirshayan, Elmira & Astiaso Garcia, Davide & Keynia, Farshid & De Santoli, Livio, 2020. "Short-term electricity price and load forecasting in isolated power grids based on composite neural network and gravitational search optimization algorithm," Applied Energy, Elsevier, vol. 277(C).
    4. Marcjasz, Grzegorz & Narajewski, Michał & Weron, Rafał & Ziel, Florian, 2023. "Distributional neural networks for electricity price forecasting," Energy Economics, Elsevier, vol. 125(C).
    5. Lu, Renzhi & Bai, Ruichang & Huang, Yuan & Li, Yuting & Jiang, Junhui & Ding, Yuemin, 2021. "Data-driven real-time price-based demand response for industrial facilities energy management," Applied Energy, Elsevier, vol. 283(C).
    6. Barja-Martinez, Sara & Aragüés-Peñalba, Mònica & Munné-Collado, Íngrid & Lloret-Gallego, Pau & Bullich-Massagué, Eduard & Villafafila-Robles, Roberto, 2021. "Artificial intelligence techniques for enabling Big Data services in distribution networks: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 150(C).
    7. Sheybanivaziri, Samaneh & Le Dréau, Jérôme & Kazmi, Hussain, 2024. "Forecasting price spikes in day-ahead electricity markets: techniques, challenges, and the road ahead," Discussion Papers 2024/1, Norwegian School of Economics, Department of Business and Management Science.
    8. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    9. Li, Wei & Becker, Denis Mike, 2021. "Day-ahead electricity price prediction applying hybrid models of LSTM-based deep learning methods and feature selection algorithms under consideration of market coupling," Energy, Elsevier, vol. 237(C).
    10. Katarzyna Maciejowska & Bartosz Uniejewski & Rafa{l} Weron, 2022. "Forecasting Electricity Prices," Papers 2204.11735, arXiv.org.
    11. Brusaferri, Alessandro & Matteucci, Matteo & Portolani, Pietro & Vitali, Andrea, 2019. "Bayesian deep learning based method for probabilistic forecast of day-ahead electricity prices," Applied Energy, Elsevier, vol. 250(C), pages 1158-1175.
    12. Liu, Luyao & Bai, Feifei & Su, Chenyu & Ma, Cuiping & Yan, Ruifeng & Li, Hailong & Sun, Qie & Wennersten, Ronald, 2022. "Forecasting the occurrence of extreme electricity prices using a multivariate logistic regression model," Energy, Elsevier, vol. 247(C).
    13. Yang, Haolin & Schell, Kristen R., 2021. "Real-time electricity price forecasting of wind farms with deep neural network transfer learning and hybrid datasets," Applied Energy, Elsevier, vol. 299(C).
    14. Shao, Zhen & Zheng, Qingru & Yang, Shanlin & Gao, Fei & Cheng, Manli & Zhang, Qiang & Liu, Chen, 2020. "Modeling and forecasting the electricity clearing price: A novel BELM based pattern classification framework and a comparative analytic study on multi-layer BELM and LSTM," Energy Economics, Elsevier, vol. 86(C).
    15. Tomasz Serafin & Bartosz Uniejewski & Rafał Weron, 2019. "Averaging Predictive Distributions Across Calibration Windows for Day-Ahead Electricity Price Forecasting," Energies, MDPI, vol. 12(13), pages 1-12, July.
    16. Micha{l} Narajewski & Florian Ziel, 2020. "Ensemble Forecasting for Intraday Electricity Prices: Simulating Trajectories," Papers 2005.01365, arXiv.org, revised Aug 2020.
    17. Daniel Manfre Jaimes & Manuel Zamudio López & Hamidreza Zareipour & Mike Quashie, 2023. "A Hybrid Model for Multi-Day-Ahead Electricity Price Forecasting considering Price Spikes," Forecasting, MDPI, vol. 5(3), pages 1-23, July.
    18. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    19. Gabrielli, Paolo & Wüthrich, Moritz & Blume, Steffen & Sansavini, Giovanni, 2022. "Data-driven modeling for long-term electricity price forecasting," Energy, Elsevier, vol. 244(PB).
    20. Olivares, Kin G. & Challu, Cristian & Marcjasz, Grzegorz & Weron, Rafał & Dubrawski, Artur, 2023. "Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx," International Journal of Forecasting, Elsevier, vol. 39(2), pages 884-900.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:energy:v:232:y:2021:i:c:s0360544221012615. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/energy .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.