IDEAS home Printed from https://ideas.repec.org/a/eee/rensus/v56y2016icp1172-1181.html
   My bibliography  Save this article

Application of fuzzy Q-learning for electricity market modeling by considering renewable power penetration

Author

Listed:
  • Salehizadeh, Mohammad Reza
  • Soltaniyan, Salman

Abstract

By increasing renewable resource penetration, the need for developing fast and reliable market modeling approaches in the presence of these resources has gained greater attention. In this paper, fuzzy Q-learning approach is proposed for hour-ahead electricity market modeling in presence of renewable resources. The proposed approach is implemented on IEEE 30-bus test system. The effectiveness of the proposed approach is evaluated and compared with Q-learning approach for both normal and stressful cases. Simulation results indicate that the proposed approach is able to model electricity market for a range of continuous multidimensional renewable power penetration in considerably less iterations compared with Q-learning approach. Moreover, the probability of finding Nash equilibrium is becoming higher by using fuzzy Q-learning approach, while the other indices such as average social welfare, average of locational marginal prices (LMPs), and average standard of deviation of LMPs do not change considerably.

Suggested Citation

  • Salehizadeh, Mohammad Reza & Soltaniyan, Salman, 2016. "Application of fuzzy Q-learning for electricity market modeling by considering renewable power penetration," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 1172-1181.
    • Handle: RePEc:eee:rensus:v:56:y:2016:i:c:p:1172-1181
    DOI: 10.1016/j.rser.2015.12.020
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1364032115014033
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.rser.2015.12.020?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Junjie Sun & Leigh Tesfatsion, 2007. "Dynamic Testing of Wholesale Power Market Designs: An Open-Source Agent-Based Framework," Computational Economics, Springer;Society for Computational Economics, vol. 30(3), pages 291-327, October.
    2. Veena, P. & Indragandhi, V. & Jeyabharath, R. & Subramaniyaswamy, V., 2014. "Review of grid integration schemes for renewable power generation system," Renewable and Sustainable Energy Reviews, Elsevier, vol. 34(C), pages 628-641.
    3. Green, Richard & Vasilakos, Nicholas, 2010. "Market behaviour with large amounts of intermittent generation," Energy Policy, Elsevier, vol. 38(7), pages 3211-3220, July.
    4. Zafeiriou, Eleni & Arabatzis, Garyfallos & Koutroumanidis, Theodoros, 2011. "The fuelwood market in Greece: An empirical approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 15(6), pages 3008-3018, August.
    5. Yekini Suberu, Mohammed & Wazir Mustafa, Mohd & Bashir, Nouruddeen, 2014. "Energy storage systems for renewable energy power sector integration and mitigation of intermittency," Renewable and Sustainable Energy Reviews, Elsevier, vol. 35(C), pages 499-514.
    6. Wiser, Ryan & Bachrach, Devra & Bolinger, Mark & Golove, William, 2004. "Comparing the risk profiles of renewable and natural gas-fired electricity contracts," Renewable and Sustainable Energy Reviews, Elsevier, vol. 8(4), pages 335-363, August.
    7. Oró, Eduard & Depoorter, Victor & Garcia, Albert & Salom, Jaume, 2015. "Energy efficiency and renewable energy integration in data centres. Strategies and modelling review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 42(C), pages 429-445.
    8. Jan Abrell & Friedrich Kunz, 2015. "Integrating Intermittent Renewable Wind Generation - A Stochastic Multi-Market Electricity Model for the European Electricity Market," Networks and Spatial Economics, Springer, vol. 15(1), pages 117-147, March.
    9. Pode, Ramchandra, 2015. "Battery charging stations for home lighting in Mekong region countries," Renewable and Sustainable Energy Reviews, Elsevier, vol. 44(C), pages 543-560.
    10. Tampakis, Stilianos & Τsantopoulos, Georgios & Arabatzis, Garyfallos & Rerras, Ioannis, 2013. "Citizens’ views on various forms of energy and their contribution to the environment," Renewable and Sustainable Energy Reviews, Elsevier, vol. 20(C), pages 473-482.
    11. Saguan, Marcelo & Meeus, Leonardo, 2014. "Impact of the regulatory framework for transmission investments on the cost of renewable energy in the EU," Energy Economics, Elsevier, vol. 43(C), pages 185-194.
    12. Gürkan, Gül & Langestraat, Romeo, 2014. "Modeling and analysis of renewable energy obligations and technology bandings in the UK electricity market," Energy Policy, Elsevier, vol. 70(C), pages 85-95.
    13. Ventosa, Mariano & Baillo, Alvaro & Ramos, Andres & Rivier, Michel, 2005. "Electricity market modeling trends," Energy Policy, Elsevier, vol. 33(7), pages 897-913, May.
    14. Traber, Thure & Kemfert, Claudia, 2011. "Gone with the wind? -- Electricity market prices and incentives to invest in thermal power plants under increasing wind energy supply," Energy Economics, Elsevier, vol. 33(2), pages 249-256, March.
    15. Weidlich, Anke & Veit, Daniel, 2008. "A critical survey of agent-based wholesale electricity market models," Energy Economics, Elsevier, vol. 30(4), pages 1728-1759, July.
    16. Richard Green & Nicholas Vasilakos, 2011. "The Long-term Impact of Wind Power on Electricity Prices and Generating Capacity," Discussion Papers 11-09, Department of Economics, University of Birmingham.
    17. Ayodele, T.R. & Ogunjuyigbe, A.S.O., 2015. "Mitigation of wind power intermittency: Storage technology approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 44(C), pages 447-456.
    18. Aghaei, Jamshid & Alizadeh, Mohammad-Iman, 2013. "Demand response in smart electricity grids equipped with renewable energy sources: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 18(C), pages 64-72.
    19. Hota, Ashish Ranjan & Juvvanapudi, Mahesh & Bajpai, Prabodh, 2014. "Issues and solution approaches in PHEV integration to smart grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 30(C), pages 217-229.
    20. Reddy, K.S. & Kumar, Madhusudan & Mallick, T.K. & Sharon, H. & Lokeswaran, S., 2014. "A review of Integration, Control, Communication and Metering (ICCM) of renewable energy based smart grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 38(C), pages 180-192.
    21. Dallinger, David & Wietschel, Martin, 2012. "Grid integration of intermittent renewable energy sources using price-responsive plug-in electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 16(5), pages 3370-3382.
    22. Richstein, Jörn C. & Chappin, Émile J.L. & de Vries, Laurens J., 2015. "Adjusting the CO2 cap to subsidised RES generation: Can CO2 prices be decoupled from renewable policy?," Applied Energy, Elsevier, vol. 156(C), pages 693-702.
    23. Widén, Joakim & Carpman, Nicole & Castellucci, Valeria & Lingfors, David & Olauson, Jon & Remouit, Flore & Bergkvist, Mikael & Grabbe, Mårten & Waters, Rafael, 2015. "Variability assessment and forecasting of renewables: A review for solar, wind, wave and tidal resources," Renewable and Sustainable Energy Reviews, Elsevier, vol. 44(C), pages 356-375.
    24. Thure Traber & Claudia Kemfert, 2009. "Impacts of the German Support for Renewable Energy on Electricity Prices, Emissions, and Firms," The Energy Journal, International Association for Energy Economics, vol. 0(Number 3), pages 155-178.
    25. Sichao, Kan & Yamamoto, Hiromi & Yamaji, Kenji, 2010. "Evaluation of CO2 free electricity trading market in Japan by multi-agent simulations," Energy Policy, Elsevier, vol. 38(7), pages 3309-3319, July.
    26. Dominguez, R. & Baringo, L. & Conejo, A.J., 2012. "Optimal offering strategy for a concentrating solar power plant," Applied Energy, Elsevier, vol. 98(C), pages 316-325.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Koppelaar, Rembrandt H.E.M. & Keirstead, James & Shah, Nilay & Woods, Jeremy, 2016. "A review of policy analysis purpose and capabilities of electricity system models," Renewable and Sustainable Energy Reviews, Elsevier, vol. 59(C), pages 1531-1544.
    2. Vázquez-Canteli, José R. & Nagy, Zoltán, 2019. "Reinforcement learning for demand response: A review of algorithms and modeling techniques," Applied Energy, Elsevier, vol. 235(C), pages 1072-1089.
    3. Alqahtani, Mohammed & Hu, Mengqi, 2022. "Dynamic energy scheduling and routing of multiple electric vehicles using deep reinforcement learning," Energy, Elsevier, vol. 244(PA).
    4. Mohammad Reza Salehizadeh & Mahdi Amidi Koohbijari & Hassan Nouri & Akın Taşcıkaraoğlu & Ozan Erdinç & João P. S. Catalão, 2019. "Bi-Objective Optimization Model for Optimal Placement of Thyristor-Controlled Series Compensator Devices," Energies, MDPI, vol. 12(13), pages 1-16, July.
    5. Ning Wang & Weisheng Xu & Weihui Shao & Zhiyu Xu, 2019. "A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids," Energies, MDPI, vol. 12(15), pages 1-26, July.
    6. Huiru Zhao & Yuwei Wang & Mingrui Zhao & Qingkun Tan & Sen Guo, 2017. "Day-Ahead Market Modeling for Strategic Wind Power Producers under Robust Market Clearing," Energies, MDPI, vol. 10(7), pages 1-27, July.
    7. Huiru Zhao & Yuwei Wang & Sen Guo & Mingrui Zhao & Chao Zhang, 2016. "Application of a Gradient Descent Continuous Actor-Critic Algorithm for Double-Side Day-Ahead Electricity Market Modeling," Energies, MDPI, vol. 9(9), pages 1-20, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Würzburg, Klaas & Labandeira, Xavier & Linares, Pedro, 2013. "Renewable generation and electricity prices: Taking stock and new evidence for Germany and Austria," Energy Economics, Elsevier, vol. 40(S1), pages 159-171.
    2. Browne, Oliver & Poletti, Stephen & Young, David, 2015. "How does market power affect the impact of large scale wind investment in 'energy only' wholesale electricity markets?," Energy Policy, Elsevier, vol. 87(C), pages 17-27.
    3. Susanne Koschker & Dominik Möst, 2016. "Perfect competition vs. strategic behaviour models to derive electricity prices and the influence of renewables on market power," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 38(3), pages 661-686, July.
    4. Ringler, Philipp & Keles, Dogan & Fichtner, Wolf, 2016. "Agent-based modelling and simulation of smart electricity grids and markets – A literature review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 57(C), pages 205-215.
    5. Balint, T. & Lamperti, F. & Mandel, A. & Napoletano, M. & Roventini, A. & Sapio, A., 2017. "Complexity and the Economics of Climate Change: A Survey and a Look Forward," Ecological Economics, Elsevier, vol. 138(C), pages 252-265.
    6. Jean-Luc Gaffard & Mauro Napoletano, 2012. "Agent-based models and economic policy," Sciences Po publications info:hdl:2441/53r60a8s3ku, Sciences Po.
    7. Bublitz, Andreas & Keles, Dogan & Fichtner, Wolf, 2017. "An analysis of the decline of electricity spot prices in Europe: Who is to blame?," Energy Policy, Elsevier, vol. 107(C), pages 323-336.
    8. Katsanevakis, Markos & Stewart, Rodney A. & Lu, Junwei, 2017. "Aggregated applications and benefits of energy storage systems with application-specific control methods: A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 75(C), pages 719-741.
    9. Wolf-Peter Schill & Claudia Kemfert, 2011. "Modeling Strategic Electricity Storage: The Case of Pumped Hydro Storage in Germany," The Energy Journal, International Association for Energy Economics, vol. 0(Number 3), pages 59-88.
    10. repec:hal:spmain:info:hdl:2441/53r60a8s3kup1vc9l564igg8g is not listed on IDEAS
    11. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    12. Bhagwat, Pradyumna C. & Marcheselli, Anna & Richstein, Jörn C. & Chappin, Emile J.L. & De Vries, Laurens J., 2017. "An analysis of a forward capacity market with long-term contracts," Energy Policy, Elsevier, vol. 111(C), pages 255-267.
    13. Zamani-Dehkordi, Payam & Rakai, Logan & Zareipour, Hamidreza, 2016. "Deciding on the support schemes for upcoming wind farms in competitive electricity markets," Energy, Elsevier, vol. 116(P1), pages 8-19.
    14. repec:hal:spmain:info:hdl:2441/1nlv566svi86iqtetenms15tc4 is not listed on IDEAS
    15. repec:hal:spmain:info:hdl:2441/5qr7f0k4sk8rbq4do5u6v70rm0 is not listed on IDEAS
    16. Ren, Guorui & Liu, Jinfu & Wan, Jie & Guo, Yufeng & Yu, Daren, 2017. "Overview of wind power intermittency: Impacts, measurements, and mitigation solutions," Applied Energy, Elsevier, vol. 204(C), pages 47-65.
    17. repec:hal:wpspec:info:hdl:2441/53r60a8s3kup1vc9l564igg8g is not listed on IDEAS
    18. Poghosyan, Anush & Greetham, Danica Vukadinović & Haben, Stephen & Lee, Tamsin, 2015. "Long term individual load forecast under different electrical vehicles uptake scenarios," Applied Energy, Elsevier, vol. 157(C), pages 699-709.
    19. Farid, Amro M. & Jiang, Bo & Muzhikyan, Aramazd & Youcef-Toumi, Kamal, 2016. "The need for holistic enterprise control assessment methods for the future electricity grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 669-685.
    20. Sébastien Phan & Fabien Roques, 2015. "Is the depressive effect of renewables on power prices contagious? A cross border econometric analysis," Cambridge Working Papers in Economics 1527, Faculty of Economics, University of Cambridge.
    21. Foteinis, S. & Tsoutsos, T., 2017. "Strategies to improve sustainability and offset the initial high capital expenditure of wave energy converters (WECs)," Renewable and Sustainable Energy Reviews, Elsevier, vol. 70(C), pages 775-785.
    22. repec:spo:wpecon:info:hdl:2441/53r60a8s3kup1vc9l564igg8g is not listed on IDEAS
    23. Eid, Cherrelle & Codani, Paul & Perez, Yannick & Reneses, Javier & Hakvoort, Rudi, 2016. "Managing electric flexibility from Distributed Energy Resources: A review of incentives for market design," Renewable and Sustainable Energy Reviews, Elsevier, vol. 64(C), pages 237-247.
    24. Feuerriegel, Stefan & Neumann, Dirk, 2014. "Measuring the financial impact of demand response for electricity retailers," Energy Policy, Elsevier, vol. 65(C), pages 359-368.
    25. De Jonghe, C. & Hobbs, B. F. & Belmans, R., 2011. "Integrating short-term demand response into long-term investment planning," Cambridge Working Papers in Economics 1132, Faculty of Economics, University of Cambridge.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:rensus:v:56:y:2016:i:c:p:1172-1181. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600126/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.