IDEAS home Printed from https://ideas.repec.org/a/eee/rensus/v91y2018icp59-69.html
   My bibliography  Save this article

Compression of smart meter big data: A survey

Author

Listed:
  • Wen, Lulu
  • Zhou, Kaile
  • Yang, Shanlin
  • Li, Lanlan

Abstract

In recent years, the smart grid has attracted wide attention from around the world. Large scale data are collected by sensors and measurement devices in a smart grid. Smart meters can record fine-grained information about electricity consumption in near real-time, thus forming the smart meter big data. Smart meter big data has provided new opportunities for electric load forecasting, anomaly detection, and demand side management. However, the high-dimensional and massive smart meter big data not only creates great pressure on data transmission lines, but also incur enormous storage costs on data centres. Therefore, to reduce the transmission pressure and storage overhead, improve data mining efficiency, and thus fulfil the potential of smart meter big data. This study presents a comprehensive study on the compression techniques for smart meter big data. The development of smart grids and the characteristics and application challenges of electric power big data are first introduced, followed by analysis of the characteristics and benefits of smart meter big data. Finally, this study focuses on the potential data compression methods for smart meter big data, and discusses the evaluation methods for smart meter big data compression.

Suggested Citation

  • Wen, Lulu & Zhou, Kaile & Yang, Shanlin & Li, Lanlan, 2018. "Compression of smart meter big data: A survey," Renewable and Sustainable Energy Reviews, Elsevier, vol. 91(C), pages 59-69.
    • Handle: RePEc:eee:rensus:v:91:y:2018:i:c:p:59-69
    DOI: 10.1016/j.rser.2018.03.088
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1364032118301849
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.rser.2018.03.088?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Viegas, Joaquim L. & Vieira, Susana M. & Melício, R. & Mendes, V.M.F. & Sousa, João M.C., 2016. "Classification of new electricity customers based on surveys and smart metering data," Energy, Elsevier, vol. 107(C), pages 804-817.
    2. Siano, Pierluigi, 2014. "Demand response and smart grids—A survey," Renewable and Sustainable Energy Reviews, Elsevier, vol. 30(C), pages 461-478.
    3. Zhou, Kai-le & Yang, Shan-lin & Shen, Chao, 2013. "A review of electric load classification in smart grid environment," Renewable and Sustainable Energy Reviews, Elsevier, vol. 24(C), pages 103-110.
    4. Tang, Chor Foon, 2008. "A re-examination of the relationship between electricity consumption and economic growth in Malaysia," Energy Policy, Elsevier, vol. 36(8), pages 3067-3075, August.
    5. Pingkuo, Liu & Zhongfu, Tan, 2016. "How to develop distributed generation in China: In the context of the reformation of electric power system," Renewable and Sustainable Energy Reviews, Elsevier, vol. 66(C), pages 10-26.
    6. Zhou, Kaile & Yang, Shanlin & Shen, Chao & Ding, Shuai & Sun, Chaoping, 2015. "Energy conservation and emission reduction of China’s electric power industry," Renewable and Sustainable Energy Reviews, Elsevier, vol. 45(C), pages 10-19.
    7. Zhou, Kaile & Yang, Shanlin, 2016. "Understanding household energy consumption behavior: The contribution of energy big data analytics," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 810-819.
    8. Zhou, Kaile & Yang, Shanlin & Chen, Zhiqiang & Ding, Shuai, 2014. "Optimal load distribution model of microgrid in the smart grid environment," Renewable and Sustainable Energy Reviews, Elsevier, vol. 35(C), pages 304-310.
    9. Schleich, Joachim & Faure, Corinne & Klobasa, Marian, 2017. "Persistence of the effects of providing feedback alongside smart metering devices on household electricity demand," Energy Policy, Elsevier, vol. 107(C), pages 225-233.
    10. McLoughlin, Fintan & Duffy, Aidan & Conlon, Michael, 2015. "A clustering approach to domestic electricity load profile characterisation using smart metering data," Applied Energy, Elsevier, vol. 141(C), pages 190-199.
    11. Zhou, Kaile & Fu, Chao & Yang, Shanlin, 2016. "Big data driven smart energy management: From big data to big insights," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 215-225.
    12. Leiva, Javier & Palacios, Alfonso & Aguado, José A., 2016. "Smart metering trends, implications and necessities: A policy review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 55(C), pages 227-233.
    13. Kabalci, Yasin, 2016. "A survey on smart metering and smart grid communication," Renewable and Sustainable Energy Reviews, Elsevier, vol. 57(C), pages 302-318.
    14. Depuru, Soma Shekara Sreenadh Reddy & Wang, Lingfeng & Devabhaktuni, Vijay, 2011. "Smart meters for power grid: Challenges, issues, advantages and status," Renewable and Sustainable Energy Reviews, Elsevier, vol. 15(6), pages 2736-2742, August.
    15. Ahmad, Tanveer, 2017. "Non-technical loss analysis and prevention using smart meters," Renewable and Sustainable Energy Reviews, Elsevier, vol. 72(C), pages 573-589.
    16. Yu, Mengmeng & Hong, Seung Ho, 2016. "Supply–demand balancing for power management in smart grid: A Stackelberg game approach," Applied Energy, Elsevier, vol. 164(C), pages 702-710.
    17. Al-Wakeel, Ali & Wu, Jianzhong & Jenkins, Nick, 2016. "State estimation of medium voltage distribution networks using smart meter measurements," Applied Energy, Elsevier, vol. 184(C), pages 207-218.
    18. Amjady, N. & Keynia, F., 2009. "Short-term load forecasting of power systems by combination of wavelet transform and neuro-evolutionary algorithm," Energy, Elsevier, vol. 34(1), pages 46-57.
    19. Reddy, K.S. & Kumar, Madhusudan & Mallick, T.K. & Sharon, H. & Lokeswaran, S., 2014. "A review of Integration, Control, Communication and Metering (ICCM) of renewable energy based smart grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 38(C), pages 180-192.
    20. Beckel, Christian & Sadamori, Leyna & Staake, Thorsten & Santini, Silvia, 2014. "Revealing household characteristics from smart meter data," Energy, Elsevier, vol. 78(C), pages 397-410.
    21. Vardakas, John S. & Zorba, Nizar & Verikoukis, Christos V., 2015. "Performance evaluation of power demand scheduling scenarios in a smart grid environment," Applied Energy, Elsevier, vol. 142(C), pages 164-178.
    22. Every, Jeremy & Li, Li & Dorrell, David G., 2017. "Leveraging smart meter data for economic optimization of residential photovoltaics under existing tariff structures and incentive schemes," Applied Energy, Elsevier, vol. 201(C), pages 158-173.
    23. Liu, Xiufeng & Nielsen, Per Sieverts, 2016. "A hybrid ICT-solution for smart meter data analytics," Energy, Elsevier, vol. 115(P3), pages 1710-1722.
    24. Lai, T.M. & To, W.M. & Lo, W.C. & Choy, Y.S. & Lam, K.H., 2011. "The causal relationship between electricity consumption and economic growth in a Gaming and Tourism Center: The case of Macao SAR, the People’s Republic of China," Energy, Elsevier, vol. 36(2), pages 1134-1142.
    25. Dyson, Mark E.H. & Borgeson, Samuel D. & Tabone, Michaelangelo D. & Callaway, Duncan S., 2014. "Using smart meter data to estimate demand response potential, with application to solar energy integration," Energy Policy, Elsevier, vol. 73(C), pages 607-619.
    26. Sharma, Susan Sunila, 2010. "The relationship between energy and economic growth: Empirical evidence from 66 countries," Applied Energy, Elsevier, vol. 87(11), pages 3565-3574, November.
    27. Zhou, Kaile & Yang, Shanlin, 2015. "Demand side management in China: The context of China’s power industry reform," Renewable and Sustainable Energy Reviews, Elsevier, vol. 47(C), pages 954-965.
    28. Tu, Chunming & He, Xi & Shuai, Zhikang & Jiang, Fei, 2017. "Big data issues in smart grid – A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 79(C), pages 1099-1107.
    29. Irfan, Muhammad & Iqbal, Jamshed & Iqbal, Adeel & Iqbal, Zahid & Riaz, Raja Ali & Mehmood, Adeel, 2017. "Opportunities and challenges in control of smart grids – Pakistani perspective," Renewable and Sustainable Energy Reviews, Elsevier, vol. 71(C), pages 652-674.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vangelis Marinakis, 2020. "Big Data for Energy Management and Energy-Efficient Buildings," Energies, MDPI, vol. 13(7), pages 1-18, March.
    2. Chen, Yibo & Zhang, Fengyi & Berardi, Umberto, 2020. "Day-ahead prediction of hourly subentry energy consumption in the building sector using pattern recognition algorithms," Energy, Elsevier, vol. 211(C).
    3. Raneen Younis & Andreas Reinhardt, 2020. "A Study on Fundamental Waveform Shapes in Microscopic Electrical Load Signatures," Energies, MDPI, vol. 13(12), pages 1-19, June.
    4. Jia, Kunqi & Guo, Ge & Xiao, Jucheng & Zhou, Huan & Wang, Zhihua & He, Guangyu, 2019. "Data compression approach for the home energy management system," Applied Energy, Elsevier, vol. 247(C), pages 643-656.
    5. Berghout, Tarek & Benbouzid, Mohamed & Muyeen, S.M., 2022. "Machine learning for cybersecurity in smart grids: A comprehensive review-based study on methods, solutions, and prospects," International Journal of Critical Infrastructure Protection, Elsevier, vol. 38(C).
    6. Muhammad Shafiq & Ivar Kiitam & Kimmo Kauhaniemi & Paul Taklaja & Lauri Kütt & Ivo Palu, 2020. "Performance Comparison of PD Data Acquisition Techniques for Condition Monitoring of Medium Voltage Cables," Energies, MDPI, vol. 13(16), pages 1-14, August.
    7. Motlagh, Omid & Berry, Adam & O'Neil, Lachlan, 2019. "Clustering of residential electricity customers using load time series," Applied Energy, Elsevier, vol. 237(C), pages 11-24.
    8. Zheng, Junjie & Lai, Chun Sing & Yuan, Haoliang & Dong, Zhao Yang & Meng, Ke & Lai, Loi Lei, 2020. "Electricity plan recommender system with electrical instruction-based recovery," Energy, Elsevier, vol. 203(C).
    9. Liu, Bo & Hou, Yufan & Luan, Wenpeng & Liu, Zishuai & Chen, Sheng & Yu, Yixin, 2023. "A divide-and-conquer method for compression and reconstruction of smart meter data," Applied Energy, Elsevier, vol. 336(C).
    10. Stracqualursi, Erika & Rosato, Antonello & Di Lorenzo, Gianfranco & Panella, Massimo & Araneo, Rodolfo, 2023. "Systematic review of energy theft practices and autonomous detection through artificial intelligence methods," Renewable and Sustainable Energy Reviews, Elsevier, vol. 184(C).
    11. Yousaf Murtaza Rind & Muhammad Haseeb Raza & Muhammad Zubair & Muhammad Qasim Mehmood & Yehia Massoud, 2023. "Smart Energy Meters for Smart Grids, an Internet of Things Perspective," Energies, MDPI, vol. 16(4), pages 1-35, February.
    12. Maik Plenz & Marc Florian Meyer & Florian Grumm & Daniel Becker & Detlef Schulz & Malcom McCulloch, 2020. "Impact of Lossy Compression Techniques on the Impedance Determination," Energies, MDPI, vol. 13(14), pages 1-12, July.
    13. Luis Hernández-Callejo, 2019. "A Comprehensive Review of Operation and Control, Maintenance and Lifespan Management, Grid Planning and Design, and Metering in Smart Grids," Energies, MDPI, vol. 12(9), pages 1-50, April.
    14. Eunsung Oh & Sung-Yong Son, 2022. "Appropriate Technology-Based AMI Deployment in Multi-Dwelling Units," Energies, MDPI, vol. 15(4), pages 1-12, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhou, Kaile & Yang, Shanlin & Shao, Zhen, 2016. "Energy Internet: The business perspective," Applied Energy, Elsevier, vol. 178(C), pages 212-222.
    2. Chen, Wen & Zhou, Kaile & Yang, Shanlin & Wu, Cheng, 2017. "Data quality of electricity consumption data in a smart grid environment," Renewable and Sustainable Energy Reviews, Elsevier, vol. 75(C), pages 98-105.
    3. Zhou, Kaile & Yang, Changhui & Shen, Jianxin, 2017. "Discovering residential electricity consumption patterns through smart-meter data mining: A case study from China," Utilities Policy, Elsevier, vol. 44(C), pages 73-84.
    4. Jacqueline Nicole Adams & Zsófia Deme Bélafi & Miklós Horváth & János Balázs Kocsis & Tamás Csoknyai, 2021. "How Smart Meter Data Analysis Can Support Understanding the Impact of Occupant Behavior on Building Energy Performance: A Comprehensive Review," Energies, MDPI, vol. 14(9), pages 1-23, April.
    5. Rajabi, Amin & Eskandari, Mohsen & Ghadi, Mojtaba Jabbari & Li, Li & Zhang, Jiangfeng & Siano, Pierluigi, 2020. "A comparative study of clustering techniques for electrical load pattern segmentation," Renewable and Sustainable Energy Reviews, Elsevier, vol. 120(C).
    6. Zhou, Kaile & Yang, Shanlin, 2015. "A framework of service-oriented operation model of China׳s power system," Renewable and Sustainable Energy Reviews, Elsevier, vol. 50(C), pages 719-725.
    7. Tu, Chunming & He, Xi & Shuai, Zhikang & Jiang, Fei, 2017. "Big data issues in smart grid – A review," Renewable and Sustainable Energy Reviews, Elsevier, vol. 79(C), pages 1099-1107.
    8. van de Kaa, G. & Fens, T. & Rezaei, J. & Kaynak, D. & Hatun, Z. & Tsilimeni-Archangelidi, A., 2019. "Realizing smart meter connectivity: Analyzing the competing technologies Power line communication, mobile telephony, and radio frequency using the best worst method," Renewable and Sustainable Energy Reviews, Elsevier, vol. 103(C), pages 320-327.
    9. Pereira, Guillermo Ivan & Specht, Jan Martin & Silva, Patrícia Pereira & Madlener, Reinhard, 2018. "Technology, business model, and market design adaptation toward smart electricity distribution: Insights for policy making," Energy Policy, Elsevier, vol. 121(C), pages 426-440.
    10. Satre-Meloy, Aven & Diakonova, Marina & Grünewald, Philipp, 2020. "Cluster analysis and prediction of residential peak demand profiles using occupant activity data," Applied Energy, Elsevier, vol. 260(C).
    11. Zhou, Kaile & Yang, Shanlin, 2015. "Demand side management in China: The context of China’s power industry reform," Renewable and Sustainable Energy Reviews, Elsevier, vol. 47(C), pages 954-965.
    12. Zhou, Kaile & Fu, Chao & Yang, Shanlin, 2016. "Big data driven smart energy management: From big data to big insights," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 215-225.
    13. Colak, Ilhami & Sagiroglu, Seref & Fulli, Gianluca & Yesilbudak, Mehmet & Covrig, Catalin-Felix, 2016. "A survey on the critical issues in smart grid technologies," Renewable and Sustainable Energy Reviews, Elsevier, vol. 54(C), pages 396-405.
    14. Zhou, Kaile & Yang, Shanlin, 2016. "Understanding household energy consumption behavior: The contribution of energy big data analytics," Renewable and Sustainable Energy Reviews, Elsevier, vol. 56(C), pages 810-819.
    15. Haidar, Ahmed M.A. & Muttaqi, Kashem & Sutanto, Danny, 2015. "Smart Grid and its future perspectives in Australia," Renewable and Sustainable Energy Reviews, Elsevier, vol. 51(C), pages 1375-1389.
    16. Irfan, Muhammad & Iqbal, Jamshed & Iqbal, Adeel & Iqbal, Zahid & Riaz, Raja Ali & Mehmood, Adeel, 2017. "Opportunities and challenges in control of smart grids – Pakistani perspective," Renewable and Sustainable Energy Reviews, Elsevier, vol. 71(C), pages 652-674.
    17. Reddy, K.S. & Kumar, Madhusudan & Mallick, T.K. & Sharon, H. & Lokeswaran, S., 2014. "A review of Integration, Control, Communication and Metering (ICCM) of renewable energy based smart grid," Renewable and Sustainable Energy Reviews, Elsevier, vol. 38(C), pages 180-192.
    18. Villar-Rodriguez, Esther & Del Ser, Javier & Oregi, Izaskun & Bilbao, Miren Nekane & Gil-Lopez, Sergio, 2017. "Detection of non-technical losses in smart meter data based on load curve profiling and time series analysis," Energy, Elsevier, vol. 137(C), pages 118-128.
    19. Emilio Ghiani & Alessandro Serpi & Virginia Pilloni & Giuliana Sias & Marco Simone & Gianluca Marcialis & Giuliano Armano & Paolo Attilio Pegoraro, 2018. "A Multidisciplinary Approach for the Development of Smart Distribution Networks," Energies, MDPI, vol. 11(10), pages 1-29, September.
    20. Rabiee, Abdorreza & Sadeghi, Mohammad & Aghaeic, Jamshid & Heidari, Alireza, 2016. "Optimal operation of microgrids through simultaneous scheduling of electrical vehicles and responsive loads considering wind and PV units uncertainties," Renewable and Sustainable Energy Reviews, Elsevier, vol. 57(C), pages 721-739.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:rensus:v:91:y:2018:i:c:p:59-69. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/600126/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.