How effective are synthetic attack models to detect real-world energy theft?

Advanced Metering Infrastructure plays a significant role in smart grid systems by enabling efficient two-way communication between energy suppliers and consumers. Despite its benefits, AMI faces numerous security challenges, including vulnerabilities to cyberattacks and non-technical losses, potentially leading to substantial revenue losses. Data-driven approaches to electricity fraud detection have gained popularity due to smart grids’ big data. However, available datasets lack annotated real anomalies, which poses significant challenges in developing effective detection systems. As a response, researchers have developed synthetic attack models to simulate real-world theft. In this paper, we analyze the validity of popular synthetic attack models and their limitations in representing real-world theft scenarios. By conducting an empirical evaluation of synthetic attacks, we provide a practical assessment of their efficacy and their correlation with one another. Our findings suggest that while these models perform well when tested on synthetic attacks, their performance considerably deteriorates on real-world theft, considering the only real-world public dataset. Furthermore, some attack model pairs were found to be more correlated than others. Consequently, this paper suggests a sufficient subset of synthetic attacks for training and testing to effectively capture the full set of synthetic attacks.