NILMbench: A novel benchmark for high-frequency NILM regression models

Non-Intrusive Load Monitoring (NILM) estimates the power consumption of individual household appliances from aggregate measurements. Although NILM has been studied for decades, progress remains limited by the lack of standardized benchmarks for evaluating regression models. To address this gap, we introduce NILMbench, an open-source benchmark for NILM evaluation. NILMbench is universal in the sense that it can evaluate both regression and classification NILM models operating on high- or low-frequency data. We construct an evaluation protocol that is centered around the modified Jaccard index MJ metric with a hybrid tolerance mechanism. This metric couples appliance identification and power estimation into a single score between 0 and 1. This metric aims to address the limitations of regression metrics adopted from other fields, which can produce inflated or misleading evaluation results in NILM scenarios. We analyze these limitations through a mathematical comparison with commonly used NILM evaluation metrics from the literature. We demonstrate NILMbench by benchmarking four representative high-frequency NILM regression models, establishing a reference set of scores against which future methods can be compared.