Engineering and evaluating an unsupervised predictive maintenance solution: a cold-forming press case-study

In real-world industries, production line assets may be affected by several factors, both known and unknown, which dynamically and unpredictably evolve so that past data are of little value for present ones. In addition, data is collected without assigned labels. How can someone use run-to-failure data to develop a suitable solution toward achieving predictive maintenance (PdM) in this case? These issues arise in our case, which refers to a cold-forming press. Such a setting calls for an unsupervised solution that can predict upcoming failures investigating a wide spectrum of approaches, namely similarity-based, forecasting-based and deep-learning ones. But before we decide on the best solution, we first need to understand which key performance indicators are appropriate to evaluate the impact of each such solution. A comprehensive study of available evaluation methods is presented, highlighting misconceptions and limitations of broadly used evaluation metrics concerning run-to-failure data, while proposing an extension of state-of-the-art range-based anomaly detection evaluation metrics to serve PdM purposes. Finally, an investigation of pre-processing, distance metrics, incorporation of domain expertise, and the role of deep learning shows how to engineer an unsupervised solution for predictive maintenance providing insightful answers to all these problems. Our experimental evaluation showed that judicious design choices can improve efficiency of solutions up to two times.