Pipeformer: A multi-channel patching-based transformer method with pyramid cross-attention for leak detection of energy transportation system

Accurate leak detection in an energy transportation system can effectively reduce the likelihood of a disaster. To enhance detection accuracy and reduce false alarm rate, a multi-channel patching-based transformer method with pyramid cross-attention for pipeline (Pipeformer) in energy transportation system is proposed. First, a transformer encoder-based representation learning framework is presented to tackle the pipeline leak detection task by utilizing unsupervised clustering algorithms. Then, given the interconnections among pressure data in pipeline, a multi-channel patch embedding strategy is proposed to patch the multivariate time series at unequal time steps to capture pressure variation features at different scales. Specifically, within the same scale, it can adequately capture the differences between local and global features by patching the data, as well as consolidate comprehensive feature information through fusion among different scales. Moreover, a pyramid cross-attention mechanism is designed with the help of the feature pyramid structure that has been constructed through patching. Cross-attention between different layers is used to convey feature information, while relying on this structure to capture both short-term fine-grained patterns and long-term coarse-grained trends. The interaction of information among features at different scales enables the model to effectively learn cross-scale dependencies. Finally, leak detection task of energy transportation system is implemented by a clustering algorithm and a series of comparison and ablation experiments are carried out using the collected data. The detection accuracy and false alarm rates can achieve satisfactory results, which verifies the validity of the method.