An efficient alternating direction algorithm for high-dimensional basis precision matrix estimation via lasso penalized D-trace loss

High-dimensional compositional data, including human microbiome information, play a crucial role in advancing the understanding of health conditions like metabolic disorders, cardiovascular diseases, and inflammatory bowel diseases. However, estimating precision matrices in such datasets is difficult due to the constraints of nonnegativity, the sum-to-one property, and the high-dimensional sparse nature of the data. In particular, traditional methods are unsuitable for addressing these complexities, especially when the number of variables (p) greatly surpasses the sample size (n), a scenario commonly encountered in microbiome studies. To tackle this problem, we introduce an efficient Alternating Direction Algorithm designed for the estimation of high-dimensional Basis Precision Matrices, termed ada-BPM. By using the optimality conditions of the lasso-penalized D-trace loss function, we obtain explicit formulas for each iteration, which has a maximum computational complexity of $$O(np^2)$$ . This complexity is practically efficient, as it matches the cost of computing the sample covariance matrix, a step commonly required in compositional precision matrix estimation, and avoids the cubic-order operations present in earlier methods. In addition, we demonstrate the global convergence of ada-BPM and assess its performance through empirical evaluations conducted on both simulated and real-world datasets. The findings suggest that ada-BPM surpasses alternative methods in both performance and computational efficiency, especially when handling high-dimensional sparse data. As a result, this method offers significant advantages for analyzing large-scale compositional data, particularly in the context of microbiome research.