Closed-form estimators for multivariate regressions models -a single categorical variable approach

The maximum likelihood estimator (MLE) remains the most frequently used method to estimate the parameters of generalized linear models. But even for distributions within the exponential family, MLEs are not always tractable and need to be computed with time consuming numerical methods like the Iterative Weighted Least Square algorithm. In order to improve the computation time, closed-form estimators have been found in case of categorical explanatory variables for univariate random variables of one-parameter exponential type. In the context of multivariate generalized linear models (MGLM), we propose a new way to look at the score in case of single categorical variables for any distribution in the exponential family. Firstly, we derive closed-form MLE for MGLM assuming multinomial and negative multinomial distributions. Secondly, we deduce similar results for the multivariate normal distributions. For the Dirichlet distribution, we propose a closed-form estimator, yet not MLE, for which we prove the consistency. We illustrate the computation time gains on simulated datasets: closed-form estimators are about 1000 times faster, especially for high dimension. Closed-form estimators are computed in constant times.. Finally, we show the relevancy of the proposed estimator on real-world datasets by modeling cause-of-death mortality in US. We are able to catch the first-order effects of covid between 2019 and 2021.