Markov Cohort State-Transition Model: A Multinomial Distribution Representation

Markov cohort state-transition models have been the standard approach for simulating the prognosis of patients or, more generally, the life trajectories of individuals over a time period. Current approaches for estimating the variance of a Markov model using a Monte Carlo sampling or a master equation representation are computationally expensive and analytically difficult to express and solve. We introduce an alternative representation of a Markov model in the form of a multinomial distribution. We derive this representation from principles and then verify its veracity in a simulation exercise. This representation provides an exact and fast approach to computing the variance and a way of estimating transition probabilities in a Bayesian setting. Highlights A Markov model simulates the average experience of a cohort of patients. Monte Carlo simulation, the standard approach for estimating the variance, is computationally expensive. A multinomial distribution provides an exact representation of a Markov model. Using the known formulas of a multinomial distribution, the mean and variance of a Markov model can be readily calculated.