Statistical inference for discrete-time multistate models: extensions to Markov Chains with rewards

Markov Chains with rewards (MCWR) have been shown to be a useful modelling extension to discrete-time multistate models (DTMS). In this paper, we substantially improve and extend the possibilities that MCWR holds for DTMS. We make several contributions. First, we develop a system of creating and naming different rewards schemes, so-called "standard rewards". While some of these schemes are of interest in their own right, several new possibilities emerge when dividing one rewards result by another, the result of which we call "composite rewards". In total, we can define at least ten new useful outcome statistics based on MCWR that have not yet been used in the literature. Secondly, we derive expressions for asymptotic covariance matrices that are applicable for any standard rewards definition. Thirdly, we show how joint covariance matrices of two or more rewards results can be obtained, which leads to expressions for covariance matrices of composite rewards. Lastly, expressions for point estimates and covariance matrices of partial age ranges are derived. We confirm correctness of results by comparisons to simulation-based results (point estimates) and by comparisons to bootstrap-based results (covariance matrices).