Dynamic Factor Analysis for Short Panels: Estimating Performance Trajectories for Water Utilities

We develop a dynamic factor model for panel data with a short time dimension (i.e. n<15). Unlike most of the work in the DFM literature where one common factor is estimated for a group of cross sectional units, our interest lies in the estimation of a latent variable for each cross sectional unit at every point in time. This difference increases the computational challenges of the estimation process. To facilitate estimation we develop the “Two-Cycle Conditional Expectation-Maximization” (2CCEM) algorithm which is a variant of the EM algorithm and it’s extensions (Dempster et al. 1977; Meng and Rubin 1993; Liu and Rubin 1994). Initially, the latent variable is estimated (first cycle) and then the dynamic component is incorporated into the estimation process (second cycle). The estimates of each cycle are updated with information from the estimates of the previous cycle until convergence is achieved. We provide simulation results demonstrating consistency of our 2CCEM estimator. One of the advantages of this work is that the estimation strategy can account for multiple cross sectional units with a short time dimension, and is flexible enough to be used in different types of applications. We apply our model to a dataset of 853 water and sanitation utilities from 45 countries and use the 2CCEM algorithm to estimate performance trajectories for each utility.