Using Stata for a memory saving fixed effects estimation for the three-way error component model

"The paper proposes a memory saving decomposition of the design matrix to facilitate fixed effects estimation of the three-way error component model with high numbers of observations and groups. A common way to estimate such a model is to include two of the effects as dummy variables and to sweep out the other effect by the fixed effects trans-formation. If the number of groups is high, the design matrix that includes the dummy variables can be prohibitively large for computer packages that need to store the whole data set in memory. The decomposition of the design matrix proposed here shows a way of how to create the cross-product matrices for the least squares normal equations without explicitly creating the dummy variables for the group effects. As the cross-product matrices are of much lower dimension than the design matrix, this procedure reduces the computer memory required considerably. For example, a model computation shows that in a linked employer-employee data set with 20 million observations and 10 thousand firms, the memory requirement drops from 800 gigabytes to 1 gigabyte. The method is implemented in Stata by making use of the new Mata environment available in Stata 9.0. Besides implementing the memory-saving estimation method, the program also takes care of identification issues (grouping algorithm) and provides useful summary statistics. The paper presents the Stata program and comments its output." (Author's abstract, IAB-Doku) ((en))