Using Stata for a memory saving fixed effects estimation for the three-way error component model
Abstract"The paper proposes a memory saving decomposition of the design matrix to facilitate fixed effects estimation of the three-way error component model with high numbers of observations and groups. A common way to estimate such a model is to include two of the effects as dummy variables and to sweep out the other effect by the fixed effects trans-formation. If the number of groups is high, the design matrix that includes the dummy variables can be prohibitively large for computer packages that need to store the whole data set in memory. The decomposition of the design matrix proposed here shows a way of how to create the cross-product matrices for the least squares normal equations without explicitly creating the dummy variables for the group effects. As the cross-product matrices are of much lower dimension than the design matrix, this procedure reduces the computer memory required considerably. For example, a model computation shows that in a linked employer-employee data set with 20 million observations and 10 thousand firms, the memory requirement drops from 800 gigabytes to 1 gigabyte. The method is implemented in Stata by making use of the new Mata environment available in Stata 9.0. Besides implementing the memory-saving estimation method, the program also takes care of identification issues (grouping algorithm) and provides useful summary statistics. The paper presents the Stata program and comments its output." (Author's abstract, IAB-Doku) ((en)) Additional Information The Stata programs can be installed either by typing 'net search felsdvreg' or per download from http://www.stata-journal.com/software/sj8-2/st0143/.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany] in its series FDZ Methodenreport with number 200603_en.
Length: o. Sz.
Date of creation: 01 Oct 2006
Date of revision:
IAB-Linked-Employer-Employee-Datensatz; Datenanalyse; Software; Algorithmus; Schätzung;
Find related papers by JEL classification:
- C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Longitudinal Data; Spatial Time Series
- C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
You can help add them by filling out this form.
CitEc Project, subscribe to its RSS feed for this item.
- Chalkley, Martin & Wang, Shaolin & Tilley, Colin, 2011.
"Comparing the treatment provided by migrant and nonmigrant health professionals: dentists in Scotland,"
SIRE Discussion Papers
2011-01, Scottish Institute for Research in Economics (SIRE).
- Martin Chalkley & Colin Tilley & Shaolin Wang, 2011. "Comparing the treatment provided by migrant and non-migrant health professionals: dentists in Scotland," Dundee Discussion Papers in Economics 249, Economic Studies, University of Dundee.
- Davidson, Carl & Heyman, Fredrik & Matusz, Steven & Sjöholm, Fredrik & Zhu, Susan Chun, 2011.
"Globalization and Imperfect Labor Market Sorting,"
2011:25, Lund University, Department of Economics.
- Davidson, Carl & Heyman, Fredrik & Matusz, Steven & Sjöholm, Fredrik & Chun Zhu, Susan, 2010. "Globalization and Imperfect Labor Market Sorting," Working Paper Series 856, Research Institute of Industrial Economics.
- Carl Davidson & Fredrik Heyman & Steven Matusz, & Fredrik Sjoholm & Susan Zhu, . "Globalization and Imperfect Labor Market Sorting," Discussion Papers 10/30, University of Nottingham, GEP.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (IAB, Geschäftsbereich Dokumentation und Bibliothek).
If references are entirely missing, you can add them using this form.