Stephen Brehm () (University of Chicago) L. Philip Schumm () (Department of Health Studies, University of Chicago)
Abstract
Efficient data cleaning and management is critical to the success of any large research project. This is particularly true in the case of longitudinal studies and/or those in which the data management tasks are shared among many individuals. Faced with several such projects, we developed a flexible, easy-to-use system for cleaning and managing research datasets. The system is modular, making it easy for different individuals to work on different parts of the process. This modularity also permits substantial code reuse over multiple waves of a longitudinal study. A central focus of the system is the idea of data testing; users write tests for specific variables that may then be rerun when a new wave of data becomes available or when changes to the data have been made. Although the basic ideas could be implemented in any statistical package or programming language, Stata is particularly well-suited to the task. In addition, we have written an ado-file to automate the process of building a data set and another to generate basic tests automatically from an existing dataset. Although the system was designed for use by large, collaborative projects, individuals can also benefit from using it for personal research projects.
Download Info
To download:
If you experience problems downloading a file, check if you have the
proper application to
view it first. Information about this may be contained
in the File-Format links below. In case of further problems read
the IDEAS help
page. Note that these files are not on the IDEAS
site. Please be patient as the files may be large.