Measuring System Usage: Implications for IS Theory Testing

There is widespread agreement among researchers that system usage, defined as the utilization of information technology (IT) by individuals, groups, or organizations, is the primary variable through which IT affects white collar performance. Despite the number of studies targeted at explaining system usage, there are crucial differences in the way the variable has been conceptualized and operationalized. This wide variation of system usage measures hinders the efforts of MIS researchers to compare findings across studies, thus impeding the accumulation of knowledge and theory in this area. The purpose of this paper is to address conceptual as well as methodological issues related to measuring system usage. First, via LISREL measurement modeling techniques, we compare subjective and objective measures of system usage, namely, self-reported versus computer-recorded measures. Next, using a modified form of Davis' Technology Acceptance Model (TAM) as a nomological net, we test the nomological validity of these system usage constructs and measures. Results of the LISREL measurement and nomological net analysis suggest that system usage should be factored into self-reported system usage and computer-recorded system usage. Contrary to expectations, these constructs do not appear to be strongly related to each other. Moreover, while self-reported measures of system usage are related to self-reported measures of TAM independent variables, objective, computer-recorded measures show distinctly weaker links. In the face of such counter-evidence, it is tempting to argue that research that has relied on subjective measures of system usage (for example, research confirming TAM) may be artifactual. There are several alternative explanations, though, that maintain the integrity of TAM and studies that measure system usage subjectively. These alternative explanations suggest directions for further research as well as new approaches to measurement.