Validation of psychometric research instruments: The case of information science

Instrument validation is a critical step that researchers should employ in order to ensure the generation of scientifically valid knowledge. Without it, the basis of research findings and the generalization of such are threatened. This is especially true in the social sciences, a discipline in which the majority of published articles utilize subjective instruments in the collection of data. Consequently, instrument validation has increasingly become common practice in the social sciences, yet implementation of this practice differs greatly among the social‐science disciplines. The assessment of instrument validation undertaken in this study attempts to provide guidance for reviewers, editors, authors, and readers, by offering various techniques of validity and analyzing the quality of a set of psychometric journal articles. In this research, six attributes of instrument validation areas are identified as validation protocol. The Journal of the American Society for Information Science and Technology (JASIST), which is widely recognized as a leading journal in the field of information science, was selected for examination to determine how well a set of research articles ranked in meeting instrument‐validation protocol. Findings show that while researchers are becoming increasingly attentive to certain validation issues, standards on validation processes and reporting might prove helpful. This paper identifies areas for improvement in the reporting of validity measures and offers ways for researchers to implement them.