Evaluating Governance Indexes: Critical and Less Critical Questions

Recent years have seen a proliferation of `composite indicators´ or `indexes´ of governance. Such measures can be useful tools for analysing governance, making public policy, building scientific knowledge, and even influencing ruling elites, but some are better tools than others and some are better suited to certain purposes than others. This paper provides a framework of ten questions to help users and producers of governance indexes to evaluate them and consider key components of index design. In reviewing these ten questions–only six of which, it argues, are critical–the paper offers examples from some of the best known measures of governance and related topics. It advances two broad arguments: First, more attention should be paid to the fundamentals of social science methodology, i.e., questions about concept formation, content validity, reliability, replicability, robustness, and the relevance of particular measures to underlying research questions. Second, less attention should be paid to some other issues commonly highlighted in the literature on governance measurement, i.e., questions about descriptive complexity, theoretical fit, the precision of estimates, and correct weighting. The paper builds upon a thorough review of the literature and the author´s three years of research in practice as co-author of a well-known governance index.