Measuring institutional variation across American Indian constitutions using automated content analysis

Effectively measuring variation in institutions over time and across jurisdictions is important for examining how institutional characteristics shape political, social, and economic issues. We present a new dataset of American Indian and Alaska Native (AIAN) constitutions and a new approach for measuring variation in polities using machine learning techniques. Existing data on AIAN institutions have largely been based on costly and time-consuming expert coding and survey approaches, where the end product will become obsolete once institutions change. Our automated content analysis of AIAN constitutional documents allows for more flexible and customizable measurement of the variation, using a larger corpus of data than existing approaches, limited by data collection and coding costs. We consider variation in judicial institutions, previously shown to play a crucial role in AIAN development, and compare our machine coded measures to existing hand coded data for a sample of 97 American Indian constitutions. We show that machine coding replicates expert coded data. Our approach can be easily extended to other topics, including the executive, and shows the potential of automated measures to complement or confirm traditional coding of political institutions.