Mining Chinese historical sources at scale: A machine learning approach to Qing state capacity

Primary historical sources are often by-passed for secondary sources due to the high human costs of accessing and extracting primary information. We propose a supervised machine-learning approach to analyzing Chinese historical texts based on natural language processing techniques. An application to identifying different forms of social unrest in the Veritable Records of the Qing Dynasty shows that this approach dramatically cuts down the cost of using primary source data at the same time when it is free from human bias, reproducible, and flexible enough to address particular questions. We extract all records of unrest, tagging each record by geographical region and date, to give a comprehensive dataset for the Qing Dynasty (1644–1911) of when and where different types of episodes took place. External evidence on triggers of unrest also suggests that the computer-based approach is no less successful in identifying social unrest than human researchers are. The quantitative results show that the fault lines of state capacity began by the Jiaqing (1796–1820), well before the well-known challenges of the mid-nineteenth century, the Taiping Rebellion and the Opium War.