Moral Foundation Measurements Fail to Converge on Multilingual Party Manifestos

Moralising language is a powerful rhetorical tool for signaling political identity, persuading audiences, and mobilising voters. The valid and reliable classification of moral language is therefore a critical objective for political scientists. Recent advances in automated text analysis have introduced myriad new strategies for measuring morality in language, but have often produced conflicting, inconclusive findings. We investigate whether this diversity of moral content analyses might partially explain inconclusive findings, using a large corpus of political manifestos in four different languages (N=810 manifestos). Our results show that, despite starting from the same framework of Moral Foundations Theory (MFT), different instruments and underlying methodologies lead to remarkably different results for extracting moral foundations. Reproducing a previous study on political parties’ ideology and their use of moral foundations, we find that different measurements can lead to opposite effect directions. We discuss the relevance of our findings for research at the intersection of politics and moral rhetoric using automated text analysis.