IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v54y2020i2d10.1007_s11135-019-00885-7.html
   My bibliography  Save this article

Comparing public communication in democracies and autocracies: automated text analyses of speeches by heads of government

Author

Listed:
  • Seraphine F. Maerz

    (Bard College)

  • Carsten Q. Schneider

    (Central European University (CEU))

Abstract

Renewed efforts at empirically distinguishing between different forms of political regimes leave out the cultural dimension. In this article, we demonstrate how modern computational tools can be used to fill this gap. We employ web-scraping techniques to generate a data set of speeches by heads of government in European democracies and autocratic regimes around the globe. Our data set includes 4740 speeches delivered between 1999 and 2019 by 40 political leaders of 27 countries. By scaling the results of a dictionary application, we show how, in comparative terms, liberal or illiberal the leaders present themselves to their national and international audience. In order to gauge whether our liberalness scale reveals meaningful distinctions, we perform a series of validity tests: criterion validity, qualitative hand-coding, unsupervised topic modeling, and network analysis. All tests suggest that our liberalness scale does capture meaningful differences between political regimes despite the large heterogeneity of our data.

Suggested Citation

  • Seraphine F. Maerz & Carsten Q. Schneider, 2020. "Comparing public communication in democracies and autocracies: automated text analyses of speeches by heads of government," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(2), pages 517-545, April.
    • Handle: RePEc:spr:qualqt:v:54:y:2020:i:2:d:10.1007_s11135-019-00885-7
    DOI: 10.1007/s11135-019-00885-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-019-00885-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-019-00885-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Adcock, Robert & Collier, David, 2001. "Measurement Validity: A Shared Standard for Qualitative and Quantitative Research," American Political Science Review, Cambridge University Press, vol. 95(3), pages 529-546, September.
    2. Gerschewski, Johannes, 2013. "The three pillars of stability: legitimation, repression, and co-optation in autocratic regimes," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 20(1), pages 13-38.
    3. Vanessa A Boese, 2019. "How (not) to measure democracy," International Area Studies Review, Center for International Area Studies, Hankuk University of Foreign Studies, vol. 22(2), pages 95-127, June.
    4. Margaret E. Roberts & Brandon M. Stewart & Dustin Tingley & Christopher Lucas & Jetson Leder‐Luis & Shana Kushner Gadarian & Bethany Albertson & David G. Rand, 2014. "Structural Topic Models for Open‐Ended Survey Responses," American Journal of Political Science, John Wiley & Sons, vol. 58(4), pages 1064-1082, October.
    5. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    6. de Vries, Erik & Schoonvelde, Martijn & Schumacher, Gijs, 2018. "No Longer Lost in Translation: Evidence that Google Translate Works for Comparative Bag-of-Words Text Applications," Political Analysis, Cambridge University Press, vol. 26(4), pages 417-430, October.
    7. Denny, Matthew J. & Spirling, Arthur, 2018. "Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It," Political Analysis, Cambridge University Press, vol. 26(2), pages 168-189, April.
    8. Lucas, Christopher & Nielsen, Richard A. & Roberts, Margaret E. & Stewart, Brandon M. & Storer, Alex & Tingley, Dustin, 2015. "Computer-Assisted Text Analysis for Comparative Politics," Political Analysis, Cambridge University Press, vol. 23(2), pages 254-277, April.
    9. Laver, Michael & Benoit, Kenneth & Garry, John, 2003. "Extracting Policy Positions from Political Texts Using Words as Data," American Political Science Review, Cambridge University Press, vol. 97(2), pages 311-331, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mohamed M. Mostafa, 2023. "A one-hundred-year structural topic modeling analysis of the knowledge structure of international management research," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3905-3935, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ferrara, Federico M. & Masciandaro, Donato & Moschella, Manuela & Romelli, Davide, 2022. "Political voice on monetary policy: Evidence from the parliamentary hearings of the European Central Bank," European Journal of Political Economy, Elsevier, vol. 74(C).
    2. Karell, Daniel & Freedman, Michael Raphael, 2019. "Rhetorics of Radicalism," SocArXiv yfzsh, Center for Open Science.
    3. Michal Ovádek & Nicolas Lampach & Arthur Dyevre, 2020. "What’s the talk in Brussels? Leveraging daily news coverage to measure issue attention in the European Union," European Union Politics, , vol. 21(2), pages 204-232, June.
    4. Dehler-Holland, Joris & Schumacher, Kira & Fichtner, Wolf, 2021. "Topic Modeling Uncovers Shifts in Media Framing of the German Renewable Energy Act," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 2(1).
    5. Mohamed M. Mostafa, 2023. "A one-hundred-year structural topic modeling analysis of the knowledge structure of international management research," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3905-3935, August.
    6. Camilla Salvatore & Silvia Biffignandi & Annamaria Bianchi, 2022. "Corporate Social Responsibility Activities Through Twitter: From Topic Model Analysis to Indexes Measuring Communication Characteristics," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 164(3), pages 1217-1248, December.
    7. Pierre-Marc Daigneault & Dominic Duval & Louis M. Imbeau, 2018. "Supervised scaling of semi-structured interview transcripts to characterize the ideology of a social policy reform," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(5), pages 2151-2162, September.
    8. Dehler-Holland, Joris & Okoh, Marvin & Keles, Dogan, 2022. "Assessing technology legitimacy with topic models and sentiment analysis – The case of wind power in Germany," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    9. Ulrich Fritsche & Johannes Puckelwald, 2018. "Deciphering Professional Forecasters’ Stories - Analyzing a Corpus of Textual Predictions for the German Economy," Macroeconomics and Finance Series 201804, University of Hamburg, Department of Socioeconomics.
    10. Hager, Anselm & Hilbig, Hanno, 2020. "Does Public Opinion Affect Political Speech?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 64(4), pages 921-937.
    11. Woohyuk Kim & Sung-Bum Kim & Eunhye Park, 2021. "Mapping Tourists’ Destination (Dis)Satisfaction Attributes with User-Generated Content," Sustainability, MDPI, vol. 13(22), pages 1-16, November.
    12. Dukalskis, Alexander & Gerschewski, Johannes, 2020. "Adapting or Freezing? Ideological Reactions of Communist Regimes to a Post-Communist World," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 55(3), pages 511-532.
    13. Peter Grajzl & Peter Murrell, 2021. "Characterizing a legal–intellectual culture: Bacon, Coke, and seventeenth-century England," Cliometrica, Journal of Historical Economics and Econometric History, Association Française de Cliométrie (AFC), vol. 15(1), pages 43-88, January.
    14. Peter Grajzl & Cindy Irby, 2019. "Reflections on study abroad: a computational linguistics approach," Journal of Computational Social Science, Springer, vol. 2(2), pages 151-181, July.
    15. Greene, Zac & Ceron, Andrea & Schumacher, Gijs & Fazekas, Zoltan, 2016. "The Nuts and Bolts of Automated Text Analysis. Comparing Different Document Pre-Processing Techniques in Four Countries," OSF Preprints ghxj8, Center for Open Science.
    16. Ebadi, Ashkan & Tremblay, Stéphane & Goutte, Cyril & Schiffauerova, Andrea, 2020. "Application of machine learning techniques to assess the trends and alignment of the funded research output," Journal of Informetrics, Elsevier, vol. 14(2).
    17. Edward Kerby & Alexander Moradi & Hanjo Odendaal, 2022. "African time travellers: what can we learn from 500 years of written accounts?," Oxford Economic and Social History Working Papers _201, University of Oxford, Department of Economics.
    18. LIM Jaehwan & ITO Asei & ZHANG Hongyong, 2023. "Policy Agenda and Trajectory of the Xi Jinping Administration: Textual Evidence from 2012 to 2022," Policy Discussion Papers 23008, Research Institute of Economy, Trade and Industry (RIETI).
    19. Caroline E Nowacki & Ashby Monk & Bertrand Decoster, 2021. "Who do sovereign wealth funds say they are? Using structural topic modeling to delineate variegated capitalism in their official reports," Environment and Planning A, , vol. 53(4), pages 828-857, June.
    20. Merz, Nicolas & Regel, Sven & Lewandowski, Jirka, 2016. "The Manifesto Corpus: A new resource for research on political parties and quantitative text analysis," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 3(2 (April-), pages 1-8.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:54:y:2020:i:2:d:10.1007_s11135-019-00885-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.