IDEAS home Printed from https://ideas.repec.org/a/zbw/espost/333207.html

Capturing a News Frame – Comparing Machine-Learning Approaches to Frame Analysis with Different Degrees of Supervision

Author

Listed:
  • Eisele, Olga
  • Heidenreich, Tobias
  • Litvyak, Olga
  • Boomgaarden, Hajo G.

Abstract

The empirical identification of frames drawing on automated text analysis has been discussed intensely with regard to the validity of measurements. Adding to an evolving discussion on automated frame identification, we systematically contrast different machine-learning approaches with a manually coded gold standard to shed light on the implications of using one or the other: (1) topic modeling, (2) keyword-assisted topic modeling (keyATM), and (3) supervised machine learning as three popular and/or promising approaches. Manual coding is based on the Policy Frames codebook, providing an established base that allows future research to dovetail our contribution. Analysing a large dataset of 12 Austrian newspapers’ EU coverage over 11 years (2009–2019), we contribute to addressing the methodological challenges that have emerged for social scientists interested in employing automated tools for frame analysis. While results confirm the superiority of supervised machine-learning, the semi-supervised approach (keyATM) seems unfit for frame analysis, whereas the topic model covers the middle ground. Results are extensively discussed regarding their implications for the validity of approaches.

Suggested Citation

  • Eisele, Olga & Heidenreich, Tobias & Litvyak, Olga & Boomgaarden, Hajo G., 2023. "Capturing a News Frame – Comparing Machine-Learning Approaches to Frame Analysis with Different Degrees of Supervision," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 17(3), pages 205-226.
    • Handle: RePEc:zbw:espost:333207
    DOI: 10.1080/19312458.2023.2230560
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/333207/1/Full-text-article-Eisele-et-al-Capturing-a-News.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.1080/19312458.2023.2230560?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Grün, Bettina & Hornik, Kurt, 2011. "topicmodels: An R Package for Fitting Topic Models," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 40(i13).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sandra Wankmüller, 2023. "A comparison of approaches for imbalanced classification problems in the context of retrieving relevant documents for an analysis," Journal of Computational Social Science, Springer, vol. 6(1), pages 91-163, April.
    2. Arsenyan, Jbid & Mirowska, Agata & Piepenbrink, Anke, 2023. "Close encounters with the virtual kind: Defining a human-virtual agent coexistence framework," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    3. repec:osf:socarx:pc3za_v1 is not listed on IDEAS
    4. David Perea-Khalifi & Ana I. Irimia-Diéguez & Pedro Palos-Sánchez, 2024. "Exploring the determinants of the user experience in P2P payment systems in Spain: a text mining approach," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 10(1), pages 1-32, December.
    5. Hong Joo Lee & Hoyeon Oh, 2020. "A Study on the Deduction and Diffusion of Promising Artificial Intelligence Technology for Sustainable Industrial Development," Sustainability, MDPI, vol. 12(14), pages 1-15, July.
    6. Maksym Polyakov & Morteza Chalak & Md. Sayed Iftekhar & Ram Pandit & Sorada Tapsuwan & Fan Zhang & Chunbo Ma, 2018. "Authorship, Collaboration, Topics, and Research Gaps in Environmental and Resource Economics 1991–2015," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 71(1), pages 217-239, September.
    7. Stefano Sbalchiero & Maciej Eder, 2020. "Topic modeling, long texts and the best number of topics. Some Problems and solutions," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(4), pages 1095-1108, August.
    8. Md Nazrul Islam & Md Mofazzal Hossain & Md Shafayet Shahed Ornob, 2024. "Business research on Industry 4.0: a systematic review using topic modelling approach," Future Business Journal, Springer, vol. 10(1), pages 1-15, December.
    9. Baumgärtner, Martin & Zahner, Johannes, 2025. "Whatever it takes to understand a central banker — Embedding their words using neural networks," Journal of International Economics, Elsevier, vol. 157(C).
    10. Daoud, Adel & Kohl, Sebastian, 2016. "How much do sociologists write about economic topics? Using big data to test some conventional views in economic sociology, 1890 to 2014," MPIfG Discussion Paper 16/7, Max Planck Institute for the Study of Societies.
    11. Cardinale, Roberto & Cardinale, Ivano & Zupic, Ivan, 2024. "The EU's vulnerability to gas price and supply shocks: The role of mismatches between policy beliefs and changing international gas markets," Energy Economics, Elsevier, vol. 131(C).
    12. Peracchi, Silvia, 2025. "Migration crisis in the local news: Evidence from the French–Italian border," Journal of Urban Economics, Elsevier, vol. 150(C).
    13. Shr-Wei Kao & Pin Luarn, 2020. "Topic Modeling Analysis of Social Enterprises: Twitter Evidence," Sustainability, MDPI, vol. 12(8), pages 1-20, April.
    14. Hsia-Ching Chang, 2016. "The Synergy of Scientometric Analysis and Knowledge Mapping with Topic Models: Modelling the Development Trajectories of Information Security and Cyber-Security Research," Journal of Information & Knowledge Management (JIKM), World Scientific Publishing Co. Pte. Ltd., vol. 15(04), pages 1-33, December.
    15. Susumu Nagayama & Hitoshi Mitsuhashi, 2022. "Explosive and implosive root concepts: An analysis of music moods rooted by two influential rap artists," PLOS ONE, Public Library of Science, vol. 17(7), pages 1-25, July.
    16. Holand, Øystein & Contiero, Barbara & Næss, Marius W. & Cozzi, Giulio, 2024. "“The Times They Are A-Changin' “ – research trends and perspectives of reindeer pastoralism – A review using text mining and topic modelling," Land Use Policy, Elsevier, vol. 136(C).
    17. Ozgun, Burcu & Broekel, Tom, 2021. "The geography of innovation and technology news - An empirical study of the German news media," Technological Forecasting and Social Change, Elsevier, vol. 167(C).
    18. Anita Mendiratta & Shveta Singh & Surendra Singh Yadav & Arvind Mahajan, 2023. "Bibliometric and Topic Modeling Analysis of Corporate Social Irresponsibility," Global Journal of Flexible Systems Management, Springer;Global Institute of Flexible Systems Management, vol. 24(3), pages 319-339, September.
    19. Eric. W. K. See-To & Yang Yang, 2017. "Market sentiment dispersion and its effects on stock return and volatility," Electronic Markets, Springer;IIM University of St. Gallen, vol. 27(3), pages 283-296, August.
    20. Lüdering Jochen & Winker Peter, 2016. "Forward or Backward Looking? The Economic Discourse and the Observed Reality," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 236(4), pages 483-515, August.
    21. Sabrina L. Woltmann & Lars Alkærsig, 2018. "Tracing university–industry knowledge transfer through a text mining approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 449-472, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:espost:333207. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.