IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2408.09742.html
   My bibliography  Save this paper

Paired Completion: Flexible Quantification of Issue-framing at Scale with LLMs

Author

Listed:
  • Simon D Angus
  • Lachlan O'Neill

Abstract

Detecting issue framing in text - how different perspectives approach the same topic - is valuable for social science and policy analysis, yet challenging for automated methods due to subtle linguistic differences. We introduce `paired completion', a novel approach using LLM next-token log probabilities to detect contrasting frames using minimal examples. Through extensive evaluation across synthetic datasets and a human-labeled corpus, we demonstrate that paired completion is a cost-efficient, low-bias alternative to both prompt-based and embedding-based methods, offering a scalable solution for analyzing issue framing in large text collections, especially suited to low-resource settings.

Suggested Citation

  • Simon D Angus & Lachlan O'Neill, 2024. "Paired Completion: Flexible Quantification of Issue-framing at Scale with LLMs," Papers 2408.09742, arXiv.org, revised Jun 2025.
  • Handle: RePEc:arx:papers:2408.09742
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2408.09742
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chong, Dennis & Druckman, James N., 2007. "Framing Public Opinion in Competitive Democracies," American Political Science Review, Cambridge University Press, vol. 101(4), pages 637-655, November.
    2. Jason Barabas & Jennifer Jerit, 2009. "Estimating the Causal Effects of Media Coverage on Policy‐Specific Knowledge," American Journal of Political Science, John Wiley & Sons, vol. 53(1), pages 73-89, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Katerina Linos & Kimberly Twist, 2016. "The Supreme Court, the Media, and Public Opinion: Comparing Experimental and Observational Methods," The Journal of Legal Studies, University of Chicago Press, vol. 45(2), pages 223-254.
    2. Schnellenbach, Jan & Schubert, Christian, 2015. "Behavioral political economy: A survey," European Journal of Political Economy, Elsevier, vol. 40(PB), pages 395-417.
    3. Midha, Joshua, 2022. "The Cycle of Rule: Existential Risks, Continuity Of Governance, And Conflict-Based Preservation," SocArXiv vc7w9, Center for Open Science.
    4. Fung, Timothy K.F. & Choi, Doo Hun & Scheufele, Dietram A. & Shaw, Bret R., 2014. "Public opinion about biofuels: The interplay between party identification and risk/benefit perception," Energy Policy, Elsevier, vol. 73(C), pages 344-355.
    5. Matthew Gentzkow & Jesse M. Shapiro & Matt Taddy, 2019. "Measuring Group Differences in High‐Dimensional Choices: Method and Application to Congressional Speech," Econometrica, Econometric Society, vol. 87(4), pages 1307-1340, July.
    6. Bloemraad, Irene & Voss, Kim & Silva, Fabiana, 2014. "Framing the Immigrant Movement as about Rights, Family, or Economics: Which Appeals Resonate and for Whom?," Institute for Research on Labor and Employment, Working Paper Series qt3b32w33p, Institute of Industrial Relations, UC Berkeley.
    7. Przeszlowski, Kimberly & Guerette, Rob T., 2025. "Public perceptions on police use of information technologies: Findings from a randomized vignette experiment," Journal of Criminal Justice, Elsevier, vol. 96(C).
    8. Justin Wedeking, 2010. "Supreme Court Litigants and Strategic Framing," American Journal of Political Science, John Wiley & Sons, vol. 54(3), pages 617-631, July.
    9. Matt Guardino & Suzanne Mettler, 2020. "Revealing the “Hidden welfare state†: How policy information influences public attitudes about tax expenditures," Journal of Behavioral Public Administration, Center for Experimental and Behavioral Public Administration, vol. 3(1).
    10. Bhalotra, Sonia & Clots-Figueras, Irma & Iyer, Lakshmi, 2021. "Religion and abortion: The role of politician identity," Journal of Development Economics, Elsevier, vol. 153(C).
    11. Max Grömping & Darren R. Halpin, 2021. "Do think tanks generate media attention on issues they care about? Mediating internal expertise and prevailing governmental agendas," Policy Sciences, Springer;Society of Policy Sciences, vol. 54(4), pages 849-866, December.
    12. Reinald Besalú & Arantxa Capdevila & Carlota M. Moragas-Fernández, 2025. "Ecological Transition in Spain: Political Polarization Through Institutions and Media," Land, MDPI, vol. 14(4), pages 1-23, April.
    13. Trisha R. Shrum, 2021. "The salience of future impacts and the willingness to pay for climate change mitigation: an experiment in intergenerational framing," Climatic Change, Springer, vol. 165(1), pages 1-20, March.
    14. Gilles Grolleau & Luc Meunier & Naoufel Mzoughi, 2025. "Targeting the Sin or the Sinner? Applying Kahneman's Insights to Frame Environmental Messages for Better Waste Management," Post-Print hal-04926850, HAL.
    15. Rogers, Todd & Nickerson, David W., 2013. "Can Inaccurate Beliefs about Incumbents be Changed? And Can Reframing Change Votes?," Working Paper Series rwp13-018, Harvard University, John F. Kennedy School of Government.
    16. Hawkins, Christopher V. & Chia-Yuan, Yu, 2018. "Voter support for environmental bond referenda," Land Use Policy, Elsevier, vol. 76(C), pages 193-200.
    17. Shapiro, Matthew A. & Bolsen, Toby, 2019. "Korean perceptions of transboundary air pollution and domestic coal development: Two framing experiments," Energy Policy, Elsevier, vol. 126(C), pages 333-342.
    18. Bechtel, Michael & Hainmueller, Jens & Hangartner, Dominik & Helbling, Marc, 2015. "Reality Bites: The Limits of Framing Effects for Salient and Contested Policy Issues," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 3(3), pages 683-695.
    19. Sigurd Hilmo Lundheim & Giuseppe Pellegrini-Masini & Christian A. Klöckner & Stefan Geiss, 2022. "Developing a Theoretical Framework to Explain the Social Acceptability of Wind Energy," Energies, MDPI, vol. 15(14), pages 1-24, July.
    20. Giebler, Heiko & Banducci, Susan & Kritzinger, Sylvia, 2017. "New perspectives on information and electoral competition," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 52(4), pages 429-435.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2408.09742. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.