IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v52y2018i5d10.1007_s11135-017-0650-0.html
   My bibliography  Save this article

Supervised scaling of semi-structured interview transcripts to characterize the ideology of a social policy reform

Author

Listed:
  • Pierre-Marc Daigneault

    (Université Laval)

  • Dominic Duval

    (Université Laval)

  • Louis M. Imbeau

    (Université Laval)

Abstract

Automated content analysis methods treat “text as data” and can therefore analyze efficiently large qualitative databases. Yet, despite their potential, these methods are rarely used to supplement qualitative analysis in small-N designs. We address this gap by replicating the qualitative findings of a case study of a social policy reform using automated content analysis. To characterize the ideology of this reform, we reanalyze the same interview data with Wordscores, using academic publications as reference texts. As expected, the reform’s ideology is center/center-right, a result that we validate using content, convergent and discriminant strategies. The validation evidence suggests not only that the ideological positioning of the policy reform is credible, but also that Wordscores’ scope of application is greater than expected.

Suggested Citation

  • Pierre-Marc Daigneault & Dominic Duval & Louis M. Imbeau, 2018. "Supervised scaling of semi-structured interview transcripts to characterize the ideology of a social policy reform," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(5), pages 2151-2162, September.
    • Handle: RePEc:spr:qualqt:v:52:y:2018:i:5:d:10.1007_s11135-017-0650-0
    DOI: 10.1007/s11135-017-0650-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-017-0650-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-017-0650-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Herzog, Alexander & Benoit, Kenneth, 2015. "The most unkindest cuts: speaker selection and expressed government dissent during economic crisis," LSE Research Online Documents on Economics 62243, London School of Economics and Political Science, LSE Library.
    2. Adcock, Robert & Collier, David, 2001. "Measurement Validity: A Shared Standard for Qualitative and Quantitative Research," American Political Science Review, Cambridge University Press, vol. 95(3), pages 529-546, September.
    3. Pierre-Marc Daigneault, 2014. "Three Paradigms of Social Assistance," SAGE Open, , vol. 4(4), pages 21582440145, November.
    4. Louis M. Imbeau, 2009. "Dissonance in Fiscal Policy: A Power Approach," Studies in Public Choice, in: Louis M. Imbeau (ed.), Do They Walk Like They Talk?, chapter 0, pages 167-184, Springer.
    5. Martin, Lanny W. & Vanberg, Georg, 2008. "A Robust Transformation Procedure for Interpreting Political Text," Political Analysis, Cambridge University Press, vol. 16(1), pages 93-100, January.
    6. Baturo, Alexander & Mikhaylov, Slava, 2013. "Life of Brian Revisited: Assessing Informational and Non-Informational Leadership Tools," Political Science Research and Methods, Cambridge University Press, vol. 1(1), pages 139-157, June.
    7. Huo,Jingjing, 2009. "Third Way Reforms," Cambridge Books, Cambridge University Press, number 9780521518437.
    8. Collier, David & Brady, Henry E. & Seawright, Jason, 2010. "Outdated Views of Qualitative Methods: Time to Move On," Political Analysis, Cambridge University Press, vol. 18(4), pages 506-513.
    9. Alan Brier & Bruno Hopp, 2011. "Computer assisted text analysis in the social sciences," Quality & Quantity: International Journal of Methodology, Springer, vol. 45(1), pages 103-128, January.
    10. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    11. Emma Galli & Veronica Grembi & Fabio Padovano, 2008. "Would You Trust An Italian Politician? Preliminary Evidence From Italian Regional Politics," Working Papers 0208, CREI Università degli Studi Roma Tre, revised 2008.
    12. Ken Cousins & Wayne Mcintosh, 2005. "More than Typewriters, More than Adding Machines: Integrating Information Technology into Political Research," Quality & Quantity: International Journal of Methodology, Springer, vol. 39(5), pages 581-614, October.
    13. Jonathan B. Slapin & Sven‐Oliver Proksch, 2008. "A Scaling Model for Estimating Time‐Series Party Positions from Texts," American Journal of Political Science, John Wiley & Sons, vol. 52(3), pages 705-722, July.
    14. André Krouwel & Annemarie Elfrinkhof, 2014. "Combining strengths of methods of party positioning to counter their weaknesses: the development of a new methodology to calibrate parties on issues and ideological dimensions," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(3), pages 1455-1472, May.
    15. Benoit, Kenneth & Laver, Michael, 2008. "Compared to What? A Comment on “A Robust Transformation Procedure for Interpreting Political Text†by Martin and Vanberg," Political Analysis, Cambridge University Press, vol. 16(1), pages 101-111, January.
    16. Étienne Charbonneau, 2009. "Talking Like a Tax Collector or a Social Guardian? The Use of Administrative Discourse by U.S. State Lottery Agencies," Studies in Public Choice, in: Louis M. Imbeau (ed.), Do They Walk Like They Talk?, chapter 0, pages 223-240, Springer.
    17. Laver, Michael & Benoit, Kenneth & Garry, John, 2003. "Extracting Policy Positions from Political Texts Using Words as Data," American Political Science Review, Cambridge University Press, vol. 97(2), pages 311-331, May.
    18. Emma Galli & Veronica Grembi & Fabio Padovano, 2009. "Would You Trust an Italian Politician? Evidence from Italian Regional Politics," Studies in Public Choice, in: Louis M. Imbeau (ed.), Do They Walk Like They Talk?, chapter 0, pages 109-129, Springer.
    19. Lowe, Will, 2008. "Understanding Wordscores," Political Analysis, Cambridge University Press, vol. 16(4), pages 356-371.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alexander Herzog & Slava Mikhaylov, 2010. "A new Database of Parliamentary Debates in Ireland, 1922--2008," The Institute for International Integration Studies Discussion Paper Series iiisdp338, IIIS, revised Jul 2010.
    2. Anna Calissano & Simone Vantini & Marika Arena, 2020. "Monitoring rare categories in sentiment and opinion analysis: a Milan mega event on Twitter platform," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 29(4), pages 787-812, December.
    3. Hanna Bäck & Marc Debus & Wolfgang C. Müller, 2016. "Intra-party diversity and ministerial selection in coalition governments," Public Choice, Springer, vol. 166(3), pages 355-378, March.
    4. Alexander Herzog & Slava Mikhaylov, 2010. "Estimating Government Discretion in Fiscal Policy Making," The Institute for International Integration Studies Discussion Paper Series iiisdp339, IIIS, revised Jul 2010.
    5. André Krouwel & Annemarie Elfrinkhof, 2014. "Combining strengths of methods of party positioning to counter their weaknesses: the development of a new methodology to calibrate parties on issues and ideological dimensions," Quality & Quantity: International Journal of Methodology, Springer, vol. 48(3), pages 1455-1472, May.
    6. Marcinkiewicz, Kamil & Auffenberg, Jennie & Kittel, Bernhard, 2012. "Politikpositionen im Reformprozess des öffentlichen Dienstes: Zur Übertragbarkeit der quantitativen Textanalyse," TranState Working Papers 162, University of Bremen, Collaborative Research Center 597: Transformations of the State.
    7. Pongsak Luangaram & Yuthana Sethapramote, 2016. "Central Bank Communication and Monetary Policy Effectiveness: Evidence from Thailand," PIER Discussion Papers 20, Puey Ungphakorn Institute for Economic Research.
    8. Martin Haselmayer & Marcelo Jenny, 2017. "Sentiment analysis of political communication: combining a dictionary approach with crowdcoding," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(6), pages 2623-2646, November.
    9. Diaf, Sami & Döpke, Jörg & Fritsche, Ulrich & Rockenbach, Ida, 2022. "Sharks and minnows in a shoal of words: Measuring latent ideological positions based on text mining techniques," European Journal of Political Economy, Elsevier, vol. 75(C).
    10. Adriana Bunea & Raimondas Ibenskas, 2015. "Quantitative text analysis and the study of EU lobbying and interest groups," European Union Politics, , vol. 16(3), pages 429-455, September.
    11. Seraphine F. Maerz & Carsten Q. Schneider, 2020. "Comparing public communication in democracies and autocracies: automated text analyses of speeches by heads of government," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(2), pages 517-545, April.
    12. Rebecca Cordell & Kristian Skrede Gleditsch & Florian G Kern & Laura Saavedra-Lux, 2020. "Measuring institutional variation across American Indian constitutions using automated content analysis," Journal of Peace Research, Peace Research Institute Oslo, vol. 57(6), pages 777-788, November.
    13. Caroline Le Pennec, 2020. "Strategic Campaign Communication: Evidence from 30,000 Candidate Manifestos," SoDa Laboratories Working Paper Series 2020-05, Monash University, SoDa Laboratories.
    14. Auffenberg, Jennie & Marcinkiewicz, Kamil, 2013. "Wer gestaltet, wer verwaltet Reformen im öffentlichen Dienst? Ein Methodenvergleich zur Analyse von Arbeitsbeziehungen in Reformprozessen anhand der Polizei Brandenburg," TranState Working Papers 170, University of Bremen, Collaborative Research Center 597: Transformations of the State.
    15. Greene, Zac & Ceron, Andrea & Schumacher, Gijs & Fazekas, Zoltan, 2016. "The Nuts and Bolts of Automated Text Analysis. Comparing Different Document Pre-Processing Techniques in Four Countries," OSF Preprints ghxj8, Center for Open Science.
    16. Born, Andreas & Janssen, Aljoscha, 2020. "Does a District-Vote Matter for the Behavior of Politicians? A Textual Analysis of Parliamentary Speeches," Working Paper Series 1320, Research Institute of Industrial Economics.
    17. Marc Debus, 2009. "Pre-electoral commitments and government formation," Public Choice, Springer, vol. 138(1), pages 45-64, January.
    18. Pádraig Carmody, 2011. "An Informationalised Economy in Africa? The Impact of New ICT on the Wood Products Industry in Durban, South Africa," The Institute for International Integration Studies Discussion Paper Series iiisdp383, IIIS.
    19. Merz, Nicolas & Regel, Sven & Lewandowski, Jirka, 2016. "The Manifesto Corpus: A new resource for research on political parties and quantitative text analysis," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 3(2 (April-), pages 1-8.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:52:y:2018:i:5:d:10.1007_s11135-017-0650-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.