IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/10046.html
   My bibliography  Save this paper

A Method to Scale-Up Interpretative Qualitative Analysis, with an Application toAspirations in Cox’s Bazaar, Bangladesh

Author

Listed:
  • Ashwin,Julian
  • Rao,Vijayendra
  • Biradavolu,Monica Rao
  • Chhabra,Aditya
  • Haque,Arshia
  • Khan,Afsana Iffat
  • Krishnan,Nandini

Abstract

The qualitative analysis of open-ended interviews has vast potential in economics buthas found limited use. This is partly because the interpretative, nuanced human reading of text and codingthat it requires is labor intensive and very time consuming. This paper presents a method to simplify and shorten thecoding process by extending a small set of interpretative human-codes to a larger, representative, sample usingnatural language processing and thus analyze qualitative data at scale. It applies it to analyze 2,200 open-endedinterviews on parent’s aspirations for children with Rohingya refugees and their Bangladeshi hosts. It shows thatstudying aspirations with open-ended interviews extends the economics focus on material goals to ideas from philosophyand anthropology that emphasize aspirations for moral and religious values, and the navigational capacity to achievethese aspirations. The paper shows how to assess the robustness and reliability of this approach and finds thatextending the sample of interviews, rather than the human-coded training set, is likely to be optimal.

Suggested Citation

  • Ashwin,Julian & Rao,Vijayendra & Biradavolu,Monica Rao & Chhabra,Aditya & Haque,Arshia & Khan,Afsana Iffat & Krishnan,Nandini, 2022. "A Method to Scale-Up Interpretative Qualitative Analysis, with an Application toAspirations in Cox’s Bazaar, Bangladesh," Policy Research Working Paper Series 10046, The World Bank.
    • Handle: RePEc:wbk:wbrwps:10046
    as

    Download full text from publisher

    File URL: http://documents.worldbank.org/curated/en/099759305162210822/pdf/IDU0a357362e00b6004c580966006b1c2f2e3996.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rickard Nyman & Sujit Kapadia & David Tuckett & David Gregory & Paul Ormerod & Robert Smith, 2018. "News and narratives in financial systems: exploiting big data for systemic risk assessment," Bank of England working papers 704, Bank of England.
    2. Margaret E. Roberts & Brandon M. Stewart & Edoardo M. Airoldi, 2016. "A Model of Text for Experimentation in the Social Sciences," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 988-1003, July.
    3. Jayachandran, Seema & Biradavolu, Monica & Cooper, Jan, 2021. "Using machine learning and qualitative interviews to design a five-question women's agency index," CEPR Discussion Papers 15961, C.E.P.R. Discussion Papers.
    4. Saiz, Lorena & Ashwin, Julian & Kalamara, Eleni, 2021. "Nowcasting euro area GDP with news sentiment: a tale of two crises," Working Paper Series 2616, European Central Bank.
    5. Shapiro, Adam Hale & Sudhof, Moritz & Wilson, Daniel J., 2022. "Measuring news sentiment," Journal of Econometrics, Elsevier, vol. 228(2), pages 221-243.
    6. Larsen, Vegard H. & Thorsrud, Leif Anders & Zhulanova, Julia, 2021. "News-driven inflation expectations and information rigidities," Journal of Monetary Economics, Elsevier, vol. 117(C), pages 507-520.
    7. Fruttero,Anna & Muller,Noel & Calvo-Gonzalez,Oscar, 2021. "The Power and Roots of Aspirations : A Survey of the Empirical Evidence," Policy Research Working Paper Series 9729, The World Bank.
    8. Scott R. Baker & Nicholas Bloom & Steven J. Davis, 2016. "Measuring Economic Policy Uncertainty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1593-1636.
    9. Chris Elbers & Jean O. Lanjouw & Peter Lanjouw, 2003. "Micro--Level Estimation of Poverty and Inequality," Econometrica, Econometric Society, vol. 71(1), pages 355-364, January.
    10. Nimark, Kristoffer P. & Pitschner, Stefan, 2019. "News media and delegated information choice," Journal of Economic Theory, Elsevier, vol. 181(C), pages 160-196.
    11. Stephen Hansen & Michael McMahon & Andrea Prat, 2018. "Transparency and Deliberation Within the FOMC: A Computational Linguistics Approach," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(2), pages 801-870.
    12. Apel, Mikael & Blix Grimaldi, Marianna, 2012. "The Information Content of Central Bank Minutes," Working Paper Series 261, Sveriges Riksbank (Central Bank of Sweden).
    13. Garance Genicot & Debraj Ray, 2017. "Aspirations and Inequality," Econometrica, Econometric Society, vol. 85, pages 489-519, March.
    14. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    15. Parthasarathy, Ramya & Rao, Vijayendra & Palaniswamy, Nethra, 2019. "Deliberative Democracy in an Unequal World: A Text-As-Data Study of South India’s Village Assemblies," American Political Science Review, Cambridge University Press, vol. 113(3), pages 623-640, August.
    16. repec:cup:apsrev:v:113:y:2019:i:03:p:623-640_00 is not listed on IDEAS
    17. Christina D. Romer & David H. Romer, 2004. "A New Measure of Monetary Shocks: Derivation and Implications," American Economic Review, American Economic Association, vol. 94(4), pages 1055-1084, September.
    18. Garance Genicot & Debraj Ray, 2020. "Aspirations and Economic Behavior," Annual Review of Economics, Annual Reviews, vol. 12(1), pages 715-746, August.
    19. Deon Filmer & Lant Pritchett, 2001. "Estimating Wealth Effects Without Expenditure Data—Or Tears: An Application To Educational Enrollments In States Of India," Demography, Springer;Population Association of America (PAA), vol. 38(1), pages 115-132, February.
    20. J. Trent Alexander & Robert Andersen & Peter W. Cookson Jr. & Kathryn Edin & Jonathan Fisher & David B. Grusky & Marybeth Mattingly & Charles Varner, 2017. "A Qualitative Census of Rural and Urban Poverty," The ANNALS of the American Academy of Political and Social Science, , vol. 672(1), pages 143-161, July.
    21. Paul C. Tetlock, 2007. "Giving Content to Investor Sentiment: The Role of Media in the Stock Market," Journal of Finance, American Finance Association, vol. 62(3), pages 1139-1168, June.
    22. Nyman, Rickard & Kapadia, Sujit & Tuckett, David, 2021. "News and narratives in financial systems: Exploiting big data for systemic risk assessment," Journal of Economic Dynamics and Control, Elsevier, vol. 127(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Julian Ashwin & Aditya Chhabra & Vijayendra Rao, 2023. "Using Large Language Models for Qualitative Analysis can Introduce Serious Bias," Papers 2309.17147, arXiv.org, revised Oct 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuting Chen & Don Bredin & Valerio Potì & Roman Matkovskyy, 2022. "COVID risk narratives: a computational linguistic approach to the econometric identification of narrative risk during a pandemic," Digital Finance, Springer, vol. 4(1), pages 17-61, March.
    2. Rickard Nyman & Sujit Kapadia & David Tuckett & David Gregory & Paul Ormerod & Robert Smith, 2018. "News and narratives in financial systems: exploiting big data for systemic risk assessment," Bank of England working papers 704, Bank of England.
    3. Nyman, Rickard & Kapadia, Sujit & Tuckett, David, 2021. "News and narratives in financial systems: Exploiting big data for systemic risk assessment," Journal of Economic Dynamics and Control, Elsevier, vol. 127(C).
    4. Vegard H. Larsen & Leif Anders Thorsrud, 2018. "Business cycle narratives," Working Paper 2018/3, Norges Bank.
    5. Ardia, David & Bluteau, Keven & Boudt, Kris, 2022. "Media abnormal tone, earnings announcements, and the stock market," Journal of Financial Markets, Elsevier, vol. 61(C).
    6. Saiz, Lorena & Ashwin, Julian & Kalamara, Eleni, 2021. "Nowcasting euro area GDP with news sentiment: a tale of two crises," Working Paper Series 2616, European Central Bank.
    7. Shen, Shulin & Xia, Le & Shuai, Yulin & Gao, Da, 2022. "Measuring news media sentiment using big data for Chinese stock markets," Pacific-Basin Finance Journal, Elsevier, vol. 74(C).
    8. Andres Algaba & David Ardia & Keven Bluteau & Samuel Borms & Kris Boudt, 2020. "Econometrics Meets Sentiment: An Overview Of Methodology And Applications," Journal of Economic Surveys, Wiley Blackwell, vol. 34(3), pages 512-547, July.
    9. Kirtac, Kemal & Germano, Guido, 2024. "Sentiment trading with large language models," Finance Research Letters, Elsevier, vol. 62(PB).
    10. Youngjoon Lee & Soohyon Kim & Ki Young Park, 2018. "Deciphering Monetary Policy Committee Minutes with Text Mining Approach: A Case of South Korea," Working papers 2018rwp-132, Yonsei University, Yonsei Economics Research Institute.
    11. Aprigliano, Valentina & Emiliozzi, Simone & Guaitoli, Gabriele & Luciani, Andrea & Marcucci, Juri & Monteforte, Libero, 2023. "The power of text-based indicators in forecasting Italian economic activity," International Journal of Forecasting, Elsevier, vol. 39(2), pages 791-808.
    12. Picault, Matthieu & Renault, Thomas, 2017. "Words are not all created equal: A new measure of ECB communication," Journal of International Money and Finance, Elsevier, vol. 79(C), pages 136-156.
    13. Hubert, Paul & Labondance, Fabien, 2021. "The signaling effects of central bank tone," European Economic Review, Elsevier, vol. 133(C).
    14. Aromi, J. Daniel, 2020. "Linking words in economic discourse: Implications for macroeconomic forecasts," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1517-1530.
    15. Shapiro, Adam Hale & Sudhof, Moritz & Wilson, Daniel J., 2022. "Measuring news sentiment," Journal of Econometrics, Elsevier, vol. 228(2), pages 221-243.
    16. Simionescu, Mihaela, 2022. "Econometrics of sentiments- sentometrics and machine learning: The improvement of inflation predictions in Romania using sentiment analysis," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    17. Eleni Kalamara & Arthur Turrell & Chris Redl & George Kapetanios & Sujit Kapadia, 2020. "Making text count: economic forecasting using newspaper text," Bank of England working papers 865, Bank of England.
    18. Dooruj Rambaccussing & Craig Menzies & Andrzej Kwiatkowski, 2022. "Look who’s Talking: Individual Committee members’ impact on inflation expectations," Dundee Discussion Papers in Economics 305, Economic Studies, University of Dundee.
    19. Jean-Charles Bricongne & Baptiste Meunier & Raquel Caldeira, 2024. "Should Central Banks Care About Text Mining? A Literature Review," Working papers 950, Banque de France.
    20. Fadda, Pietro & Hanifi, Rayane & Istrefi, Klodiana & Penalver, Adrian, 2022. "Central Bank Communication of Uncertainty," CEPR Discussion Papers 17728, C.E.P.R. Discussion Papers.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:10046. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.