IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/28626.html
   My bibliography  Save this paper

Using Machine Learning and Qualitative Interviews to Design a Five-Question Women's Agency Index

Author

Listed:
  • Seema Jayachandran
  • Monica Biradavolu
  • Jan Cooper

Abstract

We propose a new method to design a short survey measure of a complex concept such as women's agency. The approach combines mixed-methods data collection and machine learning. We select the best survey questions based on how strongly correlated they are with a "gold standard'' measure of the concept derived from qualitative interviews. In our application, we measure agency for 209 women in Haryana, India, first, through a semi-structured interview and, second, through a large set of close-ended questions. We use qualitative coding methods to score each woman's agency based on the interview, which we treat as her true agency. To identify the close-ended questions most predictive of the "truth," we apply statistical algorithms that build on LASSO and random forest but constrain how many variables are selected for the model (five in our case). The resulting five-question index is as strongly correlated with the coded qualitative interview as is an index that uses all of the candidate questions. This approach of selecting survey questions based on their statistical correspondence to coded qualitative interviews could be used to design short survey modules for many other latent constructs.

Suggested Citation

  • Seema Jayachandran & Monica Biradavolu & Jan Cooper, 2021. "Using Machine Learning and Qualitative Interviews to Design a Five-Question Women's Agency Index," NBER Working Papers 28626, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:28626
    Note: CH DEV LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w28626.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ingvild Almås & Alex Armand & Orazio Attanasio & Pedro Carneiro, 2018. "Measuring and Changing Control: Women's Empowerment and Targeted Transfers," Economic Journal, Royal Economic Society, vol. 128(612), pages 609-639, July.
    2. Malapit, Hazel & Quisumbing, Agnes & Meinzen-Dick, Ruth & Seymour, Greg & Martinez, Elena M. & Heckert, Jessica & Rubin, Deborah & Vaz, Ana & Yount, Kathryn M., 2019. "Development of the project-level Women’s Empowerment in Agriculture Index (pro-WEAI)," World Development, Elsevier, vol. 122(C), pages 675-692.
    3. Ana Vaz & Pierre Pratley & Sabina Alkire, 2016. "Measuring Women's Autonomy in Chad Using the Relative Autonomy Index," Feminist Economics, Taylor & Francis Journals, vol. 22(1), pages 264-294, January.
    4. Abigail Barr & Marlene Dekker & Floyd Mwansa & Tia Linda Zuze, 2020. "Financial decision-making, gender and social norms in Zambia: Preliminary report on the quantitative data generation, analysis and results," Discussion Papers 2020-06, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    5. Lupin Rahman & Vijayendra Rao, 2004. "The Determinants of Gender Equity in India: Examining Dyson and Moore's Thesis with New Data," Population and Development Review, The Population Council, Inc., vol. 30(2), pages 239-268, June.
    6. Jha, Saumitra & Rao, Vijayendra & Woolcock, Michael, 2007. "Governance in the Gullies: Democratic Responsiveness and Leadership in Delhi's Slums," World Development, Elsevier, vol. 35(2), pages 230-246, February.
    7. Seema Jayachandran, 2015. "The Roots of Gender Inequality in Developing Countries," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 63-88, August.
    8. Rossalina Latcheva, 2011. "Cognitive interviewing and factor-analytic techniques: a mixed method approach to validity of survey items measuring national identity," Quality & Quantity: International Journal of Methodology, Springer, vol. 45(6), pages 1175-1199, October.
    9. Aletheia Donald & Gayatri Koolwal & Jeannie Annan & Kathryn Falb & Markus Goldstein, 2020. "Measuring Women’s Agency," Feminist Economics, Taylor & Francis Journals, vol. 26(3), pages 200-226, July.
    10. Knippenberg, Erwin & Jensen, Nathaniel & Constas, Mark, 2019. "Quantifying household resilience with high frequency data: Temporal dynamics and methodological options," World Development, Elsevier, vol. 121(C), pages 1-15.
    11. Shaffer, Paul, 2013. "Ten Years of “Q-Squared”: Implications for Understanding and Explaining Poverty," World Development, Elsevier, vol. 45(C), pages 269-285.
    12. Martha NUSSBAUM, 1999. "Women and equality: The capabilities approach," International Labour Review, International Labour Organization, vol. 138(3), pages 227-245, September.
    13. Laszlo, Sonia & Grantham, Kate & Oskay, Ecem & Zhang, Tingting, 2020. "Grappling with the challenges of measuring women's economic empowerment in intrahousehold settings," World Development, Elsevier, vol. 132(C).
    14. Maiorano, Diego & Shrimankar, Dishil & Thapar-Björkert, Suruchi & Blomkvist, Hans, 2021. "Measuring empowerment: Choices, values and norms," World Development, Elsevier, vol. 138(C).
    15. Giulia Greco & Jolene Skordis-Worrall & Anne Mills, 2018. "Development, Validity, and Reliability of the Women’s Capabilities Index," Journal of Human Development and Capabilities, Taylor & Francis Journals, vol. 19(3), pages 271-288, July.
    16. Robin Richardson & Norbert Schmitz & Sam Harper & Arijit Nandi, 2019. "Development of a Tool to Measure Women’s Agency in India," Journal of Human Development and Capabilities, Taylor & Francis Journals, vol. 20(1), pages 26-53, January.
    17. Hargreaves, James R. & Morison, Linda A. & Gear, John S.S. & Makhubele, Mzamani B. & Porter, John D.H. & Busza, Joanna & Watts, Charlotte & Kim, Julia C. & Pronyk, Paul M., 2007. ""Hearing the Voices of the Poor": Assigning Poverty Lines on the Basis of Local Perceptions of Poverty. A Quantitative Analysis of Qualitative Data from Participatory Wealth Ranking in Rural," World Development, Elsevier, vol. 35(2), pages 212-229, February.
    18. Alasdair Cohen & Michaela Saisana, 2014. "Quantifying the Qualitative: Eliciting Expert Input to Develop the Multidimensional Poverty Assessment Tool," Journal of Development Studies, Taylor & Francis Journals, vol. 50(1), pages 35-50, January.
    19. Deborah Naybor & Jessie P. H. Poon & Irene Casas, 2016. "Mobility Disadvantage and Livelihood Opportunities of Marginalized Widowed Women in Rural Uganda," Annals of the American Association of Geographers, Taylor & Francis Journals, vol. 106(2), pages 404-412, March.
    20. Linden McBride & Austin Nichols, 2018. "Retooling Poverty Targeting Using Out-of-Sample Validation and Machine Learning," The World Bank Economic Review, World Bank, vol. 32(3), pages 531-550.
    21. Laura Camfield & Gina Crivello & Martin Woodhead, 2009. "Wellbeing Research in Developing Countries: Reviewing the Role of Qualitative Methods," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 90(1), pages 5-31, January.
    22. Alison Woodcock & Laura Camfield & J. McGregor & Faith Martin, 2009. "Validation of the WeDQoL-Goals-Thailand Measure: Culture-Specific Individualised Quality of Life," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 94(1), pages 135-171, October.
    23. Naila Kabeer, 1999. "Resources, Agency, Achievements: Reflections on the Measurement of Women's Empowerment," Development and Change, International Institute of Social Studies, vol. 30(3), pages 435-464, July.
    24. Peterman, Amber & Schwab, Benjamin & Roy, Shalini & Hidrobo, Melissa & Gilligan, Daniel O., 2021. "Measuring women’s decisionmaking: Indicator choice and survey design experiments from cash and food transfer evaluations in Ecuador, Uganda and Yemen," World Development, Elsevier, vol. 141(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. McCarthy, Aine Seitz & Krause, Brooke, 2024. "Age and Agency: Evidence from a Women’s Empowerment Program in Tanzania," World Development, Elsevier, vol. 178(C).
    2. Augsburg, Britta & Malde, Bansi & Olorenshaw, Harriet & Wahhaj, Zaki, 2023. "To invest or not to invest in sanitation: The role of intra-household gender differences in perceptions and bargaining power," Journal of Development Economics, Elsevier, vol. 162(C).
    3. Katie Tavenner & Todd A. Crane, 2022. "Hitting the target and missing the point? On the risks of measuring women’s empowerment in agricultural development," Agriculture and Human Values, Springer;The Agriculture, Food, & Human Values Society (AFHVS), vol. 39(3), pages 849-857, September.
    4. Saha, Shree & Narayanan, Sudha, 2022. "A simplified measure of nutritional empowerment: Using machine learning to abbreviate the Women’s Empowerment in Nutrition Index (WENI)," World Development, Elsevier, vol. 154(C).
    5. Caroline Krafft & Ragui Assaad & Isabel Pastoor, 2021. "How Do Gender Norms Shape Education and Domestic Work Outcomes? The Case of Syrian Refugee Adolescents in Jordan," HiCN Working Papers 361, Households in Conflict Network.
    6. Vijayendra Rao, 2023. "Can Economics Become More Reflexive? Exploring the Potential of Mixed Methods," Springer Books, in: Ashwini Deshpande (ed.), Handbook on Economics of Discrimination and Affirmative Action, chapter 14, pages 323-349, Springer.
    7. Beltramo, Theresa P. & Calvi, Rossella & De Giorgi, Giacomo & Sarr, Ibrahima, 2023. "Child poverty among refugees," World Development, Elsevier, vol. 171(C).
    8. Ingvild Almås & Orazio Attanasio & Pamela Jervis, 2024. "Presidential Address: Economics and Measurement: New Measures to Model Decision Making," Econometrica, Econometric Society, vol. 92(4), pages 947-978, July.
    9. Bjorkegren, Dan & Blumenstock, Joshua & Knight, Samsun, 2022. "(Machine) Learning What Policies Value," CEPR Discussion Papers 17364, C.E.P.R. Discussion Papers.
    10. Bryan, Gharad & Karlan, Dean & Osman, Adam, 2024. "Big loans to small businesses: predicting winners and losers in an entrepreneurial lending experiment," LSE Research Online Documents on Economics 120637, London School of Economics and Political Science, LSE Library.
    11. Harris, J. Andrew & van der Windt, Peter, 2023. "Empowering women or increasing response bias? Experimental evidence from Congo," Journal of Development Economics, Elsevier, vol. 164(C).
    12. Ashwin,Julian & Rao,Vijayendra & Biradavolu,Monica Rao & Chhabra,Aditya & Haque,Arshia & Khan,Afsana Iffat & Krishnan,Nandini, 2022. "A Method to Scale-Up Interpretative Qualitative Analysis, with an Application toAspirations in Cox’s Bazaar, Bangladesh," Policy Research Working Paper Series 10046, The World Bank.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jayachandran, Seema & Biradavolu, Monica & Cooper, Jan, 2023. "Using machine learning and qualitative interviews to design a five-question survey module for women’s agency," World Development, Elsevier, vol. 161(C).
    2. Laszlo, Sonia & Grantham, Kate & Oskay, Ecem & Zhang, Tingting, 2020. "Grappling with the challenges of measuring women's economic empowerment in intrahousehold settings," World Development, Elsevier, vol. 132(C).
    3. Qanti, Sara Ratna & Peralta, Alexandra & Zeng, Di, 2021. "Social Norms and Perception on Women's Participation in Agricultural Decisions: The Case of West Java, Indonesia," 2021 Conference, August 17-31, 2021, Virtual 314984, International Association of Agricultural Economists.
    4. Peterman, Amber & Schwab, Benjamin & Roy, Shalini & Hidrobo, Melissa & Gilligan, Daniel O., 2021. "Measuring women’s decisionmaking: Indicator choice and survey design experiments from cash and food transfer evaluations in Ecuador, Uganda and Yemen," World Development, Elsevier, vol. 141(C).
    5. Lu Gram & Joanna Morrison & Jolene Skordis-Worrall, 2019. "Organising Concepts of ‘Women’s Empowerment’ for Measurement: A Typology," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 143(3), pages 1349-1376, June.
    6. Seymour, Greg & Peterman, Amber, 2018. "Context and measurement: An analysis of the relationship between intrahousehold decision making and autonomy," World Development, Elsevier, vol. 111(C), pages 97-112.
    7. Raj, Anita & Dey, Arnab & Rao, Namratha & Yore, Jennifer & McDougal, Lotus & Bhan, Nandita & Silverman, Jay G. & Hay, Katherine & Thomas, Edwin E. & Fotso, Jean Christophe & Lundgren, Rebecka, 2024. "The EMERGE framework to measure empowerment for health and development," Social Science & Medicine, Elsevier, vol. 351(S1).
    8. Janina Isabel Steinert & Lucie Dale Cluver & G. J. Melendez-Torres & Sebastian Vollmer, 2018. "One Size Fits All? The Validity of a Composite Poverty Index Across Urban and Rural Households in South Africa," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(1), pages 51-72, February.
    9. O'Hara, Corey & Clement, Floriane, 2018. "Power as agency: A critical reflection on the measurement of women’s empowerment in the development sector," World Development, Elsevier, vol. 106(C), pages 111-123.
    10. Alexandra Peralta, 2022. "The role of men and women in agriculture and agricultural decisions in Vanuatu," Asia and the Pacific Policy Studies, Wiley Blackwell, vol. 9(1), pages 59-80, January.
    11. Sojin Yu & Feinian Chen & Sonalde Desai, 2023. "Aligning household decision-making with work and education: A comparative analysis of women’s empowerment," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 48(19), pages 513-548.
    12. Sara Ratna Qanti & Alexandra Peralta & Di Zeng, 2022. "Social norms and perceptions drive women’s participation in agricultural decisions in West Java, Indonesia," Agriculture and Human Values, Springer;The Agriculture, Food, & Human Values Society (AFHVS), vol. 39(2), pages 645-662, June.
    13. Sharmistha Self & Richard Grabowski, 2013. "Female Autonomy In Rural North India: Impact Of Economic, Social, And Political Factors," Journal of Economic Development, Chung-Ang Unviersity, Department of Economics, vol. 38(1), pages 59-82, March.
    14. Giulia Ferrari, 2022. "What is wellbeing for rural South African women? Textual analysis of focus group discussion transcripts and implications for programme design and evaluation," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-15, December.
    15. Keetie Roelen, 2018. "Poor Children in Rich Households and Vice Versa: A Blurred Picture or Hidden Realities?," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 30(2), pages 320-341, April.
    16. Almås, Ingvild & Somville, Vincent & Vandewalle, Lore, 2020. "The Effect of Gender-Targeted Transfers: Experimental Evidence From India," Discussion Paper Series in Economics 16/2020, Norwegian School of Economics, Department of Economics.
    17. Nathan Fiala & Ana Garcia-Hernandez & Kritika Narula & Nishith Prakash, 2022. "Wheels of Change: Transforming Girls’ Lives with Bicycles," Working papers 2022-04, University of Connecticut, Department of Economics.
    18. Susana Martínez-Restrepo & Juliana Ramírez & Angélica Castillo & Laura Castrillón-Guerrero & Isabel Calero & Juliana Mejía & Lina Tafur, 2021. "El continuum de las violencias basadas en género en el contexto del conflicto armado colombiano y su relación con el empoderamiento económico de las sobrevivientes," Informes de Investigación 19447, Fedesarrollo.
    19. Erica Field & Rohini Pande & Natalia Rigol & Simone Schaner & Charity Troyer Moore, 2021. "On Her Own Account: How Strengthening Women's Financial Control Impacts Labor Supply and Gender Norms," American Economic Review, American Economic Association, vol. 111(7), pages 2342-2375, July.
    20. Panu Poutvaara & Maximilian Schwefer, 2018. "Husbands’ and wives’ diverging perceptions on who decides," ifo Working Paper Series 279, ifo Institute - Leibniz Institute for Economic Research at the University of Munich.

    More about this item

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • D13 - Microeconomics - - Household Behavior - - - Household Production and Intrahouse Allocation
    • J16 - Labor and Demographic Economics - - Demographic Economics - - - Economics of Gender; Non-labor Discrimination
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:28626. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.