IDEAS home Printed from https://ideas.repec.org/a/iwt/jounls/h048579.html

Impacts assessments without true baselines: assessing the relative effects of training on the performance of water user associations in southern Tajikistan

Author

Listed:
  • Balasubramanya, Soumya
  • Price, Joseph P. G.
  • Horbulyk, Theodore M.

Abstract

Conducting rigorous evaluations of whether the process of creating new institutions affects their performance of mandated duties presents several challenges. Not only is assignment to process often not random, but when the process of creating new institutions starts, outcomes and other performance-influencing covariates are not measurable because the yet-to-be created institutions are not functioning at baseline. This paper compares the performance of 74 ‘treated’ water user associations (WUAs) in Tajikistan that were created using a longer training process with 67 ‘control’ WUAs that were created using shorter training, to assess the impact of training on WUA performance of mandated duties. First, propensity scores were constructed to estimate the probabilities of being ‘treated’ by treatment status. These results guided the application of the difference-in-difference technique with right-hand side covariates in a context where field measures of outcomes and other performance-influencing covariates were made after the new institutions were created and functioning. The first measures were taken within 12–18 months of the new institutions being functional and the second measures were taken 24 months after the first. This choice of methods introduces a bias due to measurement error causing an underestimate of the treatment effects, while controlling for biases due to time-invariant and time-varying unobservables. An alternative method that only compared the differences in outcomes at a single point in time after the new institutions were created would have provided an inaccurate estimate of the effects of the intervention. This is a context in which methods such as synthetic controls are impossible to employ due to the nature of the intervention, other macroeconomic structural changes, and severe data restrictions. The methodology employed here generates evidence that, while biased toward generating an underestimate of effect, can still be useful and informative for policy and management purposes, and for evaluating the impact of process on the functioning of new institutions in transition settings.

Suggested Citation

  • Balasubramanya, Soumya & Price, Joseph P. G. & Horbulyk, Theodore M., 2018. "Impacts assessments without true baselines: assessing the relative effects of training on the performance of water user associations in southern Tajikistan," Papers published in Journals (Open Access), International Water Management Institute, pages 4(3):1-28.
    • Handle: RePEc:iwt:jounls:h048579
    DOI: 10.1142/S2382624X18500078
    as

    Download full text from publisher

    File URL: https://www.worldscientific.com/doi/pdf/10.1142/S2382624X18500078
    Download Restriction: no
    File URL: https://libkey.io/10.1142/S2382624X18500078?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Meinzen-Dick, Ruth & Raju, K. V. & Gulati, Ashok, 2002. "What Affects Organization and Collective Action for Managing Resources? Evidence from Canal Irrigation Systems in India," World Development, Elsevier, vol. 30(4), pages 649-666, April.
    2. Stergios Skaperdas, 2001. "The political economy of organized crime: providing protection when the state does not," Economics of Governance, Springer, vol. 2(3), pages 173-202, November.
    3. Huang, Qiuqiong & Wang, Jinxia & Easter, K. William & Rozelle, Scott, 2010. "Empirical assessment of water management institutions in northern China," Agricultural Water Management, Elsevier, vol. 98(2), pages 361-369, December.
    4. Nagrah, Aatika & Chaudhry, Anita M. & Giordano, Mark, 2016. "Collective Action in Decentralized Irrigation Systems: Evidence from Pakistan," World Development, Elsevier, vol. 84(C), pages 282-298.
    5. Denyer Willis, Graham & Mota Prado, Mariana, 2014. "Process and Pattern in Institutional Reforms: A Case Study of the Police Pacifying Units (UPPs) in Brazil," World Development, Elsevier, vol. 64(C), pages 232-242.
    6. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    7. Fischer, Maria-Elisabeth & Irlenbusch, Bernd & Sadrieh, Abdolkarim, 2004. "An intergenerational common pool resource experiment," Journal of Environmental Economics and Management, Elsevier, vol. 48(2), pages 811-836, September.
    8. Theesfeld, Insa, 2004. "Constraints on Collective Action in a Transitional Economy: The Case of Bulgaria's Irrigation Sector," World Development, Elsevier, vol. 32(2), pages 251-271, February.
    9. Mustafa, Daanish & Altz-Stamm, Amelia & Scott, Laura Mapstone, 2016. "Water User Associations and the Politics of Water in Jordan," World Development, Elsevier, vol. 79(C), pages 164-176.
    10. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    11. Kazbekov, Jusipbek & Abdullaev, Iskandar & Manthrithilake, Herath & Qureshi, Asad & Jumaboev, Kakhramon, 2009. "Evaluating planning and delivery performance of Water User Associations (WUAs) in Osh Province, Kyrgyzstan," Agricultural Water Management, Elsevier, vol. 96(8), pages 1259-1267, August.
    12. Ricks, Jacob I., 2016. "Building Participatory Organizations for Common Pool Resource Management: Water User Group Promotion in Indonesia," World Development, Elsevier, vol. 77(C), pages 34-47.
    13. Yap-Salinas, L. H., 1994. "Converging factors in the successful transfer of irrigation management responsibilities to water users associations in the Dominican Republic," Conference Papers h043535, International Water Management Institute.
    14. Tellez Foster, Edgar & Rapoport, Amnon & Dinar, Ariel, 2017. "Groundwater and electricity consumption under alternative subsidies: Evidence from laboratory experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 68(C), pages 41-52.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Balasubramanya, Soumya, 2019. "Effects of training duration and the role of gender on farm participation in water user associations in Southern Tajikistan: Implications for irrigation management," Agricultural Water Management, Elsevier, vol. 216(C), pages 1-11.
    2. Aarnoudse, E. & Closas, Alvar & Lefore, Nicole, 2018. "Water user associations: a review of approaches and alternative management options for Sub-Saharan Africa," IWMI Working Papers H048782, International Water Management Institute.
    3. Yemane Asmelash Gebremariam & Joost Dessein & Beneberu Assefa Wondimagegnhu & Mark Breusers & Lutgart Lenaerts & Enyew Adgo & Zemen Ayalew & Amare Sewenet Minale & Jan Nyssen, 2021. "Determinants of Farmers’ Level of Interaction with Agricultural Extension Agencies in Northwest Ethiopia," Sustainability, MDPI, vol. 13(6), pages 1-24, March.
    4. Ghimire, Tinusha & Harou, Aurélie P. & Balasubramanya, Soumya, 2023. "Migration, gender labor division and food insecurity in Tajikistan," Food Policy, Elsevier, vol. 116(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Balasubramanya, Soumya, 2019. "Effects of training duration and the role of gender on farm participation in water user associations in Southern Tajikistan: Implications for irrigation management," Agricultural Water Management, Elsevier, vol. 216(C), pages 1-11.
    2. Danqiu Cao & Yahua Wang & Liangzhen Zang, 2023. "The Effects of Land Reallocation on Irrigation Collective Action: Moderating Effects of Informal Organizations and Leadership," Land, MDPI, vol. 12(7), pages 1-15, July.
    3. Paolo Pinotti, 2012. "The economic costs of organized crime: evidence from southern Italy," Temi di discussione (Economic working papers) 868, Bank of Italy, Economic Research and International Relations Area.
    4. Liangzhen Zang & Yahua Wang & Yiqing Su, 2021. "Does Farmland Scale Management Promote Rural Collective Action? An Empirical Study of Canal Irrigation Systems in China," Land, MDPI, vol. 10(11), pages 1-25, November.
    5. Takayama, Taisuke & Matsuda, Hirotaka & Nakatani, Tomoaki, 2018. "The determinants of collective action in irrigation management systems: Evidence from rural communities in Japan," Agricultural Water Management, Elsevier, vol. 206(C), pages 113-123.
    6. Wang, Yahua & Huan, Meili, 2023. "The effects of socialized agricultural services on rural collective action in the irrigation commons: Evidence from China," Agricultural Water Management, Elsevier, vol. 289(C).
    7. Masahiro Kato & Akari Ohda, 2023. "Asymptotically Unbiased Synthetic Control Methods by Moment Matching," Papers 2307.11127, arXiv.org, revised Nov 2025.
    8. Qiuyue Xia & Lu Li & Jie Dong & Bin Zhang, 2021. "Reduction Effect and Mechanism Analysis of Carbon Trading Policy on Carbon Emissions from Land Use," Sustainability, MDPI, vol. 13(17), pages 1-22, August.
    9. Orhan Erdem & Sukran Erdem & Kelly Monson, 2023. "Children, vaccines, and financial incentives," International Journal of Health Economics and Management, Springer, vol. 23(4), pages 537-552, December.
    10. Becker, Sascha O. & Heblich, Stephan & Sturm, Daniel M., 2021. "The impact of public employment: Evidence from Bonn," Journal of Urban Economics, Elsevier, vol. 122(C).
    11. Jarrett, Uchechukwu & Mohaddes, Kamiar & Mohtadi, Hamid, 2019. "Oil price volatility, financial institutions and economic growth," Energy Policy, Elsevier, vol. 126(C), pages 131-144.
    12. Sarah Bohn & Magnus Lofstrom & Steven Raphael, 2014. "Did the 2007 Legal Arizona Workers Act Reduce the State's Unauthorized Immigrant Population?," The Review of Economics and Statistics, MIT Press, vol. 96(2), pages 258-269, May.
    13. Olper, Alessandro & Curzi, Daniele & Swinnen, Johan, 2018. "Trade liberalization and child mortality: A Synthetic Control Method," World Development, Elsevier, vol. 110(C), pages 394-410.
    14. Rickman, Dan S. & Wang, Hongbo, 2018. "Two tales of two U.S. states: Regional fiscal austerity and economic performance," Regional Science and Urban Economics, Elsevier, vol. 68(C), pages 46-55.
    15. Cruz A. Echevarría & Serhat Hasancebi & Javier García-Enríquez, 2022. "Economic Effects of Macao’s Integration with Mainland China: A Causal Inference Study," Journal of Economic Integration, Center for Economic Integration, Sejong University, vol. 37(2), pages 179-215.
    16. Nicolaj N. Mühlbach, 2020. "Tree-based Synthetic Control Methods: Consequences of moving the US Embassy," CREATES Research Papers 2020-04, Department of Economics and Business Economics, Aarhus University.
    17. Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2020. "Ensuring Stability, Accuracy and Meaningfulness in Synthetic Control Methods: The Regularized SHAP-Distance Method," IREA Working Papers 202005, University of Barcelona, Research Institute of Applied Economics, revised Apr 2020.
    18. Gugler, Klaus & Szücs, Florian, 2016. "Merger externalities in oligopolistic markets," International Journal of Industrial Organization, Elsevier, vol. 47(C), pages 230-254.
    19. Eliason, Paul & Lutz, Byron, 2018. "Can fiscal rules constrain the size of government? An analysis of the “crown jewel” of tax and expenditure limitations," Journal of Public Economics, Elsevier, vol. 166(C), pages 115-144.
    20. Federico Crudu & Roberta Di Stefano & Giovanni Mellace & Silvia Tiezzi, 2022. "The Gray Zone," Department of Economics University of Siena 874, Department of Economics, University of Siena.

    More about this item

    Keywords

    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iwt:jounls:h048579. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chandima Gunadasa (email available below). General contact details of provider: https://edirc.repec.org/data/iwmiclk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.