IDEAS home Printed from https://ideas.repec.org/a/wsi/wepxxx/v04y2018i03ns2382624x18500078.html
   My bibliography  Save this article

Impacts Assessments without True Baselines: Assessing the Relative Effects of Training on the Performance of Water User Associations in Southern Tajikistan

Author

Listed:
  • Soumya Balasubramanya

    (International Water Management Institute, 127 Sunil Mawatha, Pelawatte, Sri Lanka)

  • Joseph P. G. Price

    (International Water Management Institute, 127 Sunil Mawatha, Pelawatte, Sri Lanka)

  • Theodore M. Horbulyk

    (International Water Management Institute, 127 Sunil Mawatha, Pelawatte, Sri Lanka)

Abstract

Conducting rigorous evaluations of whether the process of creating new institutions affects their performance of mandated duties presents several challenges. Not only is assignment to process often not random, but when the process of creating new institutions starts, outcomes and other performance-influencing covariates are not measurable because the yet-to-be created institutions are not functioning at baseline. This paper compares the performance of 74 ‘treated’ water user associations (WUAs) in Tajikistan that were created using a longer training process with 67 ‘control’ WUAs that were created using shorter training, to assess the impact of training on WUA performance of mandated duties. First, propensity scores were constructed to estimate the probabilities of being ‘treated’ by treatment status. These results guided the application of the difference-in-difference technique with right-hand side covariates in a context where field measures of outcomes and other performance-influencing covariates were made after the new institutions were created and functioning. The first measures were taken within 12–18 months of the new institutions being functional and the second measures were taken 24 months after the first. This choice of methods introduces a bias due to measurement error causing an underestimate of the treatment effects, while controlling for biases due to time-invariant and time-varying unobservables. An alternative method that only compared the differences in outcomes at a single point in time after the new institutions were created would have provided an inaccurate estimate of the effects of the intervention. This is a context in which methods such as synthetic controls are impossible to employ due to the nature of the intervention, other macroeconomic structural changes, and severe data restrictions. The methodology employed here generates evidence that, while biased toward generating an underestimate of effect, can still be useful and informative for policy and management purposes, and for evaluating the impact of process on the functioning of new institutions in transition settings.

Suggested Citation

  • Soumya Balasubramanya & Joseph P. G. Price & Theodore M. Horbulyk, 2018. "Impacts Assessments without True Baselines: Assessing the Relative Effects of Training on the Performance of Water User Associations in Southern Tajikistan," Water Economics and Policy (WEP), World Scientific Publishing Co. Pte. Ltd., vol. 4(03), pages 1-28, July.
    • Handle: RePEc:wsi:wepxxx:v:04:y:2018:i:03:n:s2382624x18500078
    DOI: 10.1142/S2382624X18500078
    as

    Download full text from publisher

    File URL: http://www.worldscientific.com/doi/abs/10.1142/S2382624X18500078
    Download Restriction: Access to full text is restricted to subscribers
    File URL: https://libkey.io/10.1142/S2382624X18500078?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Huang, Qiuqiong & Wang, Jinxia & Easter, K. William & Rozelle, Scott, 2010. "Empirical assessment of water management institutions in northern China," Agricultural Water Management, Elsevier, vol. 98(2), pages 361-369, December.
    2. Nagrah, Aatika & Chaudhry, Anita M. & Giordano, Mark, 2016. "Collective Action in Decentralized Irrigation Systems: Evidence from Pakistan," World Development, Elsevier, vol. 84(C), pages 282-298.
    3. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    4. Theesfeld, Insa, 2004. "Constraints on Collective Action in a Transitional Economy: The Case of Bulgaria's Irrigation Sector," World Development, Elsevier, vol. 32(2), pages 251-271, February.
    5. Mustafa, Daanish & Altz-Stamm, Amelia & Scott, Laura Mapstone, 2016. "Water User Associations and the Politics of Water in Jordan," World Development, Elsevier, vol. 79(C), pages 164-176.
    6. Kazbekov, Jusipbek & Abdullaev, Iskandar & Manthrithilake, Herath & Qureshi, Asad & Jumaboev, Kakhramon, 2009. "Evaluating planning and delivery performance of Water User Associations (WUAs) in Osh Province, Kyrgyzstan," Agricultural Water Management, Elsevier, vol. 96(8), pages 1259-1267, August.
    7. Ricks, Jacob I., 2016. "Building Participatory Organizations for Common Pool Resource Management: Water User Group Promotion in Indonesia," World Development, Elsevier, vol. 77(C), pages 34-47.
    8. Tellez Foster, Edgar & Rapoport, Amnon & Dinar, Ariel, 2017. "Groundwater and electricity consumption under alternative subsidies: Evidence from laboratory experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 68(C), pages 41-52.
    9. Meinzen-Dick, Ruth & Raju, K. V. & Gulati, Ashok, 2002. "What Affects Organization and Collective Action for Managing Resources? Evidence from Canal Irrigation Systems in India," World Development, Elsevier, vol. 30(4), pages 649-666, April.
    10. Stergios Skaperdas, 2001. "The political economy of organized crime: providing protection when the state does not," Economics of Governance, Springer, vol. 2(3), pages 173-202, November.
    11. Denyer Willis, Graham & Mota Prado, Mariana, 2014. "Process and Pattern in Institutional Reforms: A Case Study of the Police Pacifying Units (UPPs) in Brazil," World Development, Elsevier, vol. 64(C), pages 232-242.
    12. Fischer, Maria-Elisabeth & Irlenbusch, Bernd & Sadrieh, Abdolkarim, 2004. "An intergenerational common pool resource experiment," Journal of Environmental Economics and Management, Elsevier, vol. 48(2), pages 811-836, September.
    13. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    14. Yap-Salinas, L. H., 1994. "Converging factors in the successful transfer of irrigation management responsibilities to water users associations in the Dominican Republic," Conference Papers h043535, International Water Management Institute.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aarnoudse, E. & Closas, Alvar & Lefore, Nicole, 2018. "Water user associations: a review of approaches and alternative management options for Sub-Saharan Africa," IWMI Working Papers H048782, International Water Management Institute.
    2. Yemane Asmelash Gebremariam & Joost Dessein & Beneberu Assefa Wondimagegnhu & Mark Breusers & Lutgart Lenaerts & Enyew Adgo & Zemen Ayalew & Amare Sewenet Minale & Jan Nyssen, 2021. "Determinants of Farmers’ Level of Interaction with Agricultural Extension Agencies in Northwest Ethiopia," Sustainability, MDPI, vol. 13(6), pages 1-24, March.
    3. Balasubramanya, Soumya, 2019. "Effects of training duration and the role of gender on farm participation in water user associations in Southern Tajikistan: Implications for irrigation management," Agricultural Water Management, Elsevier, vol. 216(C), pages 1-11.
    4. Ghimire, Tinusha & Harou, Aurélie P. & Balasubramanya, Soumya, 2023. "Migration, gender labor division and food insecurity in Tajikistan," Food Policy, Elsevier, vol. 116(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Balasubramanya, Soumya, 2019. "Effects of training duration and the role of gender on farm participation in water user associations in Southern Tajikistan: Implications for irrigation management," Agricultural Water Management, Elsevier, vol. 216(C), pages 1-11.
    2. Danqiu Cao & Yahua Wang & Liangzhen Zang, 2023. "The Effects of Land Reallocation on Irrigation Collective Action: Moderating Effects of Informal Organizations and Leadership," Land, MDPI, vol. 12(7), pages 1-15, July.
    3. Paolo Pinotti, 2012. "The economic costs of organized crime: evidence from southern Italy," Temi di discussione (Economic working papers) 868, Bank of Italy, Economic Research and International Relations Area.
    4. Liangzhen Zang & Yahua Wang & Yiqing Su, 2021. "Does Farmland Scale Management Promote Rural Collective Action? An Empirical Study of Canal Irrigation Systems in China," Land, MDPI, vol. 10(11), pages 1-25, November.
    5. Takayama, Taisuke & Matsuda, Hirotaka & Nakatani, Tomoaki, 2018. "The determinants of collective action in irrigation management systems: Evidence from rural communities in Japan," Agricultural Water Management, Elsevier, vol. 206(C), pages 113-123.
    6. Qiuyue Xia & Lu Li & Jie Dong & Bin Zhang, 2021. "Reduction Effect and Mechanism Analysis of Carbon Trading Policy on Carbon Emissions from Land Use," Sustainability, MDPI, vol. 13(17), pages 1-22, August.
    7. Echevarría, Cruz A. & Hasancebi, Serhat & García-Enríquez, Javier, 2022. "Economic Effects of Macao’s Integration with Mainland China: A Causal Inference Study," Journal of Economic Integration, Center for Economic Integration, Sejong University, vol. 37(2), pages 179-215.
    8. Nicolaj N. Mühlbach, 2020. "Tree-based Synthetic Control Methods: Consequences of moving the US Embassy," CREATES Research Papers 2020-04, Department of Economics and Business Economics, Aarhus University.
    9. Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2020. "Ensuring Stability, Accuracy and Meaningfulness in Synthetic Control Methods: The Regularized SHAP-Distance Method," IREA Working Papers 202005, University of Barcelona, Research Institute of Applied Economics, revised Apr 2020.
    10. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    11. Foellmi, Reto & Martínez, Isabel Z., 2014. "Volatile Top Income Shares in Switzerland? Reassessing the Evolution Between 1981 and 2009," CEPR Discussion Papers 10006, C.E.P.R. Discussion Papers.
    12. Mustapha Douch & Terence Huw Edwards, 2022. "The bilateral trade effects of announcement shocks: Brexit as a natural field experiment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(2), pages 305-329, March.
    13. Manuel Funke & Moritz Schularick & Christoph Trebesch, 2023. "Populist Leaders and the Economy," American Economic Review, American Economic Association, vol. 113(12), pages 3249-3288, December.
    14. Maximiliano Marzetti & Rok Spruk, 2023. "Long-Term Economic Effects of Populist Legal Reforms: Evidence from Argentina," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 65(1), pages 60-95, March.
    15. De los Santos, Babur & Kim, In Kyung & Lubensky, Dmitry, 2018. "Do MSRPs decrease prices?," International Journal of Industrial Organization, Elsevier, vol. 59(C), pages 429-457.
      • Babur De los Santos & In Kyung Kim & Dmitry Lubensky, 2013. "Do MSRPs Decrease Prices?," Working Papers 2013-13, Indiana University, Kelley School of Business, Department of Business Economics and Public Policy.
    16. Stefano Costalli & Luigi Moretti & Costantino Pischedda, 2014. "The Economic Costs of Civil War: Synthetic Counterfactual Evidence and the Effects of Ethnic Fractionalization," HiCN Working Papers 184, Households in Conflict Network.
    17. Matthias Krapf & David Staubli, 2020. "The Corporate Elasticity of Taxable Income: Event Study Evidence from Switzerland," CESifo Working Paper Series 8715, CESifo.
    18. Chiara Cavaglia & Sandra McNally & Henry G. Overman, 2020. "Devolving Skills: The Case of the Apprenticeship Grant for Employers," Fiscal Studies, John Wiley & Sons, vol. 41(4), pages 829-849, December.
    19. Corral, Leonardo R. & Schling, Maja, 2017. "The impact of shoreline stabilization on economic growth in small island developing states," Journal of Environmental Economics and Management, Elsevier, vol. 86(C), pages 210-228.
    20. Nimonka Bayale & Pouwemdéou Tchila & Jacques‐Patrick Arnold Yao & Honoré Tenakoua, 2022. "Do tax administration reforms improve tax revenue performance in Togo? Empirical insights from experimental approaches," South African Journal of Economics, Economic Society of South Africa, vol. 90(2), pages 196-213, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wsi:wepxxx:v:04:y:2018:i:03:n:s2382624x18500078. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tai Tone Lim (email available below). General contact details of provider: http://www.worldscinet.com/wep/wep.shtml .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.