IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v10y2018i11p4320-d184432.html
   My bibliography  Save this article

AES Impact Evaluation With Integrated Farm Data: Combining Statistical Matching and Propensity Score Matching

Author

Listed:
  • Riccardo D’Alberto

    (Department of Statistical Sciences “P. Fortunati”, Alma Mater Studiorum University of Bologna, Via delle Belle Arti 41, 40126 Bologna, Italy)

  • Matteo Zavalloni

    (Department of Agricultural and Food Sciences, Alma Mater Studiorum University of Bologna, Viale Fanin 50, 40127 Bologna, Italy)

  • Meri Raggi

    (Department of Statistical Sciences “P. Fortunati”, Alma Mater Studiorum University of Bologna, Via delle Belle Arti 41, 40126 Bologna, Italy)

  • Davide Viaggi

    (Department of Agricultural and Food Sciences, Alma Mater Studiorum University of Bologna, Viale Fanin 50, 40127 Bologna, Italy)

Abstract

A large share of the Common Agricultural Policy (CAP) is allocated to agri-environmental schemes (AESs), whose goal is to foster the provision of a wide range of environmental public goods. Despite this effort, little is known on the actual environmental and economic impact of the AESs, due to the non-experimental conditions of the assessment exercise and several data availability issues. The main objective of the paper is to explore the feasibility of combining the non-parametric statistical matching (SM) method and propensity score matching (PSM) counterfactual approach analysis and to test its usefulness and practicability on a case study represented by selected impacts of the AESs in Emilia-Romagna. The work hints at the potentialities of the combined use of SM and PSM as well as of the systematic collection of additional information to be included in EU-financed project surveys in order to enrich and complete data collected in the official statistics. The results show that the combination of the two methods enables us to enlarge and deepen the scope of counterfactual analysis applied to AESs. In a specific case study, AESs seem to reduce the amount of rent-in land and decrease the crop mix diversity.

Suggested Citation

  • Riccardo D’Alberto & Matteo Zavalloni & Meri Raggi & Davide Viaggi, 2018. "AES Impact Evaluation With Integrated Farm Data: Combining Statistical Matching and Propensity Score Matching," Sustainability, MDPI, vol. 10(11), pages 1-24, November.
  • Handle: RePEc:gam:jsusta:v:10:y:2018:i:11:p:4320-:d:184432
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/10/11/4320/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/10/11/4320/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Andrea Ichino & Fabrizia Mealli & Tommaso Nannicini, 2008. "From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 305-327.
    2. Andrea Pufahl & Christoph R. Weiss, 2009. "Evaluating the effects of farm programmes: results from propensity score matching," European Review of Agricultural Economics, Foundation for the European Review of Agricultural Economics, vol. 36(1), pages 79-101, March.
    3. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    4. Edi Defrancesco & Paola Gatto & Ford Runge & Samuele Trestini, 2008. "Factors Affecting Farmers’ Participation in Agri‐environmental Measures: A Northern Italian Perspective," Journal of Agricultural Economics, Wiley Blackwell, vol. 59(1), pages 114-131, February.
    5. Barbara Sianesi, 2004. "An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 133-155, February.
    6. Ratinger, T. & Medonos, Tomas & Hruška, M., 2013. "An Assessment of the Differentiated Effects of the Investment Support to Agricultural Modernisation: the Case of the Czech Republic," AGRIS on-line Papers in Economics and Informatics, Czech University of Life Sciences Prague, Faculty of Economics and Management, vol. 5(4), pages 1-12, December.
    7. Lori Lynch & Xiangping Liu, 2007. "Impact of Designated Preservation Areas on Rate of Preservation and Rate of Conversion: Preliminary Evidence," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 89(5), pages 1205-1210.
    8. Jerzy Michalek & Pavel Ciaian & d’Artis Kancs, 2014. "Capitalization of the Single Payment Scheme into Land Value: Generalized Propensity Score Evidence from the European Union," Land Economics, University of Wisconsin Press, vol. 90(2), pages 260-289.
    9. Linda Arata & Paolo Sckokai, 2016. "The Impact of Agri-environmental Schemes on Farm Performance in Five E.U. Member States: A DID-Matching Approach," Land Economics, University of Wisconsin Press, vol. 92(1), pages 167-186.
    10. Schilling, Brian J. & Attavanich, Witsanu & Sullivan, Kevin P. & Marxen, Lucas J., 2014. "Measuring the effect of farmland preservation on farm profitability," MPRA Paper 100122, University Library of Munich, Germany, revised Jun 2014.
    11. Datta, Nirupam, 2015. "Evaluating Impacts of Watershed Development Program on Agricultural Productivity, Income, and Livelihood in Bhalki Watershed of Bardhaman District, West Bengal," World Development, Elsevier, vol. 66(C), pages 443-456.
    12. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    13. Di Falco, Salvatore & Perrings, Charles, 2005. "Crop biodiversity, risk management and the implications of agricultural assistance," Ecological Economics, Elsevier, vol. 55(4), pages 459-466, December.
    14. Stefan Baumgärtner & Martin F. Quaas, 2010. "Managing increasing environmental risks through agrobiodiversity and agrienvironmental policies," Agricultural Economics, International Association of Agricultural Economists, vol. 41(5), pages 483-496, September.
    15. Arata, Linda & Donati, Michele & Sckokai, Paolo & Arfini, Filippo, 2017. "Incorporating risk in a positive mathematical programming framework: a dual approach," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 61(2), April.
    16. Jerzy Michalek, 2012. "Counterfactual impact evaluation of EU rural development programmes - Propensity Score Matching methodology applied to selected EU Member States. Volume 2: A regional approach," JRC Research Reports JRC72060, Joint Research Centre (Seville site).
    17. Black, Dan A. & Smith, J.A.Jeffrey A., 2004. "How robust is the evidence on the effects of college quality? Evidence from matching," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 99-124.
    18. Rubin, Donald B, 1986. "Statistical Matching Using File Concatenation with Adjusted Weights and Multiple Imputations," Journal of Business & Economic Statistics, American Statistical Association, vol. 4(1), pages 87-94, January.
    19. Alberto Abadie & David Drukker & Jane Leber Herr & Guido W. Imbens, 2004. "Implementing matching estimators for average treatment effects in Stata," Stata Journal, StataCorp LP, vol. 4(3), pages 290-311, September.
    20. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    21. Kirchweger, Stefan & Kantelhardt, Jochen, 2012. "Improving Farm Competitiveness through Farm-Investment Support: a Propensity Score Matching Approach," 131st Seminar, September 18-19, 2012, Prague, Czech Republic 135791, European Association of Agricultural Economists.
    22. Conti, Pier Luigi & Marella, Daniela & Scanu, Mauro, 2008. "Evaluation of matching noise for imputation techniques based on nonparametric local linear regression estimators," Computational Statistics & Data Analysis, Elsevier, vol. 53(2), pages 354-365, December.
    23. Paolo Sckokai & Daniele Moro, 2006. "Modeling the Reforms of the Common Agricultural Policy for Arable Crops under Uncertainty," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 88(1), pages 43-56.
    24. Michael W. Robbins & Sujit K. Ghosh & Joshua D. Habiger, 2013. "Imputation in High-Dimensional Economic Data as Applied to the Agricultural Resource Management Survey," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(501), pages 81-95, March.
    25. Giovanni Cerulli, 2010. "Modelling and Measuring the Effect of Public Subsidies on Business R&D: A Critical Review of the Econometric Literature," The Economic Record, The Economic Society of Australia, vol. 86(274), pages 421-449, September.
    26. Salvatore Di Falco & Charles Perrings, 2003. "Crop Genetic Diversity, Productivity and Stability of Agroecosystems. A Theoretical and Empirical Investigation," Scottish Journal of Political Economy, Scottish Economic Society, vol. 50(2), pages 207-216, May.
    27. Xiangping Liu & Lori Lynch, 2011. "Do Agricultural Land Preservation Programs Reduce Farmland Loss? Evidence from a Propensity Score Matching Estimator," Land Economics, University of Wisconsin Press, vol. 87(2), pages 183-201.
    28. Jūratė Jaraitė & Andrius Kažukauskas, 2012. "The Effect of Mandatory Agro-Environmental Policy on Farm Fertiliser and Pesticide Expenditure," Journal of Agricultural Economics, Wiley Blackwell, vol. 63(3), pages 656-676, September.
    29. Chihiro Udagawa & Ian Hodge & Mark Reader, 2014. "Farm Level Costs of Agri-environment Measures: The Impact of Entry Level Stewardship on Cereal Farm Incomes," Journal of Agricultural Economics, Wiley Blackwell, vol. 65(1), pages 212-233, January.
    30. Rebecca R. Andridge & Roderick J. A. Little, 2010. "A Review of Hot Deck Imputation for Survey Non‐response," International Statistical Review, International Statistical Institute, vol. 78(1), pages 40-64, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lajos Baráth & Imre Fertő & Štefan Bojnec, 2020. "The Effect of Investment, LFA and Agri‐environmental Subsidies on the Components of Total Factor Productivity: The Case of Slovenian Farms," Journal of Agricultural Economics, Wiley Blackwell, vol. 71(3), pages 853-876, September.
    2. Roggendorf, Wolfgang & Schwarze, Stefan, 2020. "Die Wirkung von Agrarumweltmaßnahmen auf betriebliche Stickstoffbilanzen – Empirische Ergebnisse aus Nordrhein-Westfalen," 60th Annual Conference, Halle/ Saale, Germany, September 23-25, 2020 305604, German Association of Agricultural Economists (GEWISOLA).
    3. Bartolini, Fabio & Vergamini, Daniele & Longhitano, Davide & Povellato, Andrea, 2021. "Do differential payments for agri-environment schemes affect the environmental benefits? A case study in the North-Eastern Italy," Land Use Policy, Elsevier, vol. 107(C).
    4. Michalek, Jerzy, 2022. "Environmental and farm impacts of the EU RDP agri-environmental measures: Evidence from Slovak regions," Land Use Policy, Elsevier, vol. 113(C).
    5. Roggendorf, Wolfgang & Schwarze, Stefan, 2020. "Die Wirkung von Agrarumweltmaßnahmen auf betriebliche Stickstoffbilanzen – Empirische Ergebnisse aus Nordrhein-Westfalen," 60th Annual Conference, Halle/ Saale, Germany, September 23-25, 2020 305604, German Association of Agricultural Economists (GEWISOLA).
    6. Fabio Bartolini & Daniele Vergamini, 2019. "Understanding the Spatial Agglomeration of Participation in Agri-Environmental Schemes: The Case of the Tuscany Region," Sustainability, MDPI, vol. 11(10), pages 1-18, May.
    7. Thang Quyet Nguyen & Nguyen Tan Huynh & Wen-Kai K. Hsu, 2021. "Estimate the Impact of Payments for Environmental Services on Local Livelihoods and Environment: An Application of Propensity Scores," SAGE Open, , vol. 11(3), pages 21582440211, August.
    8. Javier Castaño & Maria Blanco & Pilar Martinez, 2019. "Reviewing Counterfactual Analyses to Assess Impacts of EU Rural Development Programmes: What Lessons Can Be Learned from the 2007–2013 Ex-Post Evaluations?," Sustainability, MDPI, vol. 11(4), pages 1-22, February.
    9. Tsakiridis, Andreas & O’Donoghue, Cathal & Ryan, Mary & Cullen, Paula & Ó hUallacháin, Daire & Sheridan, Helen & Stout, Jane, 2022. "Examining the relationship between farmer participation in an agri-environment scheme and the quantity and quality of semi-natural habitats on Irish farms," Land Use Policy, Elsevier, vol. 120(C).
    10. Amer Ait Sidhoum & Carolin Canessa & Johannes Sauer, 2023. "Effects of agri‐environment schemes on farm‐level eco‐efficiency measures: Empirical evidence from EU countries," Journal of Agricultural Economics, Wiley Blackwell, vol. 74(2), pages 551-569, June.
    11. Cisilino, Federica & Bodini, Antonella & Zanoli, Agostina, 2019. "Rural development programs’ impact on environment: An ex-post evaluation of organic faming," Land Use Policy, Elsevier, vol. 85(C), pages 454-462.
    12. Tsakiridis, Andreas & O’Donoghue, Cathal & Ryan, Mary & Cullen, Paula & ÓhUallacháin, Daire & Sheridan, Helen & Stout, Jane, 2021. "Examining the relationship between farmer participation in an agri-environment scheme and the quantity and quality of semi-natural habitats on farms - An Irish case study," 2021 Conference, August 17-31, 2021, Virtual 315280, International Association of Agricultural Economists.
    13. D'Alberto, Riccardo & Zavalloni, Matteo & Raggi, Meri & Viaggi, Davide, 2021. "A Statistical Matching approach to reproduce the heterogeneity of willingness to pay in benefit transfer," Socio-Economic Planning Sciences, Elsevier, vol. 74(C).
    14. Eunji Choi & Jonghoon Park & Seongwoo Lee, 2020. "The Effect of the Comprehensive Rural Village Development Program on Farm Income in South Korea," Sustainability, MDPI, vol. 12(17), pages 1-23, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fertő, Imre & Bakucs, Zoltán & Varga, Ágnes, 2016. "Impact of EU subsidies on the of rural areas in Hungary," 160th Seminar, December 1-2, 2016, Warsaw, Poland 249826, European Association of Agricultural Economists.
    2. Zoltán Bakucs & Imre Fertő & Zsófia Benedek, 2019. "Success or Waste of Taxpayer Money? Impact Assessment of Rural Development Programs in Hungary," Sustainability, MDPI, vol. 11(7), pages 1-23, April.
    3. Michalek, Jerzy, 2022. "Environmental and farm impacts of the EU RDP agri-environmental measures: Evidence from Slovak regions," Land Use Policy, Elsevier, vol. 113(C).
    4. Lajos Baráth & Imre Fertő & Štefan Bojnec, 2018. "Are farms in less favored areas less efficient?," Agricultural Economics, International Association of Agricultural Economists, vol. 49(1), pages 3-12, January.
    5. Bakucs Zoltán & Fertő Imre, 2019. "Convergence or Divergence? Analysis of Regional Development Convergence in Hungary," Eastern European Countryside, Sciendo, vol. 25(1), pages 121-143, December.
    6. Lajos Baráth & Imre Fertő & Štefan Bojnec, 2020. "The Effect of Investment, LFA and Agri‐environmental Subsidies on the Components of Total Factor Productivity: The Case of Slovenian Farms," Journal of Agricultural Economics, Wiley Blackwell, vol. 71(3), pages 853-876, September.
    7. Roberto ESPOSTI, 2014. "To match, not to match, how to match: Estimating the farm-level impact of the CAP-first pillar reform (or: How to Apply Treatment-Effect Econometrics when the Real World is;a Mess)," Working Papers 403, Universita' Politecnica delle Marche (I), Dipartimento di Scienze Economiche e Sociali.
    8. Woodward, Richard T. & Newburn, David A. & Mezzatesta, Mariano, 2016. "Additionality and reverse crowding out for pollution offsets in water quality trading," Ecological Economics, Elsevier, vol. 128(C), pages 224-231.
    9. Kuhfuss, Laure & Subervie, Julie, 2018. "Do European Agri-environment Measures Help Reduce Herbicide Use? Evidence From Viticulture in France," Ecological Economics, Elsevier, vol. 149(C), pages 202-211.
    10. Arata, Linda & Hauschild, Sofia & Sckokai, Paolo, 2017. "Socio-economic impact of grape growing in North-eastern Brazil," 2017 Sixth AIEAA Conference, June 15-16, Piacenza, Italy 261264, Italian Association of Agricultural and Applied Economics (AIEAA).
    11. Arata, Linda & Sckokai, P., 2013. "Impact of Agri-environmental Schemes on Farm Performances in five EU Member States," 2013 Second Congress, June 6-7, 2013, Parma, Italy 149771, Italian Association of Agricultural and Applied Economics (AIEAA).
    12. Esposti, Roberto, 2015. "To match, not to matchm how to match: Estimating the farm-level impact of the 2005 CAP-first pillar reform," 2015 Conference, August 9-14, 2015, Milan, Italy 211625, International Association of Agricultural Economists.
    13. Arata, Linda & Hauschild, Sofia & Sckokai, Paolo, 2018. "Economic and social impact of grape growing in Northeastern Brazil," Bio-based and Applied Economics Journal, Italian Association of Agricultural and Applied Economics (AIEAA), vol. 6(3), May.
    14. Uehleke, Reinhard & Petrick, Martin & Hüttel, Silke, 2022. "Evaluations of agri-environmental schemes based on observational farm data: The importance of covariate selection," Land Use Policy, Elsevier, vol. 114(C).
    15. Bakucs, Z. & Ferto, I., 2018. "Analysis of Regional Development Convergence at Sub-National Level. The Case of Hungary," 2018 Conference, July 28-August 2, 2018, Vancouver, British Columbia 277230, International Association of Agricultural Economists.
    16. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.
    17. Tommaso Nannicini, 2007. "Simulation-based sensitivity analysis for matching estimators," Stata Journal, StataCorp LP, vol. 7(3), pages 334-350, September.
    18. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    19. Adewale H. Adenuga & Claire Jack & Austen Ashfield & Michael Wallace, 2021. "Assessing the Impact of Participatory Extension Programme Membership on Farm Business Performance in Northern Ireland," Agriculture, MDPI, vol. 11(10), pages 1-12, September.
    20. Tamini, Lota D., 2011. "A nonparametric analysis of the impact of agri-environmental advisory activities on best management practice adoption: A case study of Québec," Ecological Economics, Elsevier, vol. 70(7), pages 1363-1374, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:10:y:2018:i:11:p:4320-:d:184432. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.