IDEAS home Printed from https://ideas.repec.org/a/kap/enreec/v55y2013i3p357-386.html
   My bibliography  Save this article

Due Diligence in Meta-analyses to Support Benefit Transfers

Author

Listed:
  • Kevin Boyle
  • Christopher Parmeter
  • Brent Boehlert
  • Robert Paterson

Abstract

Meta-analyses are becoming a popular tool for supporting benefit transfers, but the availability of studies is a direct consequence of policy issues, research funding, and investigator interest. We investigate fragility versus robustness of the meta-equation by considering sample selection, removing one observation or study at a time with replacement, and removing/adding regressors. Several key variables are found to be robust, strengthening the argument for their use in policy prescriptions. The key insights are that these methods can be used to parse meta-data to identify the most appropriate set(s) of observations and regressors to support literature evaluations, benefit transfers and other practical applications using statical summaries of empirical data. Copyright Springer Science+Business Media Dordrecht 2013

Suggested Citation

  • Kevin Boyle & Christopher Parmeter & Brent Boehlert & Robert Paterson, 2013. "Due Diligence in Meta-analyses to Support Benefit Transfers," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 55(3), pages 357-386, July.
    • Handle: RePEc:kap:enreec:v:55:y:2013:i:3:p:357-386
    DOI: 10.1007/s10640-012-9630-y
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10640-012-9630-y
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10640-012-9630-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ryan Stapler & Robert Johnston, 2009. "Meta-Analysis, Benefit Transfer, and Methodological Covariates: Implications for Transfer Error," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 42(2), pages 227-246, February.
    2. Rosenberger, Randall S. & Stanley, Tom D., 2006. "Measurement, generalization, and publication: Sources of error in benefit transfers and their management," Ecological Economics, Elsevier, vol. 60(2), pages 372-378, December.
    3. Smith, V. Kerry & Osborne, Laura L., 1996. "Do Contingent Valuation Estimates Pass a "Scope" Test? A Meta-analysis," Journal of Environmental Economics and Management, Elsevier, vol. 31(3), pages 287-301, November.
    4. Wooldridge, Jeffrey M., 1995. "Selection corrections for panel data models under conditional mean independence assumptions," Journal of Econometrics, Elsevier, vol. 68(1), pages 115-132, July.
    5. Racine, Jeff, 1997. "Feasible Cross-Validatory Model Selection for General Stationary Processes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 12(2), pages 169-179, March-Apr.
    6. Levine, Ross & Renelt, David, 1992. "A Sensitivity Analysis of Cross-Country Growth Regressions," American Economic Review, American Economic Association, vol. 82(4), pages 942-963, September.
    7. Kaul, Sapna & Boyle, Kevin J. & Kuminoff, Nicolai V. & Parmeter, Christopher F. & Pope, Jaren C., 2013. "What can we learn from benefit transfer errors? Evidence from 20 years of research on convergent validity," Journal of Environmental Economics and Management, Elsevier, vol. 66(1), pages 90-104.
    8. Semykina, Anastasia & Wooldridge, Jeffrey M., 2010. "Estimating panel data models in the presence of endogeneity and selection," Journal of Econometrics, Elsevier, vol. 157(2), pages 375-380, August.
    9. Lindhjem, Henrik & Navrud, Ståle, 2008. "How reliable are meta-analyses for international benefit transfers?," Ecological Economics, Elsevier, vol. 66(2-3), pages 425-435, June.
    10. John C. Bergstrom & Kevin J. Boyle & Gregory L. Poe (ed.), 2001. "The Economic Value of Water Quality," Books, Edward Elgar Publishing, number 1723.
    11. Ram Shrestha & John Loomis, 2003. "Meta-Analytic Benefit Transfer of Outdoor Recreation Economic Values: Testing Out-of-Sample Convergent Validity," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 25(1), pages 79-100, May.
    12. Klaus Moeltner & Richard Woodward, 2009. "Meta-Functional Benefit Transfer for Wetland Valuation: Making the Most of Small Samples," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 42(1), pages 89-108, January.
    13. Richardson, Leslie & Loomis, John, 2009. "The total economic value of threatened, endangered and rare species: An updated meta-analysis," Ecological Economics, Elsevier, vol. 68(5), pages 1535-1548, March.
    14. Robert J. Johnston & Elena Y. Besedin & Richard Iovanna & Christopher J. Miller & Ryan F. Wardwell & Matthew H. Ranson, 2005. "Systematic Variation in Willingness to Pay for Aquatic Resource Improvements and Implications for Benefit Transfer: A Meta‐Analysis," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 53(2‐3), pages 221-248, June.
    15. Hoehn, John P., 2006. "Methods to address selection effects in the meta regression and transfer of ecosystem values," Ecological Economics, Elsevier, vol. 60(2), pages 389-398, December.
    16. Roy Brouwer & Frank Spaninks, 1999. "The Validity of Environmental Benefits Transfer: Further Empirical Testing," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 14(1), pages 95-117, July.
    17. Woodward, Richard T. & Wui, Yong-Suhk, 2001. "The economic value of wetland services: a meta-analysis," Ecological Economics, Elsevier, vol. 37(2), pages 257-270, May.
    18. T. D. Stanley, 2008. "Meta‐Regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(1), pages 103-127, February.
    19. Carson, R.T. & Mitchell, R.C. & Hanemann, W.M. & Kopp, R.J. & Presser, S. & Ruud, P.A., 1992. "A Contingent Valuation Study of Lost Passive Use Values Resulting From the Exxon Valdez Oil Spill," MPRA Paper 6984, University Library of Munich, Germany.
    20. Smith, V Kerry & Huang, Ju-Chin, 1995. "Can Markets Value Air Quality? A Meta-analysis of Hedonic Property Value Models," Journal of Political Economy, University of Chicago Press, vol. 103(1), pages 209-227, February.
    21. Tommaso Proietti, 2003. "Leave‐K‐Out Diagnostics In State‐Space Models," Journal of Time Series Analysis, Wiley Blackwell, vol. 24(2), pages 221-236, March.
    22. Bergstrom, John C. & Taylor, Laura O., 2006. "Using meta-analysis for benefits transfer: Theory and practice," Ecological Economics, Elsevier, vol. 60(2), pages 351-360, December.
    23. Kirchhoff, Stefanie & Colby, Bonnie G. & LaFrance, Jeffrey T., 1997. "Evaluating the Performance of Benefit Transfer: An Empirical Inquiry," Journal of Environmental Economics and Management, Elsevier, vol. 33(1), pages 75-93, May.
    24. Randall S. Rosenberger & Robert J. Johnston, 2009. "Selection Effects in Meta-Analysis and Benefit Transfer: Avoiding Unintended Consequences," Land Economics, University of Wisconsin Press, vol. 85(3), pages 410-428.
    25. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    26. Ian J. Bateman & Andrew P. Jones, 2003. "Contrasting Conventional with Multi-Level Modeling Approaches to Meta-Analysis: Expectation Consistency in U.K. Woodland Recreation Values," Land Economics, University of Wisconsin Press, vol. 79(2), pages 235-258.
    27. Robert J. Johnston & Randall S. Rosenberger, 2010. "Methods, Trends And Controversies In Contemporary Benefit Transfer," Journal of Economic Surveys, Wiley Blackwell, vol. 24(3), pages 479-510, July.
    28. Kevin J. Boyle & Nicolai V. Kuminoff & Christopher F. Parmeter & Jaren C. Pope, 2009. "Necessary Conditions for Valid Benefit Transfers," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 91(5), pages 1328-1334.
    29. Jon Nelson & Peter Kennedy, 2009. "The Use (and Abuse) of Meta-Analysis in Environmental and Natural Resource Economics: An Assessment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 42(3), pages 345-377, March.
    30. V. Kerry Smith & Yoshiaki Kaoru, 1990. "Signals or Noise? Explaining the Variation in Recreation Benefit Estimates," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 72(2), pages 419-433.
    31. Moeltner, Klaus & Boyle, Kevin J. & Paterson, Robert W., 2007. "Meta-analysis and benefit transfer for resource valuation-addressing classical challenges with Bayesian modeling," Journal of Environmental Economics and Management, Elsevier, vol. 53(2), pages 250-269, March.
    32. Van Houtven, George & Powers, John & Pattanayak, Subhrendu K., 2007. "Valuing water quality improvements in the United States using meta-analysis: Is the glass half-full or half-empty for national policy analysis?," Resource and Energy Economics, Elsevier, vol. 29(3), pages 206-228, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Robert J. Johnston & Elena Y. Besedin & Benedict M. Holland, 2019. "Modeling Distance Decay Within Valuation Meta-Analysis," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 72(3), pages 657-690, March.
    2. Van Houtven, George L. & Pattanayak, Subhrendu K. & Usmani, Faraz & Yang, Jui-Chen, 2017. "What are Households Willing to Pay for Improved Water Access? Results from a Meta-Analysis," Ecological Economics, Elsevier, vol. 136(C), pages 126-135.
    3. Heather Klemick & Charles Griffiths & Dennis Guignet & Patrick Walsh, 2018. "Improving Water Quality in an Iconic Estuary: An Internal Meta-analysis of Property Value Impacts Around the Chesapeake Bay," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 69(2), pages 265-292, February.
    4. Heather Klemick & Charles Griffiths & Dennis Guignet & Patrick Walsh, 2015. "Explaining Variation in the Value of Chesapeake Bay Water Quality Using Internal Meta-analysis," NCEE Working Paper Series 201504, National Center for Environmental Economics, U.S. Environmental Protection Agency, revised Nov 2015.
    5. Marisa J. Mazzotta & Elena Besedin & Ann E. Speers, 2014. "A Meta-Analysis of Hedonic Studies to Assess the Property Value Effects of Low Impact Development," Resources, MDPI, vol. 3(1), pages 1-31, January.
    6. Hermine Vedogbeton & Robert J. Johnston, 2020. "Commodity Consistent Meta-Analysis of Wetland Values: An Illustration for Coastal Marsh Habitat," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 75(4), pages 835-865, April.
    7. De Salvo, Maria & Signorello, Giovanni, 2015. "Non-market valuation of recreational services in Italy: A meta-analysis," Ecosystem Services, Elsevier, vol. 16(C), pages 47-62.
    8. Fitzpatrick, Luke & Parmeter, Christopher F. & Agar, Juan, 2017. "Threshold Effects in Meta-Analyses With Application to Benefit Transfer for Coral Reef Valuation," Ecological Economics, Elsevier, vol. 133(C), pages 74-85.
    9. Newbold, Stephen C. & Johnston, Robert J., 2020. "Valuing non-market valuation studies using meta-analysis: A demonstration using estimates of willingness-to-pay for water quality improvements," Journal of Environmental Economics and Management, Elsevier, vol. 104(C).
    10. Kevin J. Boyle & Christopher F. Parmeter, 2017. "Benefit Transfer for Ecosystem Services," Working Papers 2017-07, University of Miami, Department of Economics.
    11. Robert J. Johnston & Kevin J. Boyle & Maria L. Loureiro & Ståle Navrud & John Rolfe, 2021. "Guidance to Enhance the Validity and Credibility of Environmental Benefit Transfers," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 79(3), pages 575-624, July.
    12. Marvin Schütt, 2021. "Systematic Variation in Waste Site Effects on Residential Property Values: A Meta-Regression Analysis and Benefit Transfer," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 78(3), pages 381-416, March.
    13. Moeltner, Klaus & Puri, Roshan & Johnston, Robert J. & Besedin, Elena & Balukas, Jessica & Le, Alyssa, 2022. "Locally Weighted Meta-Regression and Benefit Transfer," 2022 Annual Meeting, July 31-August 2, Anaheim, California 322359, Agricultural and Applied Economics Association.
    14. Diluiso, Francesca & Guastella, Gianni & Pareglio, Stefano, 2021. "Changes in urban green spaces’ value perception: A meta-analytic benefit transfer function for European cities," Land Use Policy, Elsevier, vol. 101(C).
    15. Robert J. Johnston & Elena Y. Besedin & Ryan Stapler, 2017. "Enhanced Geospatial Validity for Meta-analysis and Environmental Benefit Transfer: An Application to Water Quality Improvements," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 68(2), pages 343-375, October.
    16. Siriwardena, Shyamani D. & Boyle, Kevin J. & Holmes, Thomas P. & Wiseman, P. Eric, 2016. "The implicit value of tree cover in the U.S.: A meta-analysis of hedonic property value studies," Ecological Economics, Elsevier, vol. 128(C), pages 68-76.
    17. Fitzpatrick, Luke & Parmeter, Christopher F. & Agar, Juan, 2019. "Approaches for visualizing uncertainty in benefit transfer from metaregression," Ecological Economics, Elsevier, vol. 164(C), pages 1-1.
    18. Mazzotta, Marisa & Wainger, Lisa & Sifleet, Samantha & Petty, J.Todd & Rashleigh, Brenda, 2015. "Benefit transfer with limited data: An application to recreational fishing losses from surface mining," Ecological Economics, Elsevier, vol. 119(C), pages 384-398.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Robert J. Johnston & Kevin J. Boyle & Maria L. Loureiro & Ståle Navrud & John Rolfe, 2021. "Guidance to Enhance the Validity and Credibility of Environmental Benefit Transfers," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 79(3), pages 575-624, July.
    2. Londoño, Luz M. & Johnston, Robert J., 2012. "Enhancing the reliability of benefit transfer over heterogeneous sites: A meta-analysis of international coral reef values," Ecological Economics, Elsevier, vol. 78(C), pages 80-89.
    3. Henrik Lindhjem & Tran Tuan, 2012. "Valuation of species and nature conservation in Asia and Oceania: a meta-analysis," Environmental Economics and Policy Studies, Springer;Society for Environmental Economics and Policy Studies - SEEPS, vol. 14(1), pages 1-22, January.
    4. Shuang Liu & David I Stern, 2008. "A Meta-Analysis of Contingent Valuation Studies in Coastal and Near-Shore Marine Ecosystems," Socio-Economics and the Environment in Discussion (SEED) Working Paper Series 2008-15, CSIRO Sustainable Ecosystems.
    5. Richardson, Leslie & Loomis, John & Kroeger, Timm & Casey, Frank, 2015. "The role of benefit transfer in ecosystem service valuation," Ecological Economics, Elsevier, vol. 115(C), pages 51-58.
    6. Fitzpatrick, Luke & Parmeter, Christopher F. & Agar, Juan, 2017. "Threshold Effects in Meta-Analyses With Application to Benefit Transfer for Coral Reef Valuation," Ecological Economics, Elsevier, vol. 133(C), pages 74-85.
    7. Robert J. Johnston & Elena Y. Besedin & Benedict M. Holland, 2019. "Modeling Distance Decay Within Valuation Meta-Analysis," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 72(3), pages 657-690, March.
    8. Kevin J. Boyle & Jeffrey M. Wooldridge, 2018. "Understanding Error Structures and Exploiting Panel Data in Meta-analytic Benefit Transfers," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 69(3), pages 609-635, March.
    9. Kukielka, Jessica B. & Johnston, Robert J. & Duke, Joshua M., 2008. "Systematic Variation in Willingness to Pay for Agricultural Land Preservation and Implications for Benefit Transfer: A Meta-Analysis," 2008 Annual Meeting, July 27-29, 2008, Orlando, Florida 6121, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    10. Robert J. Johnston & Elena Y. Besedin & Ryan Stapler, 2017. "Enhanced Geospatial Validity for Meta-analysis and Environmental Benefit Transfer: An Application to Water Quality Improvements," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 68(2), pages 343-375, October.
    11. Fitzpatrick, Luke & Parmeter, Christopher F. & Agar, Juan, 2019. "Approaches for visualizing uncertainty in benefit transfer from metaregression," Ecological Economics, Elsevier, vol. 164(C), pages 1-1.
    12. Lindhjem, Henrik & Navrud, Ståle, 2008. "How reliable are meta-analyses for international benefit transfers?," Ecological Economics, Elsevier, vol. 66(2-3), pages 425-435, June.
    13. Hermine Vedogbeton & Robert J. Johnston, 2020. "Commodity Consistent Meta-Analysis of Wetland Values: An Illustration for Coastal Marsh Habitat," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 75(4), pages 835-865, April.
    14. Newbold, Stephen C. & Johnston, Robert J., 2020. "Valuing non-market valuation studies using meta-analysis: A demonstration using estimates of willingness-to-pay for water quality improvements," Journal of Environmental Economics and Management, Elsevier, vol. 104(C).
    15. Dennis Guignet & Matthew T. Heberling & Michael Papenfus & Olivia Griot, 2022. "Property Values, Water Quality, and Benefit Transfer: A Nationwide Meta-analysis," Land Economics, University of Wisconsin Press, vol. 98(2), pages 191-218.
    16. Brander, Luke M. & Ghermandi, Andrea & Kuik, Onno & Markandya, Anil & Nunes, Paulo A.L.D. & Schaafsma, Marije & Wagtendonk, Alfred, 2010. "Scaling up Ecosystem Services Values: Methodology, Applicability and a Case Study," Sustainable Development Papers 60689, Fondazione Eni Enrico Mattei (FEEM).
    17. Chaikumbung, Mayula & Doucouliagos, Hristos & Scarborough, Helen, 2016. "The economic value of wetlands in developing countries: A meta-regression analysis," Ecological Economics, Elsevier, vol. 124(C), pages 164-174.
    18. Kaul, Sapna & Boyle, Kevin J. & Kuminoff, Nicolai V. & Parmeter, Christopher F. & Pope, Jaren C., 2013. "What can we learn from benefit transfer errors? Evidence from 20 years of research on convergent validity," Journal of Environmental Economics and Management, Elsevier, vol. 66(1), pages 90-104.
    19. Ryan Stapler & Robert Johnston, 2009. "Meta-Analysis, Benefit Transfer, and Methodological Covariates: Implications for Transfer Error," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 42(2), pages 227-246, February.
    20. Ghermandi, Andrea & Nunes, Paulo A.L.D., 2013. "A global map of coastal recreation values: Results from a spatially explicit meta-analysis," Ecological Economics, Elsevier, vol. 86(C), pages 1-15.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:enreec:v:55:y:2013:i:3:p:357-386. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.