IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v265y2018i1p178-194.html
   My bibliography  Save this article

Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis

Author

Listed:
  • Marttunen, Mika
  • Belton, Valerie
  • Lienert, Judit

Abstract

Procedural and behavioural biases have received little attention in recent Multi-Criteria Decision Analysis (MCDA) research. Our literature review shows that most research on biases was done 15–30 years ago. This study focuses on biases that are introduced at an early stage of MCDA when building objectives hierarchies and their effect on the weights. The main objective is to investigate whether prior findings regarding such biases, which were mostly based on laboratory experiments, can be found in real-world applications. We conducted a meta-analysis of the objectives hierarchies and weight elicitation procedures in 61 environmental and energy MCDA cases. Relationships between the structural characteristics of the objectives hierarchy and assigned objectives’ weights were analysed with statistical tests. Our main research questions were: (i) How does hierarchy size and structure affect the objectives’ weights? (ii) How are weights distributed across economic, social and environmental objectives? (iii) Is there support for the equalising bias? Our findings are mostly aligned with earlier research and suggest that the hierarchy structure and content can substantially influence weight distributions. For example, hierarchical weighting seems to be sensitive to the asymmetry bias, which can occur when a hierarchy has branches that differ in the number of sub-objectives. We found no evidence for the equalising bias. We highlight issues deserving more attention when developing objectives hierarchies and eliciting weights. The research demonstrates the potential to use meta-analysis, which has not previously been used in this way in the MCDA field, to learn from a collection of applications.

Suggested Citation

  • Marttunen, Mika & Belton, Valerie & Lienert, Judit, 2018. "Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis," European Journal of Operational Research, Elsevier, vol. 265(1), pages 178-194.
  • Handle: RePEc:eee:ejores:v:265:y:2018:i:1:p:178-194
    DOI: 10.1016/j.ejor.2017.02.038
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221717301777
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2017.02.038?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Brown, Katrina & Adger, W. Neil & Tompkins, Emma & Bacon, Peter & Shim, David & Young, Kathy, 2001. "Trade-off analysis for marine protected area management," Ecological Economics, Elsevier, vol. 37(3), pages 417-434, June.
    2. Kodikara, P.N. & Perera, B.J.C. & Kularathna, M.D.U.P., 2010. "Stakeholder preference elicitation and modelling in multi-criteria decision analysis - A case study on urban water supply," European Journal of Operational Research, Elsevier, vol. 206(1), pages 209-220, October.
    3. Van Ittersum, Koert & Pennings, Joost M.E. & Wansink, Brian & van Trijp, Hans C.M., 2007. "The validity of attribute-importance measurement: A review," Journal of Business Research, Elsevier, vol. 60(11), pages 1177-1190, November.
    4. von Winterfeldt, Detlof & Fasolo, Barbara, 2009. "Structuring decision problems: A case study and reflections for practitioners," European Journal of Operational Research, Elsevier, vol. 199(3), pages 857-866, December.
    5. Garmendia, Eneko & Gamboa, Gonzalo, 2012. "Weighting social preferences in participatory multi-criteria evaluations: A case study on sustainable natural resource management," Ecological Economics, Elsevier, vol. 84(C), pages 110-120.
    6. Craig R. Fox & Robert T. Clemen, 2005. "Subjective Probability Assessment in Decision Analysis: Partition Dependence and Bias Toward the Ignorance Prior," Management Science, INFORMS, vol. 51(9), pages 1417-1432, September.
    7. Martin Weber & Franz Eisenführ & Detlof von Winterfeldt, 1988. "The Effects of Splitting Attributes on Weights in Multiattribute Utility Measurement," Management Science, INFORMS, vol. 34(4), pages 431-445, April.
    8. Gregory W. Fischer & Nirmala Damodaran & Kathryn B. Laskey & David Lincoln, 1987. "Preferences for Proxy Attributes," Management Science, INFORMS, vol. 33(2), pages 198-214, February.
    9. Saad, Gad & Russo, J. Edward, 1996. "Stopping Criteria in Sequential Choice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 67(3), pages 258-270, September.
    10. Georgopoulou, E. & Lalas, D. & Papagiannakis, L., 1997. "A multicriteria decision aid approach for energy planning problems: The case of renewable energy option," European Journal of Operational Research, Elsevier, vol. 103(1), pages 38-54, November.
    11. Yoram Wind & Thomas L. Saaty, 1980. "Marketing Applications of the Analytic Hierarchy Process," Management Science, INFORMS, vol. 26(7), pages 641-658, July.
    12. Franco, L. Alberto & Hämäläinen, Raimo P., 2016. "Behavioural operational research: Returning to the roots of the OR profession," European Journal of Operational Research, Elsevier, vol. 249(3), pages 791-795.
    13. Macharis, Cathy & Springael, Johan & De Brucker, Klaas & Verbeke, Alain, 2004. "PROMETHEE and AHP: The design of operational synergies in multicriteria analysis.: Strengthening PROMETHEE with ideas of AHP," European Journal of Operational Research, Elsevier, vol. 153(2), pages 307-317, March.
    14. Lindhjem, Henrik & Navrud, Ståle, 2008. "How reliable are meta-analyses for international benefit transfers?," Ecological Economics, Elsevier, vol. 66(2-3), pages 425-435, June.
    15. Vanessa Silva & Danielle Morais & Adiel Almeida, 2010. "A Multicriteria Group Decision Model to Support Watershed Committees in Brazil," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 24(14), pages 4075-4091, November.
    16. Grace Lee & Edwin Chan, 2008. "The Analytic Hierarchy Process (AHP) Approach for Assessment of Urban Renewal Proposals," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 89(1), pages 155-168, October.
    17. Anna Straton & Sue Jackson & Oswald Marinoni & Wendy Proctor & Emma Woodward, 2011. "Exploring and Evaluating Scenarios for a River Catchment in Northern Australia Using Scenario Development, Multi-criteria Analysis and a Deliberative Process as a Tool for Water Planning," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 25(1), pages 141-164, January.
    18. Buchholz, Thomas & Rametsteiner, Ewald & Volk, Timothy A. & Luzadis, Valerie A., 2009. "Multi Criteria Analysis for bioenergy systems assessments," Energy Policy, Elsevier, vol. 37(2), pages 484-495, February.
    19. Ralph L. Keeney, 2002. "Common Mistakes in Making Value Trade-Offs," Operations Research, INFORMS, vol. 50(6), pages 935-945, December.
    20. Ralph L. Keeney & Robin S. Gregory, 2005. "Selecting Attributes to Measure the Achievement of Objectives," Operations Research, INFORMS, vol. 53(1), pages 1-11, February.
    21. Hämäläinen, Raimo P. & Alaja, Susanna, 2008. "The threat of weighting biases in environmental decision analysis," Ecological Economics, Elsevier, vol. 68(1-2), pages 556-569, December.
    22. Richard M. Anderson & Robert Clemen, 2013. "Toward an Improved Methodology to Construct and Reconcile Decision Analytic Preference Judgments," Decision Analysis, INFORMS, vol. 10(2), pages 121-134, June.
    23. Fischer, Gregory W., 1995. "Range Sensitivity of Attribute Weights in Multiattribute Value Models," Organizational Behavior and Human Decision Processes, Elsevier, vol. 62(3), pages 252-266, June.
    24. Scholten, Lisa & Schuwirth, Nele & Reichert, Peter & Lienert, Judit, 2015. "Tackling uncertainty in multi-criteria decision analysis – An application to water supply infrastructure planning," European Journal of Operational Research, Elsevier, vol. 242(1), pages 243-260.
    25. Richard H. Thaler & Shlomo Benartzi, 2001. "Naive Diversification Strategies in Defined Contribution Saving Plans," American Economic Review, American Economic Association, vol. 91(1), pages 79-98, March.
    26. Eneko Garmendia & Gonzalo Gamboa, 2012. "Weighting social preferences in participatory multi-criteria evaluations: a case study on sustainable natural resource management," Working Papers 2012-06, BC3.
    27. Pascoe, Sean & Proctor, Wendy & Wilcox, Chris & Innes, James & Rochester, Wayne & Dowling, Natalie, 2009. "Stakeholder objective preferences in Australian Commonwealth managed fisheries," Marine Policy, Elsevier, vol. 33(5), pages 750-758, September.
    28. Gur Huberman & Wei Jiang, 2006. "Offering versus Choice in 401(k) Plans: Equity Exposure and Number of Funds," Journal of Finance, American Finance Association, vol. 61(2), pages 763-801, April.
    29. Weber, Martin & Borcherding, Katrin, 1993. "Behavioral influences on weight judgments in multiattribute decision making," European Journal of Operational Research, Elsevier, vol. 67(1), pages 1-12, May.
    30. Ananda, Jayanath & Herath, Gamini, 2009. "A critical review of multi-criteria decision making methods with special reference to forest management and planning," Ecological Economics, Elsevier, vol. 68(10), pages 2535-2548, August.
    31. Poyhonen, Mari & Vrolijk, Hans & Hamalainen, Raimo P., 2001. "Behavioral and procedural consequences of structural variation in value trees," European Journal of Operational Research, Elsevier, vol. 134(1), pages 216-227, October.
    32. Lienert, Judit & Duygan, Mert & Zheng, Jun, 2016. "Preference stability over time with multiple elicitation methods to support wastewater infrastructure decision-making," European Journal of Operational Research, Elsevier, vol. 253(3), pages 746-760.
    33. Mohammad Rahman & Lena Jaumann & Nils Lerche & Fabian Renatus & Ann Buchs & Rudolf Gade & Jutta Geldermann & Martin Sauter, 2015. "Selection of the Best Inland Waterway Structure: A Multicriteria Decision Analysis Approach," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 29(8), pages 2733-2749, June.
    34. Sarah K. Jacobi & Benjamin F. Hobbs, 2007. "Quantifying and Mitigating the Splitting Bias and Other Value Tree-Induced Weighting Biases," Decision Analysis, INFORMS, vol. 4(4), pages 194-210, December.
    35. Doyle, John R. & Green, Rodney H. & Bottomley, Paul A., 1997. "Judging Relative Importance: Direct Rating and Point Allocation Are Not Equivalent," Organizational Behavior and Human Decision Processes, Elsevier, vol. 70(1), pages 65-72, April.
    36. Bertrand Mareschal & Jean Pierre Brans & Philippe Vincke, 1986. "How to select and how to rank projects: the Prométhée method," ULB Institutional Repository 2013/9307, ULB -- Universite Libre de Bruxelles.
    37. Bottomley, Paul A. & Doyle, John R., 2001. "A comparison of three weight elicitation methods: good, better, and best," Omega, Elsevier, vol. 29(6), pages 553-560, December.
    38. Figueira, Jose & Roy, Bernard, 2002. "Determining the weights of criteria in the ELECTRE type methods with a revised Simos' procedure," European Journal of Operational Research, Elsevier, vol. 139(2), pages 317-326, June.
    39. Jon Nelson & Peter Kennedy, 2009. "The Use (and Abuse) of Meta-Analysis in Environmental and Natural Resource Economics: An Assessment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 42(3), pages 345-377, March.
    40. Brans, J. P. & Vincke, Ph. & Mareschal, B., 1986. "How to select and how to rank projects: The method," European Journal of Operational Research, Elsevier, vol. 24(2), pages 228-238, February.
    41. Marttunen, Mika & Hamalainen, Raimo P., 1995. "Decision analysis interviews in environmental impact assessment," European Journal of Operational Research, Elsevier, vol. 87(3), pages 551-563, December.
    42. Paul J. H. Schoemaker & C. Carter Waid, 1982. "An Experimental Comparison of Different Approaches to Determining Weights in Additive Utility Models," Management Science, INFORMS, vol. 28(2), pages 182-196, February.
    43. Schuwirth, N. & Reichert, P. & Lienert, J., 2012. "Methodological aspects of multi-criteria decision analysis for policy support: A case study on pharmaceutical removal from hospital wastewater," European Journal of Operational Research, Elsevier, vol. 220(2), pages 472-483.
    44. Rüdiger von Nitzsch & Martin Weber, 1993. "The Effect of Attribute Ranges on Weights in Multiattribute Utility Measurements," Management Science, INFORMS, vol. 39(8), pages 937-943, August.
    45. Belton, Valerie, 1986. "A comparison of the analytic hierarchy process and a simple multi-attribute value function," European Journal of Operational Research, Elsevier, vol. 26(1), pages 7-21, July.
    46. Carlos Bana e Costa & Paula Antão da Silva & Francisco Nunes Correia, 2004. "Multicriteria Evaluation of Flood Control Measures: The Case of Ribeira do Livramento," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 18(3), pages 263-283, June.
    47. William G. Stillwell & Detlof von Winterfeldt & Richard S. John, 1987. "Comparing Hierarchical and Nonhierarchical Weighting Methods for Eliciting Multiattribute Value Models," Management Science, INFORMS, vol. 33(4), pages 442-450, April.
    48. Katsikopoulos, Konstantinos V., 2016. "On the role of psychological heuristics in operational research; and a demonstration in military stability operationsAuthor-Name: Keller, Niklas," European Journal of Operational Research, Elsevier, vol. 249(3), pages 1063-1073.
    49. Stefan Hajkowicz & Kerry Collins, 2007. "A Review of Multiple Criteria Analysis for Water Resource Planning and Management," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 21(9), pages 1553-1566, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cinelli, Marco & Kadziński, Miłosz & Gonzalez, Michael & Słowiński, Roman, 2020. "How to support the application of multiple criteria decision analysis? Let us start with a comprehensive taxonomy," Omega, Elsevier, vol. 96(C).
    2. Nomeda Dobrovolskienė & Anastasija Pozniak & Manuela Tvaronavičienė, 2021. "Assessment of the Sustainability of a Real Estate Project Using Multi-Criteria Decision Making," Sustainability, MDPI, vol. 13(8), pages 1-19, April.
    3. Mustajoki, Jyri & Saarikoski, Heli & Belton, Valerie & Hjerppe, Turo & Marttunen, Mika, 2020. "Utilizing ecosystem service classifications in multi-criteria decision analysis – Experiences of peat extraction case in Finland," Ecosystem Services, Elsevier, vol. 41(C).
    4. McKenna, R. & Bertsch, V. & Mainzer, K. & Fichtner, W., 2018. "Combining local preferences with multi-criteria decision analysis and linear optimization to develop feasible energy concepts in small communities," European Journal of Operational Research, Elsevier, vol. 268(3), pages 1092-1110.
    5. Ezbakhe, Fatine & Pérez-Foguet, Agustí, 2021. "Decision analysis for sustainable development: The case of renewable energy planning under uncertainty," European Journal of Operational Research, Elsevier, vol. 291(2), pages 601-613.
    6. Yuzheng Wang & Lei Nie & Min Zhang & Hong Wang & Yan Xu & Tianyu Zuo, 2020. "Assessment of Debris Flow Risk Factors Based on Meta-Analysis—Cases Study of Northwest and Southwest China," Sustainability, MDPI, vol. 12(17), pages 1-24, August.
    7. Judit Lienert & Igor Linkov, 2019. "Editorial featured papers on environmental decisions," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 7(3), pages 151-157, November.
    8. Maija Tiitu & Arto Viinikka & Leena Kopperoinen & Davide Geneletti, 2018. "Balancing Urban Green Space and Residential Infill Development: A Spatial Multi-Criteria Approach Based on Practitioner Engagement," Journal of Environmental Assessment Policy and Management (JEAPM), World Scientific Publishing Co. Pte. Ltd., vol. 20(03), pages 1-26, September.
    9. Mika Marttunen & Jyri Mustajoki, 2018. "Use of Analyst-Generated Stakeholder Preference Profiles in Multi-Criteria Decision Analysis — Experiences from an Urban Planning Case," Journal of Environmental Assessment Policy and Management (JEAPM), World Scientific Publishing Co. Pte. Ltd., vol. 20(03), pages 1-29, September.
    10. Marttunen, Mika & Haag, Fridolin & Belton, Valerie & Mustajoki, Jyri & Lienert, Judit, 2019. "Methods to inform the development of concise objectives hierarchies in multi-criteria decision analysis," European Journal of Operational Research, Elsevier, vol. 277(2), pages 604-620.
    11. Aubert, Alice H. & Esculier, Fabien & Lienert, Judit, 2020. "Recommendations for online elicitation of swing weights from citizens in environmental decision-making," Operations Research Perspectives, Elsevier, vol. 7(C).
    12. Haag, Fridolin & Lienert, Judit & Schuwirth, Nele & Reichert, Peter, 2019. "Identifying non-additive multi-attribute value functions based on uncertain indifference statements," Omega, Elsevier, vol. 85(C), pages 49-67.
    13. Paweł Karczmarek & Witold Pedrycz & Adam Kiersztyn, 2021. "Fuzzy Analytic Hierarchy Process in a Graphical Approach," Group Decision and Negotiation, Springer, vol. 30(2), pages 463-481, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marttunen, Mika & Haag, Fridolin & Belton, Valerie & Mustajoki, Jyri & Lienert, Judit, 2019. "Methods to inform the development of concise objectives hierarchies in multi-criteria decision analysis," European Journal of Operational Research, Elsevier, vol. 277(2), pages 604-620.
    2. Zheng, Jun & Lienert, Judit, 2018. "Stakeholder interviews with two MAVT preference elicitation philosophies in a Swiss water infrastructure decision: Aggregation using SWING-weighting and disaggregation using UTAGMS," European Journal of Operational Research, Elsevier, vol. 267(1), pages 273-287.
    3. Lienert, Judit & Duygan, Mert & Zheng, Jun, 2016. "Preference stability over time with multiple elicitation methods to support wastewater infrastructure decision-making," European Journal of Operational Research, Elsevier, vol. 253(3), pages 746-760.
    4. Gilberto Montibeller & Detlof von Winterfeldt, 2015. "Cognitive and Motivational Biases in Decision and Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 35(7), pages 1230-1251, July.
    5. Suk, Kwanho & Yoon, Song-Oh, 2012. "The moderating role of decision task goals in attribute weight convergence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 118(1), pages 37-45.
    6. Sarah K. Jacobi & Benjamin F. Hobbs, 2007. "Quantifying and Mitigating the Splitting Bias and Other Value Tree-Induced Weighting Biases," Decision Analysis, INFORMS, vol. 4(4), pages 194-210, December.
    7. Hämäläinen, Raimo P. & Alaja, Susanna, 2008. "The threat of weighting biases in environmental decision analysis," Ecological Economics, Elsevier, vol. 68(1-2), pages 556-569, December.
    8. Richard M. Anderson & Robert Clemen, 2013. "Toward an Improved Methodology to Construct and Reconcile Decision Analytic Preference Judgments," Decision Analysis, INFORMS, vol. 10(2), pages 121-134, June.
    9. Schuwirth, N. & Reichert, P. & Lienert, J., 2012. "Methodological aspects of multi-criteria decision analysis for policy support: A case study on pharmaceutical removal from hospital wastewater," European Journal of Operational Research, Elsevier, vol. 220(2), pages 472-483.
    10. Garmendia, Eneko & Gamboa, Gonzalo, 2012. "Weighting social preferences in participatory multi-criteria evaluations: A case study on sustainable natural resource management," Ecological Economics, Elsevier, vol. 84(C), pages 110-120.
    11. Franco, L. Alberto & Hämäläinen, Raimo P. & Rouwette, Etiënne A.J.A. & Leppänen, Ilkka, 2021. "Taking stock of behavioural OR: A review of behavioural studies with an intervention focus," European Journal of Operational Research, Elsevier, vol. 293(2), pages 401-418.
    12. Scholten, Lisa & Schuwirth, Nele & Reichert, Peter & Lienert, Judit, 2015. "Tackling uncertainty in multi-criteria decision analysis – An application to water supply infrastructure planning," European Journal of Operational Research, Elsevier, vol. 242(1), pages 243-260.
    13. Kodikara, P.N. & Perera, B.J.C. & Kularathna, M.D.U.P., 2010. "Stakeholder preference elicitation and modelling in multi-criteria decision analysis - A case study on urban water supply," European Journal of Operational Research, Elsevier, vol. 206(1), pages 209-220, October.
    14. Mika Marttunen & Jyri Mustajoki, 2018. "Use of Analyst-Generated Stakeholder Preference Profiles in Multi-Criteria Decision Analysis — Experiences from an Urban Planning Case," Journal of Environmental Assessment Policy and Management (JEAPM), World Scientific Publishing Co. Pte. Ltd., vol. 20(03), pages 1-29, September.
    15. Poyhonen, Mari & Hamalainen, Raimo P., 2001. "On the convergence of multiattribute weighting methods," European Journal of Operational Research, Elsevier, vol. 129(3), pages 569-585, March.
    16. Hämäläinen, Raimo P. & Lahtinen, Tuomas J., 2016. "Path dependence in Operational Research—How the modeling process can influence the results," Operations Research Perspectives, Elsevier, vol. 3(C), pages 14-20.
    17. Zhou, P. & Ang, B.W. & Poh, K.L., 2006. "Decision analysis in energy and environmental modeling: An update," Energy, Elsevier, vol. 31(14), pages 2604-2622.
    18. Wang, Ying-Ming & Yang, Jian-Bo & Xu, Dong-Ling, 2006. "Environmental impact assessment using the evidential reasoning approach," European Journal of Operational Research, Elsevier, vol. 174(3), pages 1885-1913, November.
    19. Landry, Peter & Webb, Ryan, 2021. "Pairwise normalization: A neuroeconomic theory of multi-attribute choice," Journal of Economic Theory, Elsevier, vol. 193(C).
    20. Saarikoski, Heli & Mustajoki, Jyri & Barton, David N. & Geneletti, Davide & Langemeyer, Johannes & Gomez-Baggethun, Erik & Marttunen, Mika & Antunes, Paula & Keune, Hans & Santos, Rui, 2016. "Multi-Criteria Decision Analysis and Cost-Benefit Analysis: Comparing alternative frameworks for integrated valuation of ecosystem services," Ecosystem Services, Elsevier, vol. 22(PB), pages 238-249.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:265:y:2018:i:1:p:178-194. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/eor .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.