IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v10y2013i2p121-134.html
   My bibliography  Save this article

Toward an Improved Methodology to Construct and Reconcile Decision Analytic Preference Judgments

Author

Listed:
  • Richard M. Anderson

    (Puget Sound Institute, Center for Urban Waters, University of Washington Tacoma, Tacoma, Washington 98421)

  • Robert Clemen

    (Fuqua School of Business, Duke University, Durham, North Carolina 27708)

Abstract

Psychologists and behavioral economists have documented a variety of judgmental flaws that people make when they face novel decision situations. Similar flaws arise when decision analysts work with decision makers to assess their preferences and trade-offs, because the methods the analyst uses are often unfamiliar to the decision makers. In this paper we describe a process designed to mitigate the occurrence of such biases; it brings together three steps. In training , the decision maker is first given values to apply in judgment tasks unrelated to the decision at hand, providing an introduction to thinking deliberately and quantitatively about preferences. In practice , the learned tasks are then applied to a familiar decision, with the goal of developing the next incremental level of expertise in using the methods. Finally, in application , the more deliberative style of thinking is used to address the problem of interest. In an environmental resource setting with two oyster habitat managers, we test the procedure by attempting to mitigate the prominence effect that has been reported in the behavioral research literature. The resulting preference weights appear to be free of the prominence effect, providing initial steps toward operationalizing the “building code” for preferences introduced by Payne et al. [Payne JW, Bettman JR, Schkade DA (1999) Measuring constructed preferences: Towards a building code. J. Risk Uncertainty 19(1--3):243--270].

Suggested Citation

  • Richard M. Anderson & Robert Clemen, 2013. "Toward an Improved Methodology to Construct and Reconcile Decision Analytic Preference Judgments," Decision Analysis, INFORMS, vol. 10(2), pages 121-134, June.
  • Handle: RePEc:inm:ordeca:v:10:y:2013:i:2:p:121-134
    DOI: 10.1287/deca.2013.0268
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.2013.0268
    Download Restriction: no

    File URL: https://libkey.io/10.1287/deca.2013.0268?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gregory, Robin, 1999. "Commentary on "Measuring Constructed Preferences: Towards a Building Code" by Payne, Bettman and Schkade," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 273-275, December.
    2. Hogarth, Robin M. (ed.), 1990. "Insights in Decision Making," University of Chicago Press Economics Books, University of Chicago Press, edition 1, number 9780226348551, September.
    3. Hobbs, Benjamin F & Horn, Graham TF, 1997. "Building public confidence in energy planning: a multimethod MCDM approach to demand-side planning at BC gas," Energy Policy, Elsevier, vol. 25(3), pages 357-375, February.
    4. Ralph L. Keeney & Timothy L. McDaniels, 1992. "Value-Focused Thinking about Strategic Decisions at BC Hydro," Interfaces, INFORMS, vol. 22(6), pages 94-109, December.
    5. Martin Weber & Franz Eisenführ & Detlof von Winterfeldt, 1988. "The Effects of Splitting Attributes on Weights in Multiattribute Utility Measurement," Management Science, INFORMS, vol. 34(4), pages 431-445, April.
    6. Payne, John W & Bettman, James R & Schkade, David A, 1999. "Measuring Constructed Preferences: Towards a Building Code," Journal of Risk and Uncertainty, Springer, vol. 19(1-3), pages 243-270, December.
    7. Gregory W. Fischer & Ziv Carmon & Dan Ariely & Gal Zauberman, 1999. "Goal-Based Construction of Preferences: Task Goals and the Prominence Effect," Management Science, INFORMS, vol. 45(8), pages 1057-1075, August.
    8. Ralph L. Keeney, 2002. "Common Mistakes in Making Value Trade-Offs," Operations Research, INFORMS, vol. 50(6), pages 935-945, December.
    9. Ralph L. Keeney & Robin S. Gregory, 2005. "Selecting Attributes to Measure the Achievement of Objectives," Operations Research, INFORMS, vol. 53(1), pages 1-11, February.
    10. F. Hutton Barron & Bruce E. Barrett, 1996. "Decision Quality Using Ranked Attribute Weights," Management Science, INFORMS, vol. 42(11), pages 1515-1523, November.
    11. Hämäläinen, Raimo P. & Alaja, Susanna, 2008. "The threat of weighting biases in environmental decision analysis," Ecological Economics, Elsevier, vol. 68(1-2), pages 556-569, December.
    12. Fischer, Gregory W., 1995. "Range Sensitivity of Attribute Weights in Multiattribute Value Models," Organizational Behavior and Human Decision Processes, Elsevier, vol. 62(3), pages 252-266, June.
    13. Edwards, Ward & Barron, F. Hutton, 1994. "SMARTS and SMARTER: Improved Simple Methods for Multiattribute Utility Measurement," Organizational Behavior and Human Decision Processes, Elsevier, vol. 60(3), pages 306-325, December.
    14. Kurt A. Carlson & Margaret G. Meloy & J. Edward Russo, 2006. "Leader-Driven Primacy: Using Attribute Order to Affect Consumer Choice," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 32(4), pages 513-518, March.
    15. Katrin Borcherding & Thomas Eppel & Detlof von Winterfeldt, 1991. "Comparison of Weighting Judgments in Multiattribute Utility Measurement," Management Science, INFORMS, vol. 37(12), pages 1603-1619, December.
    16. Peter C. Fishburn, 1967. "Methods of Estimating Additive Utilities," Management Science, INFORMS, vol. 13(7), pages 435-453, March.
    17. Sarah K. Jacobi & Benjamin F. Hobbs, 2007. "Quantifying and Mitigating the Splitting Bias and Other Value Tree-Induced Weighting Biases," Decision Analysis, INFORMS, vol. 4(4), pages 194-210, December.
    18. Keeney, Ralph L. & Renn, Ortwin & von Winterfeldt, Detlof, 1987. "Structuring West Germany's energy objectives," Energy Policy, Elsevier, vol. 15(4), pages 352-362, August.
    19. Huber, Joel & Ariely, Dan & Fischer, Gregory, 2002. "Expressing Preferences in a Principal-Agent Task: A Comparison of Choice, Rating, and Matching," Organizational Behavior and Human Decision Processes, Elsevier, vol. 87(1), pages 66-90, January.
    20. Paul J. H. Schoemaker & C. Carter Waid, 1982. "An Experimental Comparison of Different Approaches to Determining Weights in Additive Utility Models," Management Science, INFORMS, vol. 28(2), pages 182-196, February.
    21. Huber, Joel & Payne, John W & Puto, Christopher, 1982. "Adding Asymmetrically Dominated Alternatives: Violations of Regularity and the Similarity Hypothesis," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 9(1), pages 90-98, June.
    22. Robin Gregory & Baruch Fischhoff & Tim McDaniels, 2005. "Acceptable Input: Using Decision Analysis to Guide Public Policy Deliberations," Decision Analysis, INFORMS, vol. 2(1), pages 4-16, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrea C. Hupman & Jay Simon, 2023. "The Legacy of Peter Fishburn: Foundational Work and Lasting Impact," Decision Analysis, INFORMS, vol. 20(1), pages 1-15, March.
    2. Lahtinen, Tuomas J. & Hämäläinen, Raimo P., 2016. "Path dependence and biases in the even swaps decision analysis method," European Journal of Operational Research, Elsevier, vol. 249(3), pages 890-898.
    3. Kuller, M. & Beutler, P. & Lienert, J., 2023. "Preference change in stakeholder group-decision processes in the public sector: Extent, causes and implications," European Journal of Operational Research, Elsevier, vol. 308(3), pages 1268-1285.
    4. Zheng, Jun & Lienert, Judit, 2018. "Stakeholder interviews with two MAVT preference elicitation philosophies in a Swiss water infrastructure decision: Aggregation using SWING-weighting and disaggregation using UTAGMS," European Journal of Operational Research, Elsevier, vol. 267(1), pages 273-287.
    5. Marttunen, Mika & Belton, Valerie & Lienert, Judit, 2018. "Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis," European Journal of Operational Research, Elsevier, vol. 265(1), pages 178-194.
    6. Zhan, Yuanzhu & Chung, Leanne & Lim, Ming K. & Ye, Fei & Kumar, Ajay & Tan, Kim Hua, 2021. "The impact of sustainability on supplier selection: A behavioural study," International Journal of Production Economics, Elsevier, vol. 236(C).
    7. Mika Marttunen & Jyri Mustajoki, 2018. "Use of Analyst-Generated Stakeholder Preference Profiles in Multi-Criteria Decision Analysis — Experiences from an Urban Planning Case," Journal of Environmental Assessment Policy and Management (JEAPM), World Scientific Publishing Co. Pte. Ltd., vol. 20(03), pages 1-29, September.
    8. Aubert, Alice H. & Esculier, Fabien & Lienert, Judit, 2020. "Recommendations for online elicitation of swing weights from citizens in environmental decision-making," Operations Research Perspectives, Elsevier, vol. 7(C).
    9. Rakesh K. Sarin & L. Robin Keller, 2013. "From the Editors ---Group Decisions, Preference Elicitation, Experienced Utility, Survival Probabilities, and Portfolio Value of Information," Decision Analysis, INFORMS, vol. 10(2), pages 99-102, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Richard M. Anderson & Benjamin F. Hobbs, 2002. "Using a Bayesian Approach to Quantify Scale Compatibility Bias," Management Science, INFORMS, vol. 48(12), pages 1555-1568, December.
    2. Gilberto Montibeller & Detlof von Winterfeldt, 2015. "Cognitive and Motivational Biases in Decision and Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 35(7), pages 1230-1251, July.
    3. Marttunen, Mika & Belton, Valerie & Lienert, Judit, 2018. "Are objectives hierarchy related biases observed in practice? A meta-analysis of environmental and energy applications of Multi-Criteria Decision Analysis," European Journal of Operational Research, Elsevier, vol. 265(1), pages 178-194.
    4. Hämäläinen, Raimo P. & Alaja, Susanna, 2008. "The threat of weighting biases in environmental decision analysis," Ecological Economics, Elsevier, vol. 68(1-2), pages 556-569, December.
    5. Lahtinen, Tuomas J. & Hämäläinen, Raimo P., 2016. "Path dependence and biases in the even swaps decision analysis method," European Journal of Operational Research, Elsevier, vol. 249(3), pages 890-898.
    6. Scholz, Michael & Dorner, Verena & Schryen, Guido & Benlian, Alexander, 2017. "A configuration-based recommender system for supporting e-commerce decisions," European Journal of Operational Research, Elsevier, vol. 259(1), pages 205-215.
    7. Sarah K. Jacobi & Benjamin F. Hobbs, 2007. "Quantifying and Mitigating the Splitting Bias and Other Value Tree-Induced Weighting Biases," Decision Analysis, INFORMS, vol. 4(4), pages 194-210, December.
    8. Marttunen, Mika & Haag, Fridolin & Belton, Valerie & Mustajoki, Jyri & Lienert, Judit, 2019. "Methods to inform the development of concise objectives hierarchies in multi-criteria decision analysis," European Journal of Operational Research, Elsevier, vol. 277(2), pages 604-620.
    9. Milad Zamanifar & Timo Hartmann, 2021. "A prescriptive framework for recommending decision attributes of infrastructure disaster recovery problems," Environment Systems and Decisions, Springer, vol. 41(4), pages 633-650, December.
    10. Roger Chapman Burk & Richard M. Nehring, 2023. "An Empirical Comparison of Rank-Based Surrogate Weights in Additive Multiattribute Decision Analysis," Decision Analysis, INFORMS, vol. 20(1), pages 55-72, March.
    11. Suk, Kwanho & Yoon, Song-Oh, 2012. "The moderating role of decision task goals in attribute weight convergence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 118(1), pages 37-45.
    12. James S. Dyer & James E. Smith, 2021. "Innovations in the Science and Practice of Decision Analysis: The Role of Management Science," Management Science, INFORMS, vol. 67(9), pages 5364-5378, September.
    13. Schuwirth, N. & Reichert, P. & Lienert, J., 2012. "Methodological aspects of multi-criteria decision analysis for policy support: A case study on pharmaceutical removal from hospital wastewater," European Journal of Operational Research, Elsevier, vol. 220(2), pages 472-483.
    14. Garmendia, Eneko & Gamboa, Gonzalo, 2012. "Weighting social preferences in participatory multi-criteria evaluations: A case study on sustainable natural resource management," Ecological Economics, Elsevier, vol. 84(C), pages 110-120.
    15. Lahtinen, Tuomas J. & Hämäläinen, Raimo P. & Jenytin, Cosmo, 2020. "On preference elicitation processes which mitigate the accumulation of biases in multi-criteria decision analysis," European Journal of Operational Research, Elsevier, vol. 282(1), pages 201-210.
    16. de Almeida, Jonatas Araujo & Costa, Ana Paula Cabral Seixas & de Almeida-Filho, Adiel Teixeira, 2016. "A new method for elicitation of criteria weights in additive models: Flexible and interactive tradeoffAuthor-Name: de Almeida, Adiel Teixeira," European Journal of Operational Research, Elsevier, vol. 250(1), pages 179-191.
    17. Hämäläinen, Raimo P. & Lahtinen, Tuomas J., 2016. "Path dependence in Operational Research—How the modeling process can influence the results," Operations Research Perspectives, Elsevier, vol. 3(C), pages 14-20.
    18. Kurt A. Carlson & Samuel D. Bond, 2006. "Improving Preference Assessment: Limiting the Effect of Context Through Pre-exposure to Attribute Levels," Management Science, INFORMS, vol. 52(3), pages 410-421, March.
    19. Eneko Garmendia & Gonzalo Gamboa, 2012. "Weighting social preferences in participatory multi-criteria evaluations: a case study on sustainable natural resource management," Working Papers 2012-06, BC3.
    20. Butler, John C. & Dyer, James S. & Jia, Jianmin & Tomak, Kerem, 2008. "Enabling e-transactions with multi-attribute preference models," European Journal of Operational Research, Elsevier, vol. 186(2), pages 748-765, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:10:y:2013:i:2:p:121-134. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.