Comparison of Weighting Judgments in Multiattribute Utility Measurement
AbstractThis paper compares four weighting methods in multiattribute utility measurement: the ratio method, the swing weighting method, the tradeoff method and the pricing out method. 200 subjects used these methods to weight attributes for evaluating nuclear waste repository sites in the United States. The weighting methods were compared with respect to their internal consistency, convergent validity, and external validity. Internal consistency was measured by the degree to which ordinal and cardinal or ratio responses agreed within the same weighting method. Convergent validity was measured by the degree of agreement between the weights elicited with different methods. External validity was determined by the degree to which weights elicited in this experiment agreed with weights that were elicited with managers of the Department of Energy. In terms of internal consistency, the tradeoff method fared worst. In terms of convergent validity, the pricing out method turned out to be an outlier. In terms of external validity, the pricing out method showed the best results. While the ratio and swing methods are quite consistent and show a fair amount of convergent validity, their external validity problems cast doubt on their usefulness. The main recommendation for applications is to improve the internal consistency of the tradeoff method by careful interactive elicitation and to use it in conjunction with the pricing out method to enhance its external validity.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoArticle provided by INFORMS in its journal Management Science.
Volume (Year): 37 (1991)
Issue (Month): 12 (December)
tradeoffs; weights; multiattribute utility; nuclear waste disposal; decision analysis;
You can help add them by filling out this form.
CitEc Project, subscribe to its RSS feed for this item.
- Sam Park, Kyung & Sang Lee, Kyung & Seong Eum, Yun & Park, Kwangtae, 2001. "Extended methods for identifying dominance and potential optimality in multi-criteria analysis with imprecise information," European Journal of Operational Research, Elsevier, vol. 134(3), pages 557-563, November.
- Eneko Garmendia & Gonzalo Gamboa, 2012. "Weighting social preferences in participatory multi-criteria evaluations: a case study on sustainable natural resource management," Working Papers 2012-06, BC3.
- Schuwirth, N. & Reichert, P. & Lienert, J., 2012. "Methodological aspects of multi-criteria decision analysis for policy support: A case study on pharmaceutical removal from hospital wastewater," European Journal of Operational Research, Elsevier, vol. 220(2), pages 472-483.
- Falagario, Marco & Sciancalepore, Fabio & Costantino, Nicola & Pietroforte, Roberto, 2012. "Using a DEA-cross efficiency approach in public procurement tenders," European Journal of Operational Research, Elsevier, vol. 218(2), pages 523-529.
- Moshkovich, Helen M. & Mechitov, Alexander I. & Olson, David L., 2002. "Ordinal judgments in multiattribute decision analysis," European Journal of Operational Research, Elsevier, vol. 137(3), pages 625-641, March.
- Van Ittersum, Koert & Pennings, Joost M.E. & Wansink, Brian & van Trijp, Hans C.M., 2007. "The validity of attribute-importance measurement: A review," Journal of Business Research, Elsevier, vol. 60(11), pages 1177-1190, November.
- Attema, Arthur & Brouwer, Werner, 2012.
"In search of a preferred preference elicitation method: A test of the internal consistency of choice and matching tasks,"
36100, University Library of Munich, Germany.
- Attema, Arthur E. & Brouwer, Werner B.F., 2013. "In search of a preferred preference elicitation method: A test of the internal consistency of choice and matching tasks," Journal of Economic Psychology, Elsevier, vol. 39(C), pages 126-140.
- Bana e Costa, Carlos A. & Carnero, María Carmen & Oliveira, Mónica Duarte, 2012. "A multi-criteria model for auditing a Predictive Maintenance Programme," European Journal of Operational Research, Elsevier, vol. 217(2), pages 381-393.
- Meløn, Mønica García & Aragonés Beltran, Pablo & Carmen González Cruz, M., 2008. "An AHP-based evaluation procedure for Innovative Educational Projects: A face-to-face vs. computer-mediated case study," Omega, Elsevier, vol. 36(5), pages 754-765, October.
- Suk, Kwanho & Yoon, Song-Oh, 2012. "The moderating role of decision task goals in attribute weight convergence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 118(1), pages 37-45.
- Leon, Orfelio G., 1997. "On the Death of SMART and the Birth of GRAPA," Organizational Behavior and Human Decision Processes, Elsevier, vol. 71(3), pages 249-262, September.
- Teich, Jeffrey E. & Wallenius, Hannele & Wallenius, Jyrki & Koppius, Otto R., 2004. "Emerging multiple issue e-auctions," European Journal of Operational Research, Elsevier, vol. 159(1), pages 1-16, November.
- Poyhonen, Mari & Vrolijk, Hans & Hamalainen, Raimo P., 2001. "Behavioral and procedural consequences of structural variation in value trees," European Journal of Operational Research, Elsevier, vol. 134(1), pages 216-227, October.
- José A. Gómez-Limón & Ignacio Atance, 2004. "Identification of Public Objectives Related to Agricultural Sector Support," Economic Working Papers at Centro de Estudios Andaluces E2004/57, Centro de Estudios Andaluces.
- Gomez-Limon, J.A. & Atance, I., 2004. "Identification of public objectives related to agricultural sector support," Journal of Policy Modeling, Elsevier, vol. 26(8-9), pages 1045-1071, December.
- Butler, John & Jia, Jianmin & Dyer, James, 1997. "Simulation techniques for the sensitivity analysis of multi-criteria decision models," European Journal of Operational Research, Elsevier, vol. 103(3), pages 531-546, December.
- Poyhonen, Mari & Hamalainen, Raimo P., 2001. "On the convergence of multiattribute weighting methods," European Journal of Operational Research, Elsevier, vol. 129(3), pages 569-585, March.
- Huber, Joel & Ariely, Dan & Fischer, Gregory, 2002. "Expressing Preferences in a Principal-Agent Task: A Comparison of Choice, Rating, and Matching," Organizational Behavior and Human Decision Processes, Elsevier, vol. 87(1), pages 66-90, January.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mirko Janc).
If references are entirely missing, you can add them using this form.