Comparison of Weighting Judgments in Multiattribute Utility Measurement
AbstractThis paper compares four weighting methods in multiattribute utility measurement: the ratio method, the swing weighting method, the tradeoff method and the pricing out method. 200 subjects used these methods to weight attributes for evaluating nuclear waste repository sites in the United States. The weighting methods were compared with respect to their internal consistency, convergent validity, and external validity. Internal consistency was measured by the degree to which ordinal and cardinal or ratio responses agreed within the same weighting method. Convergent validity was measured by the degree of agreement between the weights elicited with different methods. External validity was determined by the degree to which weights elicited in this experiment agreed with weights that were elicited with managers of the Department of Energy. In terms of internal consistency, the tradeoff method fared worst. In terms of convergent validity, the pricing out method turned out to be an outlier. In terms of external validity, the pricing out method showed the best results. While the ratio and swing methods are quite consistent and show a fair amount of convergent validity, their external validity problems cast doubt on their usefulness. The main recommendation for applications is to improve the internal consistency of the tradeoff method by careful interactive elicitation and to use it in conjunction with the pricing out method to enhance its external validity.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoArticle provided by INFORMS in its journal Management Science.
Volume (Year): 37 (1991)
Issue (Month): 12 (December)
tradeoffs; weights; multiattribute utility; nuclear waste disposal; decision analysis;
You can help add them by filling out this form.
CitEc Project, subscribe to its RSS feed for this item.
- Falagario, Marco & Sciancalepore, Fabio & Costantino, Nicola & Pietroforte, Roberto, 2012. "Using a DEA-cross efficiency approach in public procurement tenders," European Journal of Operational Research, Elsevier, vol. 218(2), pages 523-529.
- Huber, Joel & Ariely, Dan & Fischer, Gregory, 2002. "Expressing Preferences in a Principal-Agent Task: A Comparison of Choice, Rating, and Matching," Organizational Behavior and Human Decision Processes, Elsevier, vol. 87(1), pages 66-90, January.
- Schuwirth, N. & Reichert, P. & Lienert, J., 2012. "Methodological aspects of multi-criteria decision analysis for policy support: A case study on pharmaceutical removal from hospital wastewater," European Journal of Operational Research, Elsevier, vol. 220(2), pages 472-483.
- Poyhonen, Mari & Hamalainen, Raimo P., 2001. "On the convergence of multiattribute weighting methods," European Journal of Operational Research, Elsevier, vol. 129(3), pages 569-585, March.
- Attema, Arthur & Brouwer, Werner, 2012.
"In search of a preferred preference elicitation method: A test of the internal consistency of choice and matching tasks,"
36100, University Library of Munich, Germany.
- Attema, Arthur E. & Brouwer, Werner B.F., 2013. "In search of a preferred preference elicitation method: A test of the internal consistency of choice and matching tasks," Journal of Economic Psychology, Elsevier, vol. 39(C), pages 126-140.
- Gomez-Limon, J.A. & Atance, I., 2004. "Identification of public objectives related to agricultural sector support," Journal of Policy Modeling, Elsevier, vol. 26(8-9), pages 1045-1071, December.
- Suk, Kwanho & Yoon, Song-Oh, 2012. "The moderating role of decision task goals in attribute weight convergence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 118(1), pages 37-45.
- Butler, John & Jia, Jianmin & Dyer, James, 1997. "Simulation techniques for the sensitivity analysis of multi-criteria decision models," European Journal of Operational Research, Elsevier, vol. 103(3), pages 531-546, December.
- Sam Park, Kyung & Sang Lee, Kyung & Seong Eum, Yun & Park, Kwangtae, 2001. "Extended methods for identifying dominance and potential optimality in multi-criteria analysis with imprecise information," European Journal of Operational Research, Elsevier, vol. 134(3), pages 557-563, November.
- Meløn, Mønica García & Aragonés Beltran, Pablo & Carmen González Cruz, M., 2008. "An AHP-based evaluation procedure for Innovative Educational Projects: A face-to-face vs. computer-mediated case study," Omega, Elsevier, vol. 36(5), pages 754-765, October.
- Moshkovich, Helen M. & Mechitov, Alexander I. & Olson, David L., 2002. "Ordinal judgments in multiattribute decision analysis," European Journal of Operational Research, Elsevier, vol. 137(3), pages 625-641, March.
- Teich, Jeffrey E. & Wallenius, Hannele & Wallenius, Jyrki & Koppius, Otto R., 2004. "Emerging multiple issue e-auctions," European Journal of Operational Research, Elsevier, vol. 159(1), pages 1-16, November.
- Bana e Costa, Carlos A. & Carnero, María Carmen & Oliveira, Mónica Duarte, 2012. "A multi-criteria model for auditing a Predictive Maintenance Programme," European Journal of Operational Research, Elsevier, vol. 217(2), pages 381-393.
- Eneko Garmendia & Gonzalo Gamboa, 2012. "Weighting social preferences in participatory multi-criteria evaluations: a case study on sustainable natural resource management," Working Papers 2012-06, BC3.
- Leon, Orfelio G., 1997. "On the Death of SMART and the Birth of GRAPA," Organizational Behavior and Human Decision Processes, Elsevier, vol. 71(3), pages 249-262, September.
- Poyhonen, Mari & Vrolijk, Hans & Hamalainen, Raimo P., 2001. "Behavioral and procedural consequences of structural variation in value trees," European Journal of Operational Research, Elsevier, vol. 134(1), pages 216-227, October.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mirko Janc).
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.