IDEAS home Printed from https://ideas.repec.org/a/bla/ajarec/v52y2008i3p253-282.html
   My bibliography  Save this article

Design efficiency for non-market valuation with choice modelling: how to measure it, what to report and why

Author

Listed:
  • Riccardo Scarpa
  • John M. Rose

Abstract

We review the basic principles for the evaluation of design efficiency in discrete choice modelling with a focus on efficiency of WTP estimates from the multinomial logit model. The discussion is developed under the realistic assumption that researchers can plausibly define a prior belief on the range of values for the utility coefficients. D-, A-, B-, S- and C-errors are compared as measures of design performance in applied studies and their rationale is discussed. An empirical example based on the generation and comparison of fifteen separate designs from a common set of assumptions illustrates the relevant considerations to the context of non-market valuation, with particular emphasis placed on C-efficiency. Conclusions are drawn for the practice of reporting in non-market valuation and for future work on design research. Copyright 2008 The Authors. Journal compilation 2008 Australian Agricultural and Resource Economics Society Inc. and Blackwell Publishing Asia Pty Ltd.

Suggested Citation

  • Riccardo Scarpa & John M. Rose, 2008. "Design efficiency for non-market valuation with choice modelling: how to measure it, what to report and why ," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 52(3), pages 253-282, September.
  • Handle: RePEc:bla:ajarec:v:52:y:2008:i:3:p:253-282
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/j.1467-8489.2007.00436.x
    File Function: link to full text
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. David Hensher, 2006. "Revealing Differences in Willingness to Pay due to the Dimensionality of Stated Choice Designs: An Initial Assessment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 34(1), pages 7-44, May.
    2. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555, December.
    3. Ferrini, Silvia & Scarpa, Riccardo, 2007. "Designs with a priori information for nonmarket valuation with choice experiments: A Monte Carlo study," Journal of Environmental Economics and Management, Elsevier, vol. 53(3), pages 342-363, May.
    4. DeShazo, J. R. & Fermo, German, 2002. "Designing Choice Sets for Stated Preference Methods: The Effects of Complexity on Choice Consistency," Journal of Environmental Economics and Management, Elsevier, vol. 44(1), pages 123-143, July.
    5. Riccardo Scarpa & Danny Campbell & W. George Hutchinson, 2007. "Benefit Estimates for Landscape Improvements: Sequential Bayesian Design and Respondents’ Rationality in a Choice Experiment," Land Economics, University of Wisconsin Press, vol. 83(4), pages 617-634.
    6. Louviere,Jordan J. & Hensher,David A. & Swait,Joffre D. With contributions by-Name:Adamowicz,Wiktor, 2000. "Stated Choice Methods," Cambridge Books, Cambridge University Press, number 9780521788304.
    7. Caussade, Sebastián & Ortúzar, Juan de Dios & Rizzi, Luis I. & Hensher, David A., 2005. "Assessing the influence of design dimensions on stated choice experiment estimates," Transportation Research Part B: Methodological, Elsevier, vol. 39(7), pages 621-640, August.
    8. Alberini Anna, 1995. "Optimal Designs for Discrete Choice Contingent Valuation Surveys: Single-Bound, Double-Bound, and Bivariate Models," Journal of Environmental Economics and Management, Elsevier, vol. 28(3), pages 287-306, May.
    9. David A. Hensher, 2004. "Identifying the Influence of Stated Choice Design Dimensionality on Willingness to Pay for Travel Time Savings," Journal of Transport Economics and Policy, University of Bath, vol. 38(3), pages 425-446, September.
    10. William S. Breffle & Robert D. Rowe, 2002. "Comparing Choice Question Formats for Evaluating Natural Resource Tradeoffs," Land Economics, University of Wisconsin Press, vol. 78(2), pages 298-314.
    11. David A. Hensher, 2006. "How do respondents process stated choice experiments? Attribute consideration under varying information load," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(6), pages 861-878.
    12. John Rose & Michiel Bliemer, 2013. "Sample size requirements for stated choice experiments," Transportation, Springer, vol. 40(5), pages 1021-1041, September.
    13. Olivier Toubia & John R. Hauser, 2007. "—On Managerially Efficient Experimental Designs," Marketing Science, INFORMS, vol. 26(6), pages 851-858, 11-12.
    14. Hensher, David A. & Stopher, Peter R. & Louviere, Jordan J., 2001. "An exploratory analysis of the effect of numbers of choice sets in designed choice experiments: an airline choice application," Journal of Air Transport Management, Elsevier, vol. 7(6), pages 373-379.
    15. Barbara J. Kanninen, 1993. "Optimal Experimental Design for Double-Bounded Dichotomous Choice Contingent Valuation," Land Economics, University of Wisconsin Press, vol. 69(2), pages 138-146.
    16. Arentze, Theo & Borgers, Aloys & Timmermans, Harry & DelMistro, Romano, 2003. "Transport stated choice responses: effects of task complexity, presentation format and literacy," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 39(3), pages 229-244, May.
    17. Jayson L. Lusk & F. Bailey Norwood, 2005. "Effect of Experimental Design on Choice-Based Conjoint Valuation Estimates," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 87(3), pages 771-785.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Jui-Chen & Johnson, F. Reed & Kilambi, Vikram & Mohamed, Ateesha F., 2015. "Sample size and utility-difference precision in discrete-choice experiments: A meta-simulation approach," Journal of choice modelling, Elsevier, vol. 16(C), pages 50-57.
    2. Silvia Ferrini & Riccardo Scarpa, 2005. "Experimental Designs for Environmental Valuation with Choice-Experiments: A Monte-Carlo Investigation," Working Papers in Economics 05/08, University of Waikato.
    3. Jürgen Meyerhoff & Malte Oehlmann & Priska Weller, 2015. "The Influence of Design Dimensions on Stated Choices in an Environmental Context," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 61(3), pages 385-407, July.
    4. Ferrini, Silvia & Scarpa, Riccardo, 2007. "Designs with a priori information for nonmarket valuation with choice experiments: A Monte Carlo study," Journal of Environmental Economics and Management, Elsevier, vol. 53(3), pages 342-363, May.
    5. Weng, Weizhe & Morrison, Mark D. & Boyle, Kevin J. & Boxall, Peter C. & Rose, John, 2021. "Effects of the number of alternatives in public good discrete choice experiments," Ecological Economics, Elsevier, vol. 182(C).
    6. Fraser, Iain & Balcombe, Kelvin & Williams, Louis & McSorley, Eugene, 2021. "Preference stability in discrete choice experiments. Some evidence using eye-tracking," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 94(C).
    7. Marit E. Kragt & Jeff Bennett, 2008. "Developing a Questionnaire for Valuing Changes in Natural Resource Management in the George Catchment, Tasmania," Environmental Economics Research Hub Research Reports 0808, Environmental Economics Research Hub, Crawford School of Public Policy, The Australian National University.
    8. Weng, Weizhe & Morrison, Mark & Boyle, Kevin & Boxall, Peter, 2017. "The effect of the number of alternatives in a choice experiment with an application to the Macquarie Marshes, AU," 2017 Annual Meeting, February 4-7, 2017, Mobile, Alabama 252836, Southern Agricultural Economics Association.
    9. Marit Kragt & Jeffrey Bennett, 2012. "Attribute Framing in Choice Experiments: How Do Attribute Level Descriptions Affect Value Estimates?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 51(1), pages 43-59, January.
    10. Andrew Collins & John Rose & Stephane Hess, 2012. "Interactive stated choice surveys: a study of air travel behaviour," Transportation, Springer, vol. 39(1), pages 55-79, January.
    11. Rose, John M. & Hensher, David A. & Caussade, Sebastian & Ortúzar, Juan de Dios & Jou, Rong-Chang, 2009. "Identifying differences in willingness to pay due to dimensionality in stated choice experiments: a cross country analysis," Journal of Transport Geography, Elsevier, vol. 17(1), pages 21-29.
    12. Bliemer, Michiel C.J. & Rose, John M., 2011. "Experimental design influences on stated choice outputs: An empirical study in air travel choice," Transportation Research Part A: Policy and Practice, Elsevier, vol. 45(1), pages 63-79, January.
    13. Ladenburg, Jacob & Olsen, Søren Bøye, 2014. "Augmenting short Cheap Talk scripts with a repeated Opt-Out Reminder in Choice Experiment surveys," Resource and Energy Economics, Elsevier, vol. 37(C), pages 39-63.
    14. Rose, John M. & Bliemer, Michiel C.J. & Hensher, David A. & Collins, Andrew T., 2008. "Designing efficient stated choice experiments in the presence of reference alternatives," Transportation Research Part B: Methodological, Elsevier, vol. 42(4), pages 395-406, May.
    15. Bruno Lanz & Allan Provins, 2013. "Valuing Local Environmental Amenity with Discrete Choice Experiments: Spatial Scope Sensitivity and Heterogeneous Marginal Utility of Income," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 56(1), pages 105-130, September.
    16. Elizabeth Kinter & Thomas Prior & Christopher Carswell & John Bridges, 2012. "A Comparison of Two Experimental Design Approaches in Applying Conjoint Analysis in Patient-Centered Outcomes Research," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 5(4), pages 279-294, December.
    17. Pascal Haegeli & Wolfgang Haider & Margo Longland & Ben Beardmore, 2010. "Amateur decision-making in avalanche terrain with and without a decision aid: a stated choice survey," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 52(1), pages 185-209, January.
    18. Campbell, Danny & Hutchinson, W. George & Scarpa, Riccardo, 2006. "Lexicographic Preferences in Discrete Choice Experiments: Consequences on Individual-Specific Willingness to Pay Estimates," Sustainability Indicators and Environmental Valuation Working Papers 12224, Fondazione Eni Enrico Mattei (FEEM).
    19. Mikołaj Czajkowski & Marek Giergiczny & William H. Greene, 2012. "Learning and Fatigue Effects Revisited. The Impact of Accounting for Unobservable Preference and Scale Heterogeneity on Perceived Ordering Effects in Multiple Choice Task Discrete Choice Experiments," Working Papers 2012-08, Faculty of Economic Sciences, University of Warsaw.
    20. Richard Yao & Riccardo Scarpa & John Rose & James Turner, 2015. "Experimental Design Criteria and Their Behavioural Efficiency: An Evaluation in the Field," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 62(3), pages 433-455, November.

    More about this item

    JEL classification:

    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • Q51 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Environmental Economics - - - Valuation of Environmental Effects

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:ajarec:v:52:y:2008:i:3:p:253-282. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/aaresea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/aaresea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.