IDEAS home Printed from
   My bibliography  Save this paper

The first time is the hardest: A test of ordering effects in choice experiments


  • Carlsson, Fredrik

    () (Department of Economics, School of Business, Economics and Law, Göteborg University)

  • Raun Mørkbak, Morten

    () (Institute of Food and Resource Economics, University of Copenhagen)

  • Bøye Olsen, Søren

    () (Institute of Food and Resource Economics, University of Copenhagen)


This paper addresses the issue of ordering effects in choice experiments, and in particular how learning processes potentially affect respondents’ stated preferences in a sequence of choice sets. In a case study concerning food quality attributes of chicken breast filets, we find evidence of ordering effects in a sequence of 16 choice sets, where the last 8 choice sets are identical to the first 8. The overall preference structure is found to differ significantly between the two identical sequences of choice sets, and significant increases in marginal WTP are found for two out of four attributes. We find a reduction in the error variance for the last 8 choice sets relative to the first 8 choice sets. In particular, this difference is ascribed to the first choice set obtaining a significantly higher error variance than all succeeding choice sets, suggesting institutional learning rather than preference learning effects underlying the observed ordering effect. This is further supported by the fact that the differences in WTP become insignificant when removing the first choice set from the analysis. We find no evidence of fatigue, and we argue that our findings cannot be explained by starting point or strategic behavior effects.

Suggested Citation

  • Carlsson, Fredrik & Raun Mørkbak, Morten & Bøye Olsen, Søren, 2010. "The first time is the hardest: A test of ordering effects in choice experiments," Working Papers in Economics 470, University of Gothenburg, Department of Economics.
  • Handle: RePEc:hhs:gunwpe:0470

    Download full text from publisher

    File URL:
    Download Restriction: no

    References listed on IDEAS

    1. Ladenburg, Jacob & Olsen, Søren Bøye, 2008. "Gender-specific starting point bias in choice experiments: Evidence from an empirical study," Journal of Environmental Economics and Management, Elsevier, vol. 56(3), pages 275-285, November.
    2. Day, Brett & Pinto Prades, Jose-Luis, 2010. "Ordering anomalies in choice experiments," Journal of Environmental Economics and Management, Elsevier, vol. 59(3), pages 271-285, May.
    3. Scott J. Savage & Donald M. Waldman, 2008. "Learning and fatigue during choice experiments: a comparison of online and mail survey modes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 351-371.
    4. Swait, Joffre & Adamowicz, Wiktor, 2001. " The Influence of Task Complexity on Consumer Choice: A Latent Class Model of Decision Strategy Switching," Journal of Consumer Research, Oxford University Press, vol. 28(1), pages 135-148, June.
    5. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521747387, March.
    6. Roy Brouwer & Thijs Dekker & John Rolfe & Jill Windle, 2010. "Choice Certainty and Consistency in Repeated Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 46(1), pages 93-109, May.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Aravena, Claudia & Hutchinson, W. George & Carlsson, Fredrik & Matthews, David I, 2015. "Testing preference formation in learning design contingent valuation (LDCV) using advanced information and repetitivetreatments," Working Papers in Economics 619, University of Gothenburg, Department of Economics.
    2. Xiaoying Liu, Mare Sarr and Timothy Swanson, 2014. "Resistance to the Regulation of Common Resources in Rural Tunisia," Working Papers 414, Economic Research Southern Africa.
    3. Mikołaj Czajkowski & Marek Giergiczny & William H. Greene, 2012. "Learning and Fatigue Effects Revisited. The Impact of Accounting for Unobservable Preference and Scale Heterogeneity on Perceived Ordering Effects in Multiple Choice Task Discrete Choice Experiments," Working Papers 2012-08, Faculty of Economic Sciences, University of Warsaw.
    4. Danny Campbell & Seda Erdem, 2015. "Position Bias in Best-worst Scaling Surveys: A Case Study on Trust in Institutions," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 97(2), pages 526-545.
    5. Dekker, T. & Koster, P.R. & Brouwer, R., 2012. "Changing with the tide: Semi-parametric estimation of preference dynamics," Serie Research Memoranda 0005, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics.
    6. Jacob L. Orquin & Martin P. Bagger & Simone Mueller Loose, 2013. "Learning affects top down and bottom up modulation of eye movements in decision making," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 8(6), pages 700-716, November.

    More about this item


    Choice Experiments; Fatigue; Learning; Ordering Effects; WTP;

    JEL classification:

    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hhs:gunwpe:0470. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Marie Andersson). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.