IDEAS home Printed from https://ideas.repec.org/a/spr/empeco/v53y2017i4d10.1007_s00181-016-1169-2.html
   My bibliography  Save this article

Improving the prediction of ranking data

Author

Listed:
  • Marco A. Palma

    (Texas A&M University)

Abstract

By using the same number of alternatives for every respondent, all ranking elicitation methods in the literature including full, partial, and best–worst rankings assume respondents know and are able to rank the same number of alternatives. A simple survey elicitation mechanism allowing for individual heterogeneity in the number of rankings for ranked-ordered data is proposed. Using the proposed ranking mechanism as a data augmentation tool yields higher prediction of ranking choices compared to conventional rankings and best–worst methods. The results provide robust evidence of differences in error variance scale and the structure of the underlying utility preferences across ranking stages, including best–worst rankings. The highest predictive power was achieved with the proposed ranking method using only the best ranked alternative. Including any additional rankings other than the best alternative reduces predictive power. Nevertheless, if more than one ranking is used to model preferences, then better predictions are achieved by using the top two best ranked alternatives as supposed to the exploded best–worst rankings. The results stand as a warning about equating ranking choices to true underlying utility preferences across different ranking elicitation stages or mechanisms without properly testing for symmetry and stability of preferences.

Suggested Citation

  • Marco A. Palma, 2017. "Improving the prediction of ranking data," Empirical Economics, Springer, vol. 53(4), pages 1681-1710, December.
    • Handle: RePEc:spr:empeco:v:53:y:2017:i:4:d:10.1007_s00181-016-1169-2
    DOI: 10.1007/s00181-016-1169-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00181-016-1169-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00181-016-1169-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Corrigan, Jay R. & Rousu, Matthew C., 2006. "AJAE Appendix: Posted Prices and Bid Affiliation: Evidence from Experimental Auctions," American Journal of Agricultural Economics APPENDICES, Agricultural and Applied Economics Association, vol. 88(4), pages 1-12, November.
    2. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555, January.
    3. repec:ken:wpaper:0602 is not listed on IDEAS
    4. Hausman, Jerry A. & Ruud, Paul A., 1987. "Specifying and testing econometric models for rank-ordered data," Journal of Econometrics, Elsevier, vol. 34(1-2), pages 83-104.
    5. Chiou, Lesley & Walker, Joan L., 2007. "Masking identification of discrete choice models under simulation methods," Journal of Econometrics, Elsevier, vol. 141(2), pages 683-703, December.
    6. Flynn, Terry N. & Louviere, Jordan J. & Peters, Tim J. & Coast, Joanna, 2007. "Best-worst scaling: What it can do for health care research and how to do it," Journal of Health Economics, Elsevier, vol. 26(1), pages 171-189, January.
    7. Dennis Fok & Richard Paap & Bram Van Dijk, 2012. "A Rank‐Ordered Logit Model With Unobserved Heterogeneity In Ranking Capabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(5), pages 831-846, August.
    8. Pat Auger & Timothy Devinney & Jordan Louviere, 2007. "Using Best–Worst Scaling Methodology to Investigate Consumer Ethical Beliefs Across Countries," Journal of Business Ethics, Springer, vol. 70(3), pages 299-326, February.
    9. William Vickrey, 1961. "Counterspeculation, Auctions, And Competitive Sealed Tenders," Journal of Finance, American Finance Association, vol. 16(1), pages 8-37, March.
    10. Deacue Fields & Walt Prevatt, 2008. "An Incentive Compatible Conjoint Ranking Mechanism," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 90(2), pages 487-498.
    11. Louviere, Jordan & Lings, Ian & Islam, Towhidul & Gudergan, Siegfried & Flynn, Terry, 2013. "An introduction to the application of (case 1) best–worst scaling in marketing research," International Journal of Research in Marketing, Elsevier, vol. 30(3), pages 292-303.
    12. John Calfee & Clifford Winston & Randolph Stempski, 2001. "Econometric Issues In Estimating Consumer Preferences From Stated Preference Data: A Case Study Of The Value Of Automobile Travel Time," The Review of Economics and Statistics, MIT Press, vol. 83(4), pages 699-707, November.
    13. Koop, G & Poirier, D J, 1994. "Rank-Ordered Logit Models: An Empirical Analysis of Ontario Voter Preferences," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 9(4), pages 369-388, Oct.-Dec..
    14. David A. Hensher & John M. Rose, 2012. "The Influence of Alternative Acceptability, Attribute Thresholds and Choice Response Certainty on Automobile Purchase Preferences," Journal of Transport Economics and Policy, University of Bath, vol. 46(3), pages 451-468, September.
    15. Chiang, Jeongwen & Chib, Siddhartha & Narasimhan, Chakravarthi, 1998. "Markov chain Monte Carlo and models of consideration set and parameter heterogeneity," Journal of Econometrics, Elsevier, vol. 89(1-2), pages 223-248, November.
    16. Tjur, Tue, 2009. "Coefficients of Determination in Logistic Regression Models—A New Proposal: The Coefficient of Discrimination," The American Statistician, American Statistical Association, vol. 63(4), pages 366-372.
    17. Beggs, S. & Cardell, S. & Hausman, J., 1981. "Assessing the potential demand for electric cars," Journal of Econometrics, Elsevier, vol. 17(1), pages 1-19, September.
    18. McFadden, Daniel, 1974. "The measurement of urban travel demand," Journal of Public Economics, Elsevier, vol. 3(4), pages 303-328, November.
    19. Timothy N. Cason & Charles R. Plott, 2014. "Misconceptions and Game Form Recognition: Challenges to Theories of Revealed Preference and Framing," Journal of Political Economy, University of Chicago Press, vol. 122(6), pages 1235-1270.
    20. Hess, Stephane & Rose, John M., 2009. "Allowing for intra-respondent variations in coefficients estimated on repeated choice data," Transportation Research Part B: Methodological, Elsevier, vol. 43(6), pages 708-719, July.
    21. van Ophem, Hans & Stam, Piet & Van Praag, Bernard M S, 1999. "Multichoice Logit: Modeling Incomplete Preference Rankings of Classical Concerts," Journal of Business & Economic Statistics, American Statistical Association, vol. 17(1), pages 117-128, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chen, Lijun & House, Lisa A., 2020. "The moderator effect of food lifestyle on the relationship between aging and health," 2020 Annual Meeting, July 26-28, Kansas City, Missouri 304453, Agricultural and Applied Economics Association.
    2. Shanna Trichês Lucchesi & Virginia Bergamaschi Tavares & Miriam Karla Rocha & Ana Margarita Larranaga, 2022. "Public Transport COVID-19-Safe: New Barriers and Policies to Implement Effective Countermeasures under User’s Safety Perspective," Sustainability, MDPI, vol. 14(5), pages 1-19, March.
    3. Samare P. I. Huls & Emily Lancsar & Bas Donkers & Jemimah Ride, 2022. "Two for the price of one: If moving beyond traditional single‐best discrete choice experiments, should we use best‐worst, best‐best or ranking for preference elicitation?," Health Economics, John Wiley & Sons, Ltd., vol. 31(12), pages 2630-2647, December.
    4. Ben Aoki-Sherwood & Catherine Bregou & David Liben-Nowell & Kiran Tomlinson & Thomas Zeng, 2024. "Bounding Consideration Probabilities in Consider-Then-Choose Ranking Models," Papers 2401.11016, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dennis Fok & Richard Paap & Bram Van Dijk, 2012. "A Rank‐Ordered Logit Model With Unobserved Heterogeneity In Ranking Capabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(5), pages 831-846, August.
    2. Yoo, Hong Il & Doiron, Denise, 2013. "The use of alternative preference elicitation methods in complex discrete choice experiments," Journal of Health Economics, Elsevier, vol. 32(6), pages 1166-1179.
    3. José L. Oviedo & Hong Il Yoo, 2017. "A Latent Class Nested Logit Model for Rank-Ordered Data with Application to Cork Oak Reforestation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 68(4), pages 1021-1051, December.
    4. Anoek Castelein & Dennis Fok & Richard Paap, 2020. "A multinomial and rank-ordered logit model with inter- and intra-individual heteroscedasticity," Tinbergen Institute Discussion Papers 20-069/III, Tinbergen Institute.
    5. Ben Aoki-Sherwood & Catherine Bregou & David Liben-Nowell & Kiran Tomlinson & Thomas Zeng, 2024. "Bounding Consideration Probabilities in Consider-Then-Choose Ranking Models," Papers 2401.11016, arXiv.org.
    6. Kockelman, Kara M. & Podgorski, Kaethe & Bina, Michelle & Gadda, Shashank, 2009. "Public Perceptions of Pricing Existing Roads and Other Transportation Policies: The Texas Perspective," Journal of the Transportation Research Forum, Transportation Research Forum, vol. 48(3).
    7. Hong il Yoo, 2012. "The perceived unreliability of rank-ordered data: an econometric origin and implications," Discussion Papers 2012-46, School of Economics, The University of New South Wales.
    8. Denise Doiron & Hong Il Yoo, 2020. "Stated preferences over job characteristics: A panel study," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 43-82, February.
    9. T.N. Flynn & A.A.J. Marley, 2014. "Best-worst scaling: theory and methods," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 8, pages 178-201, Edward Elgar Publishing.
    10. Samare P. I. Huls & Emily Lancsar & Bas Donkers & Jemimah Ride, 2022. "Two for the price of one: If moving beyond traditional single‐best discrete choice experiments, should we use best‐worst, best‐best or ranking for preference elicitation?," Health Economics, John Wiley & Sons, Ltd., vol. 31(12), pages 2630-2647, December.
    11. Lancsar, Emily & Louviere, Jordan & Donaldson, Cam & Currie, Gillian & Burgess, Leonie, 2013. "Best worst discrete choice experiments in health: Methods and an application," Social Science & Medicine, Elsevier, vol. 76(C), pages 74-82.
    12. Haghani, Milad & Bliemer, Michiel C.J. & Hensher, David A., 2021. "The landscape of econometric discrete choice modelling research," Journal of choice modelling, Elsevier, vol. 40(C).
    13. Yan, Jin & Yoo, Hong Il, 2019. "Semiparametric estimation of the random utility model with rank-ordered choice data," Journal of Econometrics, Elsevier, vol. 211(2), pages 414-438.
    14. John Rose & David Hensher, 2014. "Tollroads are only part of the overall trip: the error of our ways in past willingness to pay studies," Transportation, Springer, vol. 41(4), pages 819-837, July.
    15. Bairagi, Subir K. & Mohanty, Samarendu & Ynion, Jhoanne & Demont, Matty, 2017. "Determinants of Consumer Preferences for Rice Attributes: Evidence from South and Southeast Asia," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 258384, Agricultural and Applied Economics Association.
    16. Denise Doiron & Jane Hall & Patricia Kenny & Deborah J. Street, 2014. "Job preferences of students and new graduates in nursing," Applied Economics, Taylor & Francis Journals, vol. 46(9), pages 924-939, March.
    17. Wamisho, Kassu & De Laporte, Aaron & Ripplinger, David, 2015. "Biomass Contracts for Ethanol Production: The Role of Farmer’s Risk Preferences," 2015 AAEA & WAEA Joint Annual Meeting, July 26-28, San Francisco, California 205703, Agricultural and Applied Economics Association.
    18. Aizaki, Hideo & Fogarty, James, 2023. "R packages and tutorial for case 1 best–worst scaling," Journal of choice modelling, Elsevier, vol. 46(C).
    19. Costanigro, Marco & Deselnicu, Oana & Kroll, Stephan, 2012. "Truthful, Misguiding Labels: The Implications of Labeling Production Processes rather than their Outcomes," 2012 Annual Meeting, August 12-14, 2012, Seattle, Washington 124615, Agricultural and Applied Economics Association.
    20. Kassu Wamisho Hossiso & Aaron Laporte & David Ripplinger, 2017. "The Effects of Contract Mechanism Design and Risk Preferences on Biomass Supply for Ethanol Production," Agribusiness, John Wiley & Sons, Ltd., vol. 33(3), pages 339-357, June.

    More about this item

    Keywords

    Best–worst; Error variance; Random parameters; Scale parameter; Stability of preferences;
    All these keywords.

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:empeco:v:53:y:2017:i:4:d:10.1007_s00181-016-1169-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.