IDEAS home Printed from https://ideas.repec.org/a/eee/eejocm/v56y2025ics1755534525000259.html

Understanding the decision-making process of choice modellers

Author

Listed:
  • Nova, Gabriel
  • van Cranenburgh, Sander
  • Hess, Stephane

Abstract

Choice Modelling is a widely used framework for understanding human choice behaviour across disciplines. Building a choice model is a complex, semi-structured process that involves a combination of prior assumptions, behavioural theories, and statistical methods. This complex set of decisions, coupled with diverse workflows, can lead to substantial variability in model outcomes. To investigate these modelling processes, we introduce the Discrete Choice Modelling Serious Game (DCM-SG), a novel tool that mimics the workflow of choice modellers and tracks the modelling decisions participants make. In our application, participants developed models to estimate willingness-to-pay values for reducing noise pollution. Their actions were tracked, enabling analysis of workflow patterns and modelling strategies. Forty participants, most with over five years of experience, completed the game. Our contributions are twofold. Methodologically, the DCM-SG captures sequential data on modellers’ workflows, which we analyse using telemetry and sequential pattern mining techniques to uncover dynamic patterns of in-game tool usage, phase transitions, and model specification approaches. Substantively, there was a strong preference for data visualisation and frequent specification of simpler models (Multinomial Logit), alongside attempts to specify more complex models. These findings suggest that in time-constrained or resource-limited settings, modellers may underexplore important factors such as covariates, non-linearities, and complex specifications. Moreover, participants who engaged more thoroughly in data exploration and iterative model comparison consistently achieved superior model fit and parsimony. These results demonstrate how sequential data from the DCM-SG can uncover variations in modelling practices and provide a foundation for understanding the art of choice modelling.

Suggested Citation

  • Nova, Gabriel & van Cranenburgh, Sander & Hess, Stephane, 2025. "Understanding the decision-making process of choice modellers," Journal of choice modelling, Elsevier, vol. 56(C).
    • Handle: RePEc:eee:eejocm:v:56:y:2025:i:c:s1755534525000259
    DOI: 10.1016/j.jocm.2025.100562
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1755534525000259
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jocm.2025.100562?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Ortelli, Nicola & Hillel, Tim & Pereira, Francisco C. & de Lapparent, Matthieu & Bierlaire, Michel, 2021. "Assisted specification of discrete choice models," Journal of choice modelling, Elsevier, vol. 39(C).
    2. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555, August.
    3. Parady, Giancarlos & Ory, David & Walker, Joan, 2021. "The overreliance on statistical goodness-of-fit and under-reliance on model validation in discrete choice models: A review of validation practices in the transportation academic literature," Journal of choice modelling, Elsevier, vol. 38(C).
    4. repec:osf:osfxxx:umq8d_v1 is not listed on IDEAS
    5. Caussade, Sebastián & Ortúzar, Juan de Dios & Rizzi, Luis I. & Hensher, David A., 2005. "Assessing the influence of design dimensions on stated choice experiment estimates," Transportation Research Part B: Methodological, Elsevier, vol. 39(7), pages 621-640, August.
    6. Joan Walker & Jieping Li, 2007. "Latent lifestyle preferences and household location decisions," Journal of Geographical Systems, Springer, vol. 9(1), pages 77-101, April.
    7. Paz, Alexander & Arteaga, Cristian & Cobos, Carlos, 2019. "Specification of mixed logit models assisted by an optimization framework," Journal of choice modelling, Elsevier, vol. 30(C), pages 50-60.
    8. David A. Hensher, 2004. "Identifying the Influence of Stated Choice Design Dimensionality on Willingness to Pay for Travel Time Savings," Journal of Transport Economics and Policy, University of Bath, vol. 38(3), pages 425-446, September.
    9. Wicherts, Jelte M. & Veldkamp, Coosje Lisabet Sterre & Augusteijn, Hilde & Bakker, Marjan & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2016. "Degrees of freedom in planning, running, analyzing, and reporting psychological studies A checklist to avoid p-hacking," OSF Preprints umq8d, Center for Open Science.
    10. Stephane Hess & Andrew Daly, 2024. "Introduction to the Handbook of Choice Modelling," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 1, pages 1-4, Edward Elgar Publishing.
    11. Beeramoole, Prithvi Bhat & Arteaga, Cristian & Pinz, Alban & Haque, Md Mazharul & Paz, Alexander, 2023. "Extensive hypothesis testing for estimation of mixed-Logit models," Journal of choice modelling, Elsevier, vol. 47(C).
    12. Kay W. Axhausen, 2024. "Self-tracing and reporting: state-of-the-art in the capture of revealed behaviour," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 6, pages 147-171, Edward Elgar Publishing.
    13. Hess, Stephane & Palma, David, 2019. "Apollo: A flexible, powerful and customisable freeware package for choice model estimation and application," Journal of choice modelling, Elsevier, vol. 32(C), pages 1-1.
    14. Jürgen Meyerhoff & Malte Oehlmann & Priska Weller, 2015. "The Influence of Design Dimensions on Stated Choices in an Environmental Context," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 61(3), pages 385-407, July.
    15. McFadden, Daniel, 1974. "The measurement of urban travel demand," Journal of Public Economics, Elsevier, vol. 3(4), pages 303-328, November.
    16. Hensher,David A. & Rose,John M. & Greene,William H., 2015. "Applied Choice Analysis," Cambridge Books, Cambridge University Press, number 9781107465923.
    17. Daniel McFadden & Kenneth Train, 2000. "Mixed MNL models for discrete response," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(5), pages 447-470.
    18. Daly, Andrew & Hess, Stephane & de Jong, Gerard, 2012. "Calculating errors for measures derived from choice modelling estimates," Transportation Research Part B: Methodological, Elsevier, vol. 46(2), pages 333-341.
    19. Rotem Botvinik-Nezer & Felix Holzmeister & Colin F. Camerer & Anna Dreber & Juergen Huber & Magnus Johannesson & Michael Kirchler & Roni Iwanir & Jeanette A. Mumford & R. Alison Adcock & Paolo Avesani, 2020. "Variability in the analysis of a single neuroimaging dataset by many teams," Nature, Nature, vol. 582(7810), pages 84-88, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabriel Nova & Sander van Cranenburgh & Stephane Hess, 2024. "Understanding the decision-making process of choice modellers," Papers 2411.01704, arXiv.org, revised Jun 2025.
    2. Haghani, Milad & Bliemer, Michiel C.J. & Hensher, David A., 2021. "The landscape of econometric discrete choice modelling research," Journal of choice modelling, Elsevier, vol. 40(C).
    3. Weng, Weizhe & Morrison, Mark & Boyle, Kevin J. & Boxall, Peter C., "undated". "The effect of the number of alternatives in choice experiment questions," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 259179, Agricultural and Applied Economics Association.
    4. Ahi, Jülide Ceren & Aanesen, Margrethe & Kipperberg, Gorm, 2023. "Testing the sensitivity of stated environmental preferences to variations in choice architecture," Ecological Economics, Elsevier, vol. 205(C).
    5. Weng, Weizhe & Morrison, Mark & Boyle, Kevin & Boxall, Peter, 2017. "The effect of the number of alternatives in a choice experiment with an application to the Macquarie Marshes, AU," 2017 Annual Meeting, February 4-7, 2017, Mobile, Alabama 252836, Southern Agricultural Economics Association.
    6. Beeramoole, Prithvi Bhat & Arteaga, Cristian & Pinz, Alban & Haque, Md Mazharul & Paz, Alexander, 2023. "Extensive hypothesis testing for estimation of mixed-Logit models," Journal of choice modelling, Elsevier, vol. 47(C).
    7. Qian, Lixian & Grisolía, Jose M. & Soopramanien, Didier, 2019. "The impact of service and government-policy attributes on consumer preferences for electric vehicles in China," Transportation Research Part A: Policy and Practice, Elsevier, vol. 122(C), pages 70-84.
    8. Ju-Hee Kim & Younggew Kim & Seung-Hoon Yoo, 2021. "Using a choice experiment to explore the public willingness to pay for the impacts of improving energy efficiency of an apartment," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(5), pages 1775-1793, October.
    9. Uemura, Hiroshi & Parady, Giancarlos & Takami, Kiyoshi, 2026. "Analyzing preferences for shared micromobility services in Japanese urban areas," Transportation Research Part A: Policy and Practice, Elsevier, vol. 206(C).
    10. Isler, Cassiano Augusto & Blumenfeld, Marcelo & Caldeira, Gabriel Pereira & Roberts, Clive, 2024. "Long-Distance railway mode choice in Brazil: Evidence from a discrete choice experiment," Research in Transportation Economics, Elsevier, vol. 104(C).
    11. Kim, Sung Hoo & Mokhtarian, Patricia L., 2023. "Finite mixture (or latent class) modeling in transportation: Trends, usage, potential, and future directions," Transportation Research Part B: Methodological, Elsevier, vol. 172(C), pages 134-173.
    12. Schmid, Basil & Becker, Felix & Axhausen, Kay W. & Widmer, Paul & Stein, Petra, 2023. "A simultaneous model of residential location, mobility tool ownership and mode choice using latent variables," Transportation Research Part A: Policy and Practice, Elsevier, vol. 178(C).
    13. Haque, Md Bashirul & Choudhury, Charisma & Hess, Stephane, 2020. "Understanding differences in residential location preferences between ownership and renting: A case study of London," Journal of Transport Geography, Elsevier, vol. 88(C).
    14. Sharma, Reema Bera & Majumdar, Bandhan Bandhu & Maitra, Bhargab, 2024. "Commuter and non-commuter preferences for plug-in hybrid electric vehicle: A case study of Delhi and Kolkata, India," Research in Transportation Economics, Elsevier, vol. 103(C).
    15. Tinessa, Fiore & Marzano, Vittorio & Papola, Andrea, 2020. "Mixing distributions of tastes with a Combination of Nested Logit (CoNL) kernel: Formulation and performance analysis," Transportation Research Part B: Methodological, Elsevier, vol. 141(C), pages 1-23.
    16. Kim, Ju-Hee & Yoo, Seung-Hoon, 2020. "Public perspective on the environmental impacts of sea sand mining: Evidence from a choice experiment in South Korea," Resources Policy, Elsevier, vol. 69(C).
    17. Dylan Thompson, 2024. "Front row or backstage? Evidence on concert ticket preferences from a discrete choice experiment," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 48(3), pages 463-491, September.
    18. Campbell, Danny, 2007. "Combining mixed logit models and random effects models to identify the determinants of willingness to pay for rural landscape improvements," 81st Annual Conference, April 2-4, 2007, Reading University, UK 7975, Agricultural Economics Society.
    19. Zhang, Hanyuan & Qiu, Richard T.R. & Wen, Long & Song, Haiyan & Liu, Chang, 2023. "Has COVID-19 changed tourist destination choice?," Annals of Tourism Research, Elsevier, vol. 103(C).
    20. Oehlmann, Malte & Meyerhoff, Jürgen & Mariel, Petr & Weller, Priska, 2017. "Uncovering context-induced status quo effects in choice experiments," Journal of Environmental Economics and Management, Elsevier, vol. 81(C), pages 59-73.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:eejocm:v:56:y:2025:i:c:s1755534525000259. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/journal-of-choice-modelling .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.