IDEAS home Printed from https://ideas.repec.org/a/eee/eejocm/v38y2021ics1755534520300543.html
   My bibliography  Save this article

The overreliance on statistical goodness-of-fit and under-reliance on model validation in discrete choice models: A review of validation practices in the transportation academic literature

Author

Listed:
  • Parady, Giancarlos
  • Ory, David
  • Walker, Joan

Abstract

An examination of model validation practices in the peer-reviewed transportation literature published between 2014 and 2018 reveals that 92% of studies reported goodness-of-fit statistics, and 64.6% reported some sort of policy-relevant inference analysis. However, only 18.1% reported validation performance measures, out of which 78% (14.2% of all studies) consisted of internal validation and 22% (4% of all studies) consisted of external validation. The proposition put forward in this paper is that the reliance on goodness-of-fit measures rather than validation performance is unwise, especially given the dependence of the transportation research field on observational (non-experimental) studies. Model validation should be a non-negotiable part of presenting a model for peer-review in academic journals. For that purpose, we propose a simple heuristic to select validation methods given the resources available to the researcher.

Suggested Citation

  • Parady, Giancarlos & Ory, David & Walker, Joan, 2021. "The overreliance on statistical goodness-of-fit and under-reliance on model validation in discrete choice models: A review of validation practices in the transportation academic literature," Journal of choice modelling, Elsevier, vol. 38(C).
    • Handle: RePEc:eee:eejocm:v:38:y:2021:i:c:s1755534520300543
    DOI: 10.1016/j.jocm.2020.100257
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1755534520300543
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jocm.2020.100257?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Haghani, Milad & Bliemer, Michiel C.J. & Rose, John M. & Oppewal, Harmen & Lancsar, Emily, 2021. "Hypothetical bias in stated choice experiments: Part II. Conceptualisation of external validity, sources and explanations of bias and effectiveness of mitigation methods," Journal of choice modelling, Elsevier, vol. 41(C).
    2. Chung, Yi-Shih & Ku, Ya-Han, 2023. "Effect of time stress and store visibility on the dynamics of passenger activity choices at airport terminals based on indoor trajectory data," Journal of Retailing and Consumer Services, Elsevier, vol. 73(C).
    3. Kim, Sung Hoo & Mokhtarian, Patricia L., 2023. "Finite mixture (or latent class) modeling in transportation: Trends, usage, potential, and future directions," Transportation Research Part B: Methodological, Elsevier, vol. 172(C), pages 134-173.
    4. Krueger, Rico & Bierlaire, Michel & Daziano, Ricardo A. & Rashidi, Taha H. & Bansal, Prateek, 2021. "Evaluating the predictive abilities of mixed logit models with unobserved inter- and intra-individual heterogeneity," Journal of choice modelling, Elsevier, vol. 41(C).
    5. Jia, Wenjian & Chen, T. Donna, 2023. "Investigating heterogeneous preferences for plug-in electric vehicles: Policy implications from different choice models," Transportation Research Part A: Policy and Practice, Elsevier, vol. 173(C).
    6. Han, Chenglin & Luo, Lichen & Parady, Giancarlos & Takami, Kiyoshi & Chikaraishi, Makoto & Harata, Noboru, 2023. "Modeling joint eating-out destination choices incorporating group-level impedance: A case study of the Greater Tokyo Area," Journal of Transport Geography, Elsevier, vol. 111(C).
    7. Parady, Giancarlos & Frei, Andreas & Kowald, Matthias & Guidon, Sergio & Wicki, Michael & van den Berg, Pauline & Carrasco, Juan-Antonio & Arentze, Theo & Timmermans, Harry & Wellman, Barry & Takami, , 2021. "A comparative study of social interaction frequencies among social network members in five countries," Journal of Transport Geography, Elsevier, vol. 90(C).
    8. Hasselwander, Marc & Bigotte, Joao F. & Antunes, Antonio P. & Sigua, Ricardo G., 2022. "Towards sustainable transport in developing countries: Preliminary findings on the demand for mobility-as-a-service (MaaS) in Metro Manila," Transportation Research Part A: Policy and Practice, Elsevier, vol. 155(C), pages 501-518.
    9. Gutierrez-Lythgoe, Antonio, 2023. "Autoempleo y Machine Learning: Una aplicación para España [Self-employment and Machine Learning: An application for Spain]," MPRA Paper 117275, University Library of Munich, Germany.
    10. Xinyi Wang & F. Atiyya Shaw & Patricia L. Mokhtarian & Kari E. Watkins, 2023. "Response willingness in consecutive travel surveys: an investigation based on the National Household Travel Survey using a sample selection model," Transportation, Springer, vol. 50(6), pages 2339-2373, December.
    11. Beeramoole, Prithvi Bhat & Arteaga, Cristian & Pinz, Alban & Haque, Md Mazharul & Paz, Alexander, 2023. "Extensive hypothesis testing for estimation of mixed-Logit models," Journal of choice modelling, Elsevier, vol. 47(C).
    12. Perez-Lopez, Jose-Benito & Novales, Margarita & Orro, Alfonso, 2022. "Spatially correlated nested logit model for spatial location choice," Transportation Research Part B: Methodological, Elsevier, vol. 161(C), pages 1-12.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:eejocm:v:38:y:2021:i:c:s1755534520300543. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/journal-of-choice-modelling .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.