IDEAS home Printed from https://ideas.repec.org/a/eee/jeborg/v148y2018icp344-375.html
   My bibliography  Save this article

Improving out-of-sample predictions using response times and a model of the decision process

Author

Listed:
  • Clithero, John A.

Abstract

A basic problem in empirical economics involves using data from one domain to make out-of-sample predictions for a different, but related environment. When the choice data are binary, a canonical method for making these types of predictions is the logistic choice model. This paper investigates whether it is possible to improve out-of-sample predictions by changing two aspects of the canonical approach: 1) Using response times in addition to the choice data, and 2) Combining them using a model from the psychology and neuroscience literature, the Drift-Diffusion Model (DDM). Two experiments compare the out-of-sample choice prediction accuracies of both methods and in both cases the DDM method outperforms a logistic prediction method. Furthermore, the DDM allows for out-of-sample process predictions. Both experiments validate the DDM as a method for predicting out-of-sample response times.

Suggested Citation

  • Clithero, John A., 2018. "Improving out-of-sample predictions using response times and a model of the decision process," Journal of Economic Behavior & Organization, Elsevier, vol. 148(C), pages 344-375.
  • Handle: RePEc:eee:jeborg:v:148:y:2018:i:c:p:344-375
    DOI: 10.1016/j.jebo.2018.02.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167268118300398
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jebo.2018.02.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Patil, Anand & Huard, David & Fonnesbeck, Christopher J., 2010. "PyMC: Bayesian Stochastic Modelling in Python," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 35(i04).
    2. Jörg Rieskamp & Jerome R. Busemeyer & Barbara A. Mellers, 2006. "Extending the Bounds of Rationality: Evidence and Theories of Preferential Choice," Journal of Economic Literature, American Economic Association, vol. 44(3), pages 631-661, September.
    3. Marina Agranov & Andrew Caplin & Chloe Tergiman, 2015. "Naive play and the process of choice in guessing games," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(2), pages 146-157, December.
    4. Filip Matêjka & Alisdair McKay, 2015. "Rational Inattention to Discrete Choices: A New Foundation for the Multinomial Logit Model," American Economic Review, American Economic Association, vol. 105(1), pages 272-298, January.
    5. Peter Moffatt, 2005. "Stochastic Choice and the Allocation of Cognitive Effort," Experimental Economics, Springer;Economic Science Association, vol. 8(4), pages 369-388, December.
    6. Justin Wolfers & Eric Zitzewitz, 2004. "Prediction Markets," Journal of Economic Perspectives, American Economic Association, vol. 18(2), pages 107-126, Spring.
    7. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2013. "Salience and Consumer Choice," Journal of Political Economy, University of Chicago Press, vol. 121(5), pages 803-843.
    8. Goeree, Jacob K. & Holt, Charles A., 2004. "A model of noisy introspection," Games and Economic Behavior, Elsevier, vol. 46(2), pages 365-382, February.
    9. Aldo Rustichini, 2009. "Is There a Method of Neuroeconomics?," American Economic Journal: Microeconomics, American Economic Association, vol. 1(2), pages 48-59, August.
    10. Elena Reutskaja & Rosemarie Nagel & Colin F. Camerer & Antonio Rangel, 2011. "Search Dynamics in Consumer Choice under Time Pressure: An Eye-Tracking Study," American Economic Review, American Economic Association, vol. 101(2), pages 900-926, April.
    11. Isabelle Brocas & Juan D. Carrillo & Stephanie W. Wang & Colin F. Camerer, 2014. "Imperfect Choice or Imperfect Attention? Understanding Strategic Thinking in Private Information Games," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(3), pages 944-970.
    12. Ian Krajbich & Todd Hare & Björn Bartling & Yosuke Morishima & Ernst Fehr, 2015. "A Common Mechanism Underlying Food Choice and Social Decisions," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-24, October.
    13. Basten U & Biele G. P. & Heekeren H. R. & Fiebach, 2010. "How the brain integrates costs and benefits during decision making," SFB 649 Discussion Papers SFB649DP2010-063, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    14. Ariel Rubinstein, 2007. "Instinctive and Cognitive Reasoning: A Study of Response Times," Economic Journal, Royal Economic Society, vol. 117(523), pages 1243-1259, October.
    15. Faruk Gul & Paulo Natenzon & Wolfgang Pesendorfer, 2014. "Random Choice as Behavioral Optimization," Econometrica, Econometric Society, vol. 82, pages 1873-1912, September.
    16. Clément Levallois & John A. Clithero & Paul Wouters & Ale Smidts & Scott A. Huettel, 2012. "Translating upwards : linking the neural and social sciences via neuroeconomics," Post-Print hal-02313160, HAL.
    17. Ernst Fehr & Antonio Rangel, 2011. "Neuroeconomic Foundations of Economic Choice--Recent Advances," Journal of Economic Perspectives, American Economic Association, vol. 25(4), pages 3-30, Fall.
    18. Christian A Rodriguez & Brandon M Turner & Samuel M McClure, 2014. "Intertemporal Choice as Discounted Value Accumulation," PLOS ONE, Public Library of Science, vol. 9(2), pages 1-9, February.
    19. Alec Smith & B. Douglas Bernheim & Colin F. Camerer & Antonio Rangel, 2014. "Neural Activity Reveals Preferences without Choices," American Economic Journal: Microeconomics, American Economic Association, vol. 6(2), pages 1-36, May.
    20. repec:cup:judgdm:v:5:y:2010:i:6:p:437-449 is not listed on IDEAS
    21. Joseph Tao-yi Wang & Michael Spezio & Colin F. Camerer, 2010. "Pinocchio's Pupil: Using Eyetracking and Pupil Dilation to Understand Truth Telling and Deception in Sender-Receiver Games," American Economic Review, American Economic Association, vol. 100(3), pages 984-1007, June.
    22. H. R. Heekeren & S. Marrett & P. A. Bandettini & L. G. Ungerleider, 2004. "A general mechanism for perceptual decision-making in the human brain," Nature, Nature, vol. 431(7010), pages 859-862, October.
    23. Hyunyoung Choi & Hal Varian, 2012. "Predicting the Present with Google Trends," The Economic Record, The Economic Society of Australia, vol. 88(s1), pages 2-9, June.
    24. B. Douglas Bernheim, 2009. "On the Potential of Neuroeconomics: A Critical (but Hopeful) Appraisal," American Economic Journal: Microeconomics, American Economic Association, vol. 1(2), pages 1-41, August.
    25. Brocas, Isabelle, 2012. "Information processing and decision-making: Evidence from the brain sciences and implications for economics," Journal of Economic Behavior & Organization, Elsevier, vol. 83(3), pages 292-310.
    26. David J. Spiegelhalter & Nicola G. Best & Bradley P. Carlin & Angelika Van Der Linde, 2002. "Bayesian measures of model complexity and fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 583-639, October.
    27. Thomas Otter & Joe Johnson & Jörg Rieskamp & Greg Allenby & Jeff Brazell & Adele Diederich & J. Hutchinson & Steven MacEachern & Shiling Ruan & Jim Townsend, 2008. "Sequential sampling models of choice: Some recent advances," Marketing Letters, Springer, vol. 19(3), pages 255-267, December.
    28. Ariel Rubinstein, 2016. "A Typology of Players: Between Instinctive and Contemplative," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(2), pages 859-890.
    29. Mervyn Stone, 1960. "Models for choice-reaction time," Psychometrika, Springer;The Psychometric Society, vol. 25(3), pages 251-260, September.
    30. Ian Krajbich & Bastiaan Oud & Ernst Fehr, 2014. "Benefits of Neuroeconomic Modeling: New Policy Interventions and Predictors of Preference," American Economic Review, American Economic Association, vol. 104(5), pages 501-506, May.
    31. Wilcox, Nathaniel T, 1993. "Lottery Choice: Incentives, Complexity and Decision Time," Economic Journal, Royal Economic Society, vol. 103(421), pages 1397-1417, November.
    32. Komlos, John & Kelly, Inas (ed.), 2016. "The Oxford Handbook of Economics and Human Biology," OUP Catalogue, Oxford University Press, number 9780199389292.
    33. Colin F. Camerer, 2013. "Goals, Methods, and Progress in Neuroeconomics," Annual Review of Economics, Annual Reviews, vol. 5(1), pages 425-455, May.
    34. Ariel Rubinstein, 2007. "Instinctive and Cognitive Reasoning: Response Times Study," Levine's Bibliography 321307000000001011, UCLA Department of Economics.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carlos Alós-Ferrer & Ernst Fehr & Michele Garagnani, 2022. "Identifying nontransitive preferences," ECON - Working Papers 415, Department of Economics - University of Zurich, revised Jan 2023.
    2. Aleksandr Alekseev, 2019. "Using response times to measure ability on a cognitive task," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 65-75, August.
    3. Ryan Webb, 2019. "The (Neural) Dynamics of Stochastic Choice," Management Science, INFORMS, vol. 65(1), pages 230-255, January.
    4. Clithero, John A., 2018. "Response times in economics: Looking through the lens of sequential sampling models," Journal of Economic Psychology, Elsevier, vol. 69(C), pages 61-86.
    5. Shuo Liu & Nick Netzer, 2023. "Happy Times: Measuring Happiness Using Response Times," American Economic Review, American Economic Association, vol. 113(12), pages 3289-3322, December.
    6. Dewan, Ambuj & Neligh, Nathaniel, 2020. "Estimating information cost functions in models of rational inattention," Journal of Economic Theory, Elsevier, vol. 187(C).
    7. Benjamin Hébert & Michael Woodford, 2021. "Neighborhood-Based Information Costs," American Economic Review, American Economic Association, vol. 111(10), pages 3225-3255, October.
    8. Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci & Aldo Rustichini, 2023. "Multinomial Logit Processes and Preference Discovery: Inside and Outside the Black Box," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(3), pages 1155-1194.
    9. Andrew Schotter & Isabel Trevino, 2021. "Is response time predictive of choice? An experimental study of threshold strategies," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 87-117, March.
    10. Carlos Alós-Ferrer & Ernst Fehr & Nick Netzer, 2021. "Time Will Tell: Recovering Preferences When Choices Are Noisy," Journal of Political Economy, University of Chicago Press, vol. 129(6), pages 1828-1877.
    11. S. Cerreia-Vioglio & F. Maccheroni & M. Marinacci & A. Rustichini, 2017. "Multinomial logit processes and preference discovery: inside and outside the black box," Working Papers 615, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    12. Hebert, Benjamin & Woodford, Michael, 2018. "Information Costs and Sequential Information Sampling," Research Papers 3751, Stanford University, Graduate School of Business.
    13. Huseynov, Samir & Palma, Marco A. & Ahmad, Ghufran, 2021. "Does the magnitude of relative calorie distance affect food consumption?," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 530-551.
    14. Carlos Alós-Ferrer & Michele Garagnani, 2022. "Strength of preference and decisions under risk," Journal of Risk and Uncertainty, Springer, vol. 64(3), pages 309-329, June.
    15. Carlo Baldassi & Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci & Marco Pirazzini, 2020. "A Behavioral Characterization of the Drift Diffusion Model and Its Multialternative Extension for Choice Under Time Pressure," Management Science, INFORMS, vol. 66(11), pages 5075-5093, November.
    16. Sangil Lee & Chris M. Glaze & Eric T. Bradlow & Joseph W. Kable, 2020. "Flexible Utility Function Approximation via Cubic Bezier Splines," Psychometrika, Springer;The Psychometric Society, vol. 85(3), pages 716-737, September.
    17. Hébert, Benjamin & Woodford, Michael, 2023. "Rational inattention when decisions take time," Journal of Economic Theory, Elsevier, vol. 208(C).
    18. Guangzhong Hu & Yuming Liu & Kai Liu & Xiaoxu Yang, 2023. "Research on Data-Driven Dynamic Decision-Making Mechanism of Mega Infrastructure Project Construction," Sustainability, MDPI, vol. 15(12), pages 1-25, June.
    19. Valdes Salvador & Gonzalo ValdesEdwards, 2023. "Microfoundations of Expected Utility and Response Times," Papers 2302.09421, arXiv.org.
    20. Konrad Grabiszewski & Alex Horenstein, 2022. "Profiling dynamic decision-makers," PLOS ONE, Public Library of Science, vol. 17(4), pages 1-22, April.
    21. John A. Clithero & Jae Joon Lee & Joshua Tasoff, 2019. "Supervised Machine Learning for Eliciting Individual Demand," Papers 1904.13329, arXiv.org, revised Feb 2021.
    22. David J. Cooper & Ian Krajbich & Charles N. Noussair, 2019. "Choice-Process Data in Experimental Economics," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 1-13, August.
    23. Vriens, M. & Vidden, C. & Schomaker, J., 2020. "What I see is what I want: Top-down attention biasing choice behavior," Journal of Business Research, Elsevier, vol. 111(C), pages 262-269.
    24. Cary Frydman & Ian Krajbich, 2022. "Using Response Times to Infer Others’ Private Information: An Application to Information Cascades," Management Science, INFORMS, vol. 68(4), pages 2970-2986, April.
    25. Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci, 2020. "Multinomial logit processes and preference discovery: outside and inside the black box," Working Papers 663, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clithero, John A., 2018. "Response times in economics: Looking through the lens of sequential sampling models," Journal of Economic Psychology, Elsevier, vol. 69(C), pages 61-86.
    2. Guidon Fenig & Giovanni Gallipoli & Yoram Halevy, 2018. "Piercing the 'Payoff Function' Veil: Tracing Beliefs and Motives," Working Papers tecipa-619, University of Toronto, Department of Economics.
    3. Recalde, María P. & Riedl, Arno & Vesterlund, Lise, 2018. "Error-prone inference from response time: The case of intuitive generosity in public-good games," Journal of Public Economics, Elsevier, vol. 160(C), pages 132-147.
    4. Fadong Chen & Urs Fischbacher, 2016. "Response time and click position: cheap indicators of preferences," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 2(2), pages 109-126, November.
    5. Arkady Konovalov & Ian Krajbich, 2016. "Revealed Indifference: Using Response Times to Infer Preferences," Working Papers 16-01, Ohio State University, Department of Economics.
    6. Cary Frydman & Ian Krajbich, 2022. "Using Response Times to Infer Others’ Private Information: An Application to Information Cascades," Management Science, INFORMS, vol. 68(4), pages 2970-2986, April.
    7. Daniel Navarro-Martinez & Graham Loomes & Andrea Isoni & David Butler & Larbi Alaoui, 2018. "Boundedly rational expected utility theory," Journal of Risk and Uncertainty, Springer, vol. 57(3), pages 199-223, December.
    8. Carlos Alós-Ferrer & Ernst Fehr & Nick Netzer, 2021. "Time Will Tell: Recovering Preferences When Choices Are Noisy," Journal of Political Economy, University of Chicago Press, vol. 129(6), pages 1828-1877.
    9. Carlos Alós-Ferrer & Johannes Buckenmaier, 2021. "Cognitive sophistication and deliberation times," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 558-592, June.
    10. Jan Hausfeld & Sven Resnjanskij, 2017. "Risky Decisions and the Opportunity Costs of Time," TWI Research Paper Series 108, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    11. Grabiszewski, Konrad & Horenstein, Alex, 2020. "Effort is not a monotonic function of skills: Results from a global mobile experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 176(C), pages 634-652.
    12. repec:cup:judgdm:v:14:y:2019:i:4:p:381-394 is not listed on IDEAS
    13. Arkady Konovalov & Ian Krajbich, 2019. "Revealed strength of preference: Inference from response times," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(4), pages 381-394, July.
    14. Ryan Webb, 2019. "The (Neural) Dynamics of Stochastic Choice," Management Science, INFORMS, vol. 65(1), pages 230-255, January.
    15. Martinovici, A., 2019. "Revealing attention - how eye movements predict brand choice and moment of choice," Other publications TiSEM 7dca38a5-9f78-4aee-bd81-c, Tilburg University, School of Economics and Management.
    16. Brocas, Isabelle & Carrillo, Juan D., 2021. "Value computation and modulation: A neuroeconomic theory of self-control as constrained optimization," Journal of Economic Theory, Elsevier, vol. 198(C).
    17. Nagel, Rosemarie & Bühren, Christoph & Frank, Björn, 2017. "Inspired and inspiring: Hervé Moulin and the discovery of the beauty contest game," Mathematical Social Sciences, Elsevier, vol. 90(C), pages 191-207.
    18. Ryan Webb & Paul W. Glimcher & Kenway Louie, 2021. "The Normalization of Consumer Valuations: Context-Dependent Preferences from Neurobiological Constraints," Management Science, INFORMS, vol. 67(1), pages 93-125, January.
    19. S. Cerreia-Vioglio & F. Maccheroni & M. Marinacci & A. Rustichini, 2017. "Multinomial logit processes and preference discovery: inside and outside the black box," Working Papers 615, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    20. Grabiszewski, Konrad & Horenstein, Alex, 2022. "Measuring tree complexity with response times," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
    21. Andrew Schotter & Isabel Trevino, 2021. "Is response time predictive of choice? An experimental study of threshold strategies," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 87-117, March.

    More about this item

    Keywords

    Drift diffusion; Neuroeconomics; Prediction; Response times;
    All these keywords.

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • D03 - Microeconomics - - General - - - Behavioral Microeconomics: Underlying Principles
    • D87 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Neuroeconomics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jeborg:v:148:y:2018:i:c:p:344-375. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jebo .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.