IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v65y2019i1p256-271.html
   My bibliography  Save this article

Random Projection Estimation of Discrete-Choice Models with Large Choice Sets

Author

Listed:
  • Khai Xiang Chiong

    (Naveen Jindal School of Management, University of Texas at Dallas, Richardson, Texas 75080)

  • Matthew Shum

    (California Institute of Technology, Pasadena, California 91125)

Abstract

We introduce random projection , an important dimension-reduction tool from machine learning, for the estimation of aggregate discrete-choice models with high-dimensional choice sets. Initially, high-dimensional data are projected into a lower-dimensional Euclidean space using random projections. Subsequently, estimation proceeds using cyclical monotonicity moment inequalities implied by the multinomial choice model; the estimation procedure is semiparametric and does not require explicit distributional assumptions to be made regarding the random utility errors. Our procedure is justified via the Johnson–Lindenstrauss lemma—the pairwise distances between data points are preserved through random projections. The estimator works well in simulations and in an application to a supermarket scanner data set.

Suggested Citation

  • Khai Xiang Chiong & Matthew Shum, 2019. "Random Projection Estimation of Discrete-Choice Models with Large Choice Sets," Management Science, INFORMS, vol. 65(1), pages 256-271, January.
  • Handle: RePEc:inm:ormnsc:v:65:y:2019:i:1:p:256-271
    DOI: 10.1287/mnsc.2017.2928
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/mnsc.2017.2928
    Download Restriction: no

    File URL: https://libkey.io/10.1287/mnsc.2017.2928?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Emerson Melo & Kirill Pogorelskiy & Matthew Shum, 2019. "Testing The Quantal Response Hypothesis," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 60(1), pages 53-74, February.
    2. Schneider, Matthew J. & Gupta, Sachin, 2016. "Forecasting sales of new and existing products using consumer reviews: A random projections approach," International Journal of Forecasting, Elsevier, vol. 32(2), pages 243-256.
    3. Daniel L. McFadden & Mogens Fosgerau, 2012. "A theory of the perturbed consumer with general budgets," NBER Working Papers 17953, National Bureau of Economic Research, Inc.
    4. Federico Ciliberto & Elie Tamer, 2009. "Market Structure and Multiple Equilibria in Airline Markets," Econometrica, Econometric Society, vol. 77(6), pages 1791-1828, November.
    5. Jeremy T. Fox, 2007. "Semiparametric estimation of multinomial discrete-choice models using a subset of choices," RAND Journal of Economics, RAND Corporation, vol. 38(4), pages 1002-1019, December.
    6. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    7. Steven T. Berry & Philip A. Haile, 2014. "Identification in Differentiated Products Markets Using Market Level Data," Econometrica, Econometric Society, vol. 82(5), pages 1749-1797, September.
    8. Amit Gandhi Gandhi & Zhentong Lu & Xiaoxia Shi, 2013. "Estimating demand for differentiated products with error in market shares," CeMMAP working papers 03/13, Institute for Fiscal Studies.
    9. V. Joseph Hotz & Robert A. Miller, 1993. "Conditional Choice Probabilities and the Estimation of Dynamic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 60(3), pages 497-529.
    10. Philip A. Haile & Ali Hortaçsu & Grigory Kosenok, 2008. "On the Empirical Content of Quantal Response Equilibrium," American Economic Review, American Economic Association, vol. 98(1), pages 180-200, March.
    11. Lee, Lung-fei, 1995. "Semiparametric maximum likelihood estimation of polychotomous and sequential choice models," Journal of Econometrics, Elsevier, vol. 65(2), pages 381-428, February.
    12. Hausman, Jerry & McFadden, Daniel, 1984. "Specification Tests for the Multinomial Logit Model," Econometrica, Econometric Society, vol. 52(5), pages 1219-1240, September.
    13. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    14. Jacob Goeree & Charles Holt & Thomas Palfrey, 2005. "Regular Quantal Response Equilibrium," Experimental Economics, Springer;Economic Science Association, vol. 8(4), pages 347-367, December.
    15. Victor Chernozhukov & Han Hong & Elie Tamer, 2007. "Estimation and Confidence Regions for Parameter Sets in Econometric Models," Econometrica, Econometric Society, vol. 75(5), pages 1243-1284, September.
    16. Kenneth E. Train & Daniel L. McFadden & Moshe Ben-Akiva, 1987. "The Demand for Local Telephone Service: A Fully Discrete Model of Residential Calling Patterns and Service Choices," RAND Journal of Economics, The RAND Corporation, vol. 18(1), pages 109-123, Spring.
    17. Mogens Fosgerau & André de Palma, 2016. "Generalized entropy models," Working Papers hal-01291347, HAL.
    18. Manski, Charles F., 1985. "Semiparametric analysis of discrete response : Asymptotic properties of the maximum score estimator," Journal of Econometrics, Elsevier, vol. 27(3), pages 313-333, March.
    19. Matthew Gentzkow & Jesse Shapiro & Matt Taddy, 2016. "Measuring Polarization in High-Dimensional Data: Method and Application to Congressional Speech," Working Papers id:11114, eSocialSciences.
    20. Khai Xiang Chiong & Alfred Galichon & Matt Shum, 2016. "Duality in dynamic discrete‐choice models," Quantitative Economics, Econometric Society, vol. 7(1), pages 83-115, March.
    21. Xiao Liu & Param Vir Singh & Kannan Srinivasan, 2016. "A Structured Analysis of Unstructured Big Data by Leveraging Cloud Computing," Marketing Science, INFORMS, vol. 35(3), pages 363-388, May.
    22. Hausman, J. A. & Abrevaya, Jason & Scott-Morton, F. M., 1998. "Misclassification of the dependent variable in a discrete-response setting," Journal of Econometrics, Elsevier, vol. 87(2), pages 239-269, September.
    23. Jeremy T. Fox & Patrick Bajari, 2013. "Measuring the Efficiency of an FCC Spectrum Auction," American Economic Journal: Microeconomics, American Economic Association, vol. 5(1), pages 100-146, February.
    24. Ori Heffetz & Katrina Ligett, 2014. "Privacy and Data-Based Research," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 75-98, Spring.
    25. Bruno J.D. Jacobs & Bas Donkers & Dennis Fok, 2016. "Model-Based Purchase Predictions for Large Assortments," Marketing Science, INFORMS, vol. 35(3), pages 389-404, May.
    26. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    27. Olivier Coibion & Yuriy Gorodnichenko & Gee Hee Hong, 2015. "The Cyclicality of Sales, Regular and Effective Prices: Business Cycle and Policy Implications," American Economic Review, American Economic Association, vol. 105(3), pages 993-1029, March.
    28. Han, Aaron K., 1987. "Non-parametric analysis of a generalized regression model : The maximum rank correlation estimator," Journal of Econometrics, Elsevier, vol. 35(2-3), pages 303-316, July.
    29. Judith A. Chevalier & Anil K. Kashyap & Peter E. Rossi, 2003. "Why Don't Prices Rise During Periods of Peak Demand? Evidence from Scanner Data," American Economic Review, American Economic Association, vol. 93(1), pages 15-37, March.
    30. Dongling Huang & Lan Luo, 2016. "Consumer Preference Elicitation of Complex Products Using Fuzzy Support Vector Machine Active Learning," Marketing Science, INFORMS, vol. 35(3), pages 445-464, May.
    31. Manski, Charles F., 1975. "Maximum score estimation of the stochastic utility model of choice," Journal of Econometrics, Elsevier, vol. 3(3), pages 205-228, August.
    32. Pradeep Chintagunta & Dominique M. Hanssens & John R. Hauser, 2016. "Editorial—Marketing Science and Big Data," Marketing Science, INFORMS, vol. 35(3), pages 341-342, May.
    33. Pollard, David, 1991. "Asymptotics for Least Absolute Deviation Regression Estimators," Econometric Theory, Cambridge University Press, vol. 7(2), pages 186-199, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sørensen, Jesper R.-V. & Fosgerau, Mogens, 2022. "How McFadden met Rockafellar and learned to do more with less," Journal of Mathematical Economics, Elsevier, vol. 100(C).
    2. Haoying Wang & Guohui Wu, 2022. "Modeling discrete choices with large fine-scale spatial data: opportunities and challenges," Journal of Geographical Systems, Springer, vol. 24(3), pages 325-351, July.
    3. Mogens Fosgerau & Emerson Melo & André de Palma & Matthew Shum, 2020. "Discrete Choice And Rational Inattention: A General Equivalence Result," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 61(4), pages 1569-1589, November.
    4. Smeele, Nicholas V.R. & Chorus, Caspar G. & Schermer, Maartje H.N. & de Bekker-Grob, Esther W., 2023. "Towards machine learning for moral choice analysis in health economics: A literature review and research agenda," Social Science & Medicine, Elsevier, vol. 326(C).
    5. Fosgerau, Mogens & Melo, Emerson & Shum, Matthew & Sørensen, Jesper R.-V., 2021. "Some remarks on CCP-based estimators of dynamic models," Economics Letters, Elsevier, vol. 204(C).
    6. Ma, Liye & Sun, Baohong, 2020. "Machine learning and AI in marketing – Connecting computing power to human insights," International Journal of Research in Marketing, Elsevier, vol. 37(3), pages 481-504.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jeremy T. Fox, 2018. "Estimating matching games with transfers," Quantitative Economics, Econometric Society, vol. 9(1), pages 1-38, March.
    2. Wayne Yuan Gao & Ming Li, 2020. "Robust Semiparametric Estimation in Panel Multinomial Choice Models," Papers 2009.00085, arXiv.org.
    3. Patrick Bajari & Jeremy Fox & Stephen Ryan, 2008. "Evaluating wireless carrier consolidation using semiparametric demand estimation," Quantitative Marketing and Economics (QME), Springer, vol. 6(4), pages 299-338, December.
    4. Crawford, Gregory S. & Griffith, Rachel & Iaria, Alessandro, 2021. "A survey of preference estimation with unobserved choice set heterogeneity," Journal of Econometrics, Elsevier, vol. 222(1), pages 4-43.
    5. Yan, Jin & Yoo, Hong Il, 2019. "Semiparametric estimation of the random utility model with rank-ordered choice data," Journal of Econometrics, Elsevier, vol. 211(2), pages 414-438.
    6. Chen, Le-Yu & Lee, Sokbae, 2019. "Breaking the curse of dimensionality in conditional moment inequalities for discrete choice models," Journal of Econometrics, Elsevier, vol. 210(2), pages 482-497.
    7. Sørensen, Jesper R.-V. & Fosgerau, Mogens, 2022. "How McFadden met Rockafellar and learned to do more with less," Journal of Mathematical Economics, Elsevier, vol. 100(C).
    8. Arthur Lewbel, 2019. "The Identification Zoo: Meanings of Identification in Econometrics," Journal of Economic Literature, American Economic Association, vol. 57(4), pages 835-903, December.
    9. Pietro Tebaldi & Alexander Torgovitsky & Hanbin Yang, 2023. "Nonparametric Estimates of Demand in the California Health Insurance Exchange," Econometrica, Econometric Society, vol. 91(1), pages 107-146, January.
    10. Jiarui Liu, 2021. "Sequential Search Models: A Pairwise Maximum Rank Approach," Papers 2104.13865, arXiv.org, revised Nov 2021.
    11. Fu Ouyang & Thomas Tao Yang, 2020. "Semiparametric Discrete Choice Models for Bundles," Discussion Papers Series 625, School of Economics, University of Queensland, Australia.
    12. Steven T Berry & Giovanni Compiani, 2023. "An Instrumental Variable Approach to Dynamic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(4), pages 1724-1758.
    13. Liang Chen & Eugene Choo & Alfred Galichon & Simon Weber, 2023. "Existence of a Competitive Equilibrium with Substitutes, with Applications to Matching and Discrete Choice Models," Papers 2309.11416, arXiv.org.
    14. Taisuke Otsu & Myung Hwan Seo, 2014. "Asymptotics for maximum score method under general conditions," STICERD - Econometrics Paper Series 571, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    15. repec:cep:stiecm:em/2012/559 is not listed on IDEAS
    16. Odran Bonnet & Alfred Galichon & Yu-Wei Hsieh & Keith O’Hara & Matt Shum, 2022. "Yogurts Choose Consumers? Estimation of Random-Utility Models via Two-Sided Matching [Unobserved Product Differentiation in Discrete-Choice Models: Estimating Price Elasticities and Welfare Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(6), pages 3085-3114.
    17. Francesca Molinari, 2020. "Microeconometrics with Partial Identi?cation," CeMMAP working papers CWP15/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    18. Jason R. Blevins, 2013. "Non-Standard Rates of Convergence of Criterion-Function-Based Set Estimators," Working Papers 13-02, Ohio State University, Department of Economics.
    19. Tatiana Komarova, 2012. "Binary Choice Models with Discrete Regressors: Identification and Misspecification," STICERD - Econometrics Paper Series 559, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    20. Steven T. Berry & Philip A. Haile, 2009. "Nonparametric Identification of Multinomial Choice Demand Models with Heterogeneous Consumers," NBER Working Papers 15276, National Bureau of Economic Research, Inc.
    21. Aradillas-Lopez, Andres, 2012. "Pairwise-difference estimation of incomplete information games," Journal of Econometrics, Elsevier, vol. 168(1), pages 120-140.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:65:y:2019:i:1:p:256-271. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.