IDEAS home Printed from https://ideas.repec.org/p/hig/wpaper/200-ec-2018.html
   My bibliography  Save this paper

The Ensemble Method For Censored Demand Prediction

Author

Listed:
  • Evgeniy M. Ozhegov

    (National Research University Higher School of Economics)

  • Daria Teterina

    (National Research University Higher School of Economics)

Abstract

Many economic applications, including optimal pricing and inventory management, require predictions of demand based on sales data and the estimation of the reaction of sales to price change. There is a wide range of econometric approaches used to correct biases in the estimates of demand parameters on censored sales data. These approaches can also be applied to various classes of machine learning (ML) models to reduce the prediction error of sales volumes. In this study we construct two ensemble models for demand prediction with and without accounting for demand censorship. Accounting for sales censorship is based on a censored quantile regression where the model estimation was split into two separate parts: a) a prediction of zero sales by the classification model; and b) a prediction of non-zero sales by the regression model. Models with and without censorship are based on the prediction aggregations of least squares, Ridge and Lasso regressions and the Random Forest model. Having estimated the predictive properties of both models, we empirically test the best predictive power of the model taking into account the censored nature of demand. We also show that ML with censorship provides bias corrected estimates of demand sensitivity to price change similar to econometric models

Suggested Citation

  • Evgeniy M. Ozhegov & Daria Teterina, 2018. "The Ensemble Method For Censored Demand Prediction," HSE Working papers WP BRP 200/EC/2018, National Research University Higher School of Economics.
  • Handle: RePEc:hig:wpaper:200/ec/2018
    as

    Download full text from publisher

    File URL: https://wp.hse.ru/data/2018/10/25/1142226371/200EC2018.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lind, Patrik & Mellander, Erik, 2016. "Relations between immigration and adult skills: findings based on PIAAC," Working Paper Series 2016:21, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    2. Chernozhukov, Victor & Fernández-Val, Iván & Kowalski, Amanda E., 2015. "Quantile regression with censoring and endogeneity," Journal of Econometrics, Elsevier, vol. 186(1), pages 201-221.
    3. Jean-Michel Paguet, 1994. "Les conglomérats financiers en Europe, stratégie et concurrence," Revue d'Économie Financière, Programme National Persée, vol. 28(1), pages 7-26.
    4. Pentland, Brian T., 1993. "Getting comfortable with the numbers: Auditing and the micro-production of macro-order," Accounting, Organizations and Society, Elsevier, vol. 18(7-8), pages 605-620.
    5. Hoderlein, Stefan & White, Halbert, 2012. "Nonparametric identification in nonseparable panel data models with generalized fixed effects," Journal of Econometrics, Elsevier, vol. 168(2), pages 300-314.
    6. Michael P. Keane, 2013. "Panel data discrete choice models of consumer demand," Economics Papers 2013-W08, Economics Group, Nuffield College, University of Oxford.
    7. Patrick Bajari & Denis Nekipelov & Stephen P. Ryan & Miaoyu Yang, 2015. "Demand Estimation with Machine Learning and Model Combination," NBER Working Papers 20955, National Bureau of Economic Research, Inc.
    8. Matzkin, Rosa L., 2012. "Identification in nonparametric limited dependent variable models with simultaneity and unobserved heterogeneity," Journal of Econometrics, Elsevier, vol. 166(1), pages 106-115.
    9. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," Review of Economic Studies, Oxford University Press, vol. 81(2), pages 608-650.
    10. Maria Iannario, 2015. "Detecting latent components in ordinal data with overdispersion by means of a mixture distribution," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 977-987, May.
    11. Patrick Bajari & Denis Nekipelov & Stephen P. Ryan & Miaoyu Yang, 2015. "Machine Learning Methods for Demand Estimation," American Economic Review, American Economic Association, vol. 105(5), pages 481-485, May.
    12. Bonnet, Céline & Richards, Timothy J., 2016. "Models of Consumer Demand for Differentiated Products," TSE Working Papers 16-741, Toulouse School of Economics (TSE).
    13. Liran Einav & Mark Jenkins & Jonathan Levin, 2012. "Contract Pricing in Consumer Credit Markets," Econometrica, Econometric Society, vol. 80(4), pages 1387-1432, July.
    14. Mónica Carabias à lvaro, 2016. "Disparos Fotogrã Ficos En El Marco Del Conflicto. Espaã‘A, 1936-1939," Arte y sociedad. Revista de investigación, Servicios Académicos Intercontinentales SL, issue 11, october.
    15. Araújo, Luis Cesar Gonçalves de, 1975. "Administração de pessoal (Desenvolvimento de recursos humanos)," RAE - Revista de Administração de Empresas, FGV-EAESP Escola de Administração de Empresas de São Paulo (Brazil), vol. 15(3), May.
    16. Heien, Dale & Wessells, Cathy Roheim, 1990. "Demand Systems Estimation with Microdata: A Censored Regression Approach," Journal of Business & Economic Statistics, American Statistical Association, vol. 8(3), pages 365-371, July.
    17. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    18. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    19. Soizic Adam & J. Bonnans & Ruben Paraisy & Sébastien Veyrat, 2011. "Application of convex lexicographical optimization to the balance of GRTgaz gas grid," Journal of Global Optimization, Springer, vol. 49(3), pages 415-423, March.
    20. Neha Khanna & Duane Chapman, 2010. "Guns And Oil: An Analysis Of Conventional Weapons Trade In The Post‐Cold War Era," Economic Inquiry, Western Economic Association International, vol. 48(2), pages 434-459, April.
    21. Suresh Divakar & Brian T. Ratchford & Venkatesh Shankar, 2005. "Practice Prize Article—: A Multichannel, Multiregion Sales Forecasting Model and Decision Support System for Consumer Packaged Goods," Marketing Science, INFORMS, vol. 24(3), pages 334-350, July.
    22. Hal R. Varian, 2014. "Big Data: New Tricks for Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 3-28, Spring.
    23. Cao, Lei & Tang, Yaojie & Fang, Guiyin, 2015. "Preparation and properties of shape-stabilized phase change materials based on fatty acid eutectics and cellulose composites for thermal energy storage," Energy, Elsevier, vol. 80(C), pages 98-103.
    24. Felipe Caro & Jérémie Gallien, 2010. "Inventory Management of a Fast-Fashion Retail Network," Operations Research, INFORMS, vol. 58(2), pages 257-273, April.
    25. Bester, C. Alan & Hansen, Christian, 2009. "Identification of Marginal Effects in a Nonparametric Correlated Random Effects Model," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(2), pages 235-250.
    26. Liran Einav & Theresa Kuchler & Jonathan Levin & Neel Sundaresan, 2015. "Assessing Sale Strategies in Online Markets Using Matched Listings," American Economic Journal: Microeconomics, American Economic Association, vol. 7(2), pages 215-247, May.
    27. Vidossich, Franco, 1971. "Transferencia del conocimiento técnico en la industria de máquinas-herramientas del Brasil, segunda parte: aplicación al Brasil," Sede de la CEPAL en Santiago (Estudios e Investigaciones) 29876, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL).
    28. James Roumasset, 1997. "Preservationism Versus Sustainable Development in the Asia Pacific Region," Working Papers 199709-R, University of Hawaii at Manoa, Department of Economics.
    29. Faith, T., 1991. "Bertrand-Edgeworth Competition With Sequential Capacity Choice," Purdue University Economics Working Papers 997, Purdue University, Department of Economics.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Evgeny A. Antipov & Elena B. Pokryshevskaya, 2020. "Interpretable machine learning for demand modeling with high-dimensional data using Gradient Boosting Machines and Shapley values," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 19(5), pages 355-364, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Evgeniy M. Ozhegov & Alina Ozhegova, 2020. "Regression tree model for prediction of demand with heterogeneity and censorship," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(3), pages 489-500, April.
    2. Evgeniy M. Ozhegov & Alina Ozhegova, 2017. "Regression Tree Model for Analysis of Demand with Heterogeneity and Censorship," HSE Working papers WP BRP 174/EC/2017, National Research University Higher School of Economics.
    3. Arthur Charpentier & Emmanuel Flachaire & Antoine Ly, 2017. "Econom\'etrie et Machine Learning," Papers 1708.06992, arXiv.org, revised Mar 2018.
    4. Bryan T. Kelly & Asaf Manela & Alan Moreira, 2019. "Text Selection," NBER Working Papers 26517, National Bureau of Economic Research, Inc.
    5. Gregory S. Crawford & Nicola Pavanini & Fabiano Schivardi, 2018. "Asymmetric Information and Imperfect Competition in Lending Markets," American Economic Review, American Economic Association, vol. 108(7), pages 1659-1701, July.
    6. Arthur Charpentier & Emmanuel Flachaire & Antoine Ly, 2018. "Économétrie & Machine Learning," Working Papers hal-01568851, HAL.
    7. Daniele Guariso, 2018. "Terrorist Attacks and Immigration Rhetoric: A Natural Experiment on British MPs," Working Paper Series 1218, Department of Economics, University of Sussex Business School.
    8. Raval, Devesh & Rosenbaum, Ted & Wilson, Nathan E., 2021. "How do machine learning algorithms perform in predicting hospital choices? evidence from changing environments," Journal of Health Economics, Elsevier, vol. 78(C).
    9. Bonnet, Céline & Richards, Timothy J., 2016. "Models of Consumer Demand for Differentiated Products," TSE Working Papers 16-741, Toulouse School of Economics (TSE).
    10. Xiaohong Chen & Andres Santos, 2018. "Overidentification in Regular Models," Econometrica, Econometric Society, vol. 86(5), pages 1771-1817, September.
    11. Chernozhukov, Victor & Fernández-Val, Iván & Hoderlein, Stefan & Holzmann, Hajo & Newey, Whitney, 2015. "Nonparametric identification in panels using quantiles," Journal of Econometrics, Elsevier, vol. 188(2), pages 378-392.
    12. Paul Ellickson & Sanjog Misra, 2012. "Enriching interactions: Incorporating outcome data into static discrete games," Quantitative Marketing and Economics (QME), Springer, vol. 10(1), pages 1-26, March.
    13. Männasoo, Kadri, 2022. "Working hours and gender wage differentials: Evidence from the American Working Conditions Survey," Labour Economics, Elsevier, vol. 76(C).
    14. Sumit Agarwal & Souphala Chomsisengphet & Neale Mahoney & Strö & Johannes bel, 2015. "Do Banks Pass Through Credit Expansions? The Marginal Profitability of Consumer Lending During the Great Recession," CESifo Working Paper Series 5521, CESifo.
    15. Dunker, Fabian & Hoderlein, Stefan & Kaido, Hiroaki, 2014. "Nonparametric Identification of Endogenous and Heterogeneous Aggregate Demand Models: Complements, Bundles and the Market Level," Economics Series 307, Institute for Advanced Studies.
    16. Lidia Ceriani & Sergio Olivieri & Marco Ranzani, 2023. "Housing, imputed rent, and household welfare," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 21(1), pages 131-168, March.
    17. Croux, Christophe & Jagtiani, Julapa & Korivi, Tarunsai & Vulanovic, Milos, 2020. "Important factors determining Fintech loan default: Evidence from a lendingclub consumer platform," Journal of Economic Behavior & Organization, Elsevier, vol. 173(C), pages 270-296.
    18. Timothy B. Armstrong & Michal Kolesár, 2021. "Sensitivity analysis using approximate moment condition models," Quantitative Economics, Econometric Society, vol. 12(1), pages 77-108, January.
    19. Halko, Marja-Liisa & Lappalainen, Olli & Sääksvuori, Lauri, 2021. "Do non-choice data reveal economic preferences? Evidence from biometric data and compensation-scheme choice," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 87-104.
    20. Chernozhukov, Victor & Fernández-Val, Iván & Newey, Whitney K., 2019. "Nonseparable multinomial choice models in cross-section and panel data," Journal of Econometrics, Elsevier, vol. 211(1), pages 104-116.

    More about this item

    Keywords

    demand; censorship; machine learning; prediction.;
    All these keywords.

    JEL classification:

    • D12 - Microeconomics - - Household Behavior - - - Consumer Economics: Empirical Analysis
    • C24 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Truncated and Censored Models; Switching Regression Models; Threshold Regression Models
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hig:wpaper:200/ec/2018. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Shamil Abdulaev or Shamil Abdulaev (email available below). General contact details of provider: https://edirc.repec.org/data/hsecoru.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.