IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/25899.html
   My bibliography  Save this paper

A Machine Learning Analysis of Seasonal and Cyclical Sales in Weekly Scanner Data

Author

Listed:
  • Rishab Guha
  • Serena Ng

Abstract

This paper analyzes weekly scanner data collected for 108 groups at the county level between 2006 and 2014. The data display multi-dimensional weekly seasonal effects that are not exactly periodic but are cross-sectionally dependent. Existing univariate procedures are imperfect and yield adjusted series that continue to display strong seasonality upon aggregation. We suggest augmenting the univariate adjustments with a panel data step that pools information across counties. Machine learning tools are then used to remove the within-year seasonal variations. A demand analysis of the adjusted budget shares finds three factors: one that is trending, and two cyclical ones that are well aligned with the level and change in consumer confidence. The effects of the Great Recession vary across locations and product groups, with consumers substituting towards home cooking away from non-essential goods. The adjusted data also reveal changes in spending to unanticipated shocks at the local level. The data are thus informative about both local and aggregate economic conditions once the seasonal effects are removed. The two-step methodology can be adapted to remove other types of nuisance variations provided that these variations are cross-sectionally dependent.

Suggested Citation

  • Rishab Guha & Serena Ng, 2019. "A Machine Learning Analysis of Seasonal and Cyclical Sales in Weekly Scanner Data," NBER Working Papers 25899, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:25899
    Note: EFG ME
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w25899.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Rachel Griffith & Martin O'Connell & Kate Smith, 2016. "Shopping Around: How Households Adjusted Food Spending Over the Great Recession," Economica, London School of Economics and Political Science, vol. 83(330), pages 247-280, April.
    2. Engle, Robert F & Kozicki, Sharon, 1993. "Testing for Common Features," Journal of Business & Economic Statistics, American Statistical Association, vol. 11(4), pages 369-380, October.
    3. Pierce, David A & Grupe, Michael R & Cleveland, William P, 1984. "Seasonal Adjustment of the Weekly Monetary Aggregates: A Model-based Approach," Journal of Business & Economic Statistics, American Statistical Association, vol. 2(3), pages 260-270, July.
    4. Engle, Robert F & Kozicki, Sharon, 1993. "Testing for Common Features: Reply," Journal of Business & Economic Statistics, American Statistical Association, vol. 11(4), pages 393-395, October.
    5. Tucker McElroy, 2017. "Multivariate Seasonal Adjustment, Economic Identities, and Seasonal Taxonomy," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 35(4), pages 611-625, October.
    6. Arthur Lewbel, 2003. "A rational rank four demand system," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(2), pages 127-135.
    7. Bai, Jushan & Ng, Serena, 2019. "Rank regularized estimation of approximate factor models," Journal of Econometrics, Elsevier, vol. 212(1), pages 78-96.
    8. Fok, Dennis & Hans Franses, Philip & Paap, Richard, 2007. "Seasonality and non-linear price effects in scanner-data-based market-response models," Journal of Econometrics, Elsevier, vol. 138(1), pages 231-251, May.
    9. Lewbel, Arthur, 1991. "The Rank of Demand Systems: Theory and Nonparametric Estimation," Econometrica, Econometric Society, vol. 59(3), pages 711-730, May.
    10. Serena Ng, 2017. "Opportunities and Challenges: Lessons from Analyzing Terabytes of Scanner Data," NBER Working Papers 23673, National Bureau of Economic Research, Inc.
    11. John Muellbauer, 1975. "Aggregation, Income Distribution and Consumer Demand," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 42(4), pages 525-543.
    12. Deaton, Angus S & Muellbauer, John, 1980. "An Almost Ideal Demand System," American Economic Review, American Economic Association, vol. 70(3), pages 312-326, June.
    13. Atif Mian & Amir Sufi, 2010. "Household Leverage and the Recession of 2007–09," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 58(1), pages 74-117, August.
    14. John Geweke, 1978. "The Temporal and Sectoral Aggregation of Seasonally Adjusted Time Series," NBER Chapters, in: Seasonal Analysis of Economic Time Series, pages 411-432, National Bureau of Economic Research, Inc.
    15. Harvey, Andrew & Koopman, Siem Jan & Riani, Marco, 1997. "The Modeling and Seasonal Adjustment of Weekly Observations," Journal of Business & Economic Statistics, American Statistical Association, vol. 15(3), pages 354-368, July.
    16. James Banks & Richard Blundell & Arthur Lewbel, 1997. "Quadratic Engel Curves And Consumer Demand," The Review of Economics and Statistics, MIT Press, vol. 79(4), pages 527-539, November.
    17. Bai, Jushan & Ng, Serena, 2008. "Large Dimensional Factor Analysis," Foundations and Trends(R) in Econometrics, now publishers, vol. 3(2), pages 89-163, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nicolaj N. Mühlbach, 2020. "Tree-based Synthetic Control Methods: Consequences of moving the US Embassy," CREATES Research Papers 2020-04, Department of Economics and Business Economics, Aarhus University.
    2. Venera Timiryanova & Irina Lakman & Vadim Prudnikov & Dina Krasnoselskaya, 2022. "Spatial Dependence of Average Prices for Product Categories and Its Change over Time: Evidence from Daily Data," Forecasting, MDPI, vol. 5(1), pages 1-25, December.
    3. Timiryanova, Venera & Krasnoselskaya, Dina, 2022. "Влияние пандемии Сovid-19 на пространственную динамику продовольственных цен [Covid-19 impact on spatial food prices dynamics]," MPRA Paper 114638, University Library of Munich, Germany.
    4. Christopher Dobronyi & Christian Gouri'eroux, 2020. "Consumer Theory with Non-Parametric Taste Uncertainty and Individual Heterogeneity," Papers 2010.13937, arXiv.org, revised Jan 2021.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Thomas F. Crossley & Hamish W. Low, 2011. "Is The Elasticity Of Intertemporal Substitution Constant?," Journal of the European Economic Association, European Economic Association, vol. 9(1), pages 87-105, February.
    2. LaFrance, Jeffrey T., 2008. "The structure of US food demand," Journal of Econometrics, Elsevier, vol. 147(2), pages 336-349, December.
    3. Veeramani, Venkat N. & Maynard, Leigh J., 2004. "Testing For The Existence Of Price Points In Retail Milk Scanner Data Using Microeconomic Theory," 2004 Annual meeting, August 1-4, Denver, CO 20378, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    4. Barnett, William A. & Serletis, Apostolos, 2008. "Consumer preferences and demand systems," Journal of Econometrics, Elsevier, vol. 147(2), pages 210-224, December.
    5. Toshinobu Matsuda, 2006. "A trigonometric flexible consumer demand system," Canadian Journal of Economics, Canadian Economics Association, vol. 39(1), pages 145-162, February.
    6. Cash, Sean B. & Goddard, Ellen W., 2006. "New Directions in Consumer Behaviour Research," CAFRI: Current Agriculture, Food and Resource Issues, Canadian Agricultural Economics Society, issue 7, pages 1-10, August.
    7. Mette Christensen, 2007. "Integrability of Demand Accounting for Unobservable Heterogeneity: A Test on Panel Data," Economics Discussion Paper Series 0713, Economics, The University of Manchester.
    8. Barnett, William A. & Serletis, Apostolos, 2008. "Measuring Consumer Preferences and Estimating Demand Systems," MPRA Paper 12318, University Library of Munich, Germany.
    9. Crossley, T.F. & Low, H.W., 2005. "Unexploited Connections Between Intra- and Inter-temporal Allocation," Cambridge Working Papers in Economics 0537, Faculty of Economics, University of Cambridge.
    10. Lyssiotou, Panayiota & Pashardes, Panos & Stengos, Thanasis, 2002. "Nesting quadratic logarithmic demand systems," Economics Letters, Elsevier, vol. 76(3), pages 369-374, August.
    11. Matteo Barigozzi & Alessio Moneta, 2016. "Identifying the Independent Sources of Consumption Variation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(2), pages 420-449, March.
    12. Chavas, Jean-Paul & Menon, Martina & Pagani, Elisa & Perali, Federico, 2018. "Collective household welfare and intra-household inequality," Theoretical Economics, Econometric Society, vol. 13(2), May.
    13. Cranfield, John A.L., 2005. "Demands for Food Products Across the Development Spectrum: Application of a Rank Four Demand System," Working Papers 34111, University of Guelph, Department of Food, Agricultural and Resource Economics.
    14. Claudio Michelini, 1999. "Equivalence scales and consumption inequality: a study of household consumption patterns in italy," Departmental Working Papers 1999-02, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    15. Neary, Peter, 2000. "True Multilateral Indexes for International Comparisons of Real Income: Theory and Empirics," CEPR Discussion Papers 2590, C.E.P.R. Discussion Papers.
    16. Jeffrey LaFrance & Rulon Pope, 2008. "The Generalized Quadratic Expenditure System," Working Papers 2008-27, School of Economic Sciences, Washington State University.
    17. H. Youn Kim & Keith R. McLaren & K. K. Gary Wong, 2020. "Valuation of public goods: an intertemporal mixed demand approach," Empirical Economics, Springer, vol. 59(5), pages 2223-2253, November.
    18. Xavier Labandeira & José M. Labeaga & Miguel Rodríguez, 2006. "A Residential Energy Demand System for Spain," The Energy Journal, , vol. 27(2), pages 87-112, April.
    19. Frank Denton & Dean Mountain, 2004. "Aggregation effects on price and expenditure elasticities in a quadratic almost ideal demand system," Canadian Journal of Economics, Canadian Economics Association, vol. 37(3), pages 613-628, August.
    20. Kira Lancker & Julia Bronnmann, 2022. "Substitution Preferences for Fish in Senegal," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 82(4), pages 1015-1045, August.

    More about this item

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • E21 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Consumption; Saving; Wealth
    • E32 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Business Fluctuations; Cycles

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:25899. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.