IDEAS home Printed from
   My bibliography  Save this paper

An analysis of consumer panel data


  • Andrew Leicester

    () (Institute for Fiscal Studies)

  • Zoe Oldfield

    () (Institute for Fiscal Studies and Institute for Fiscal Studies)


In terms of collecting comprehensive panel expenditure data, there are trade-offs to be made in terms of the demands imposed on respondents and the level of detail and spending coverage collected. Existing comprehensive spending data tends to be cross-sectional whilst panel studies include only limited expenditure questions that record spending only as broad aggregates. More recently, economists have begun to use spending information collected by market research companies that records very detailed spending down to the barcode level from a panel of households, usually recorded by in-home barcode scanners, which may provide considerable advantages over existing data more commonly used in social sciences. However, there has not been a comprehensive assessment of the strengths and weaknesses of this kind of data collection method and the potential implications of survey mode on the recorded data. This paper seeks to address this, by an in-depth examination of scanner data from one company, Taylor Nelson Sofres (TNS), on grocery purchases over a five-year period. We assess how far the ongoing demands of participation inherent in this kind of survey lead to 'fatigue' in respondents' recording of their spending and compare the demographic representativeness of the data to the well-established Expenditure and Food Survey (EFS), constructing weights for the TNS that account for observed demographic differences. We also look at demographic transitions, comparing the panel aspect of the TNS to the British Household Panel Study (BHPS). We examine in detail the expenditure data in the TNS and EFS surveys and discuss the implications of this method of data collection for survey attrition. Broadly, we suggest that problems of fatigue and attrition may not be so severe as may be expected, though there are some differences in expenditure levels (and to some extent patterns of spending) that cannot be attributed to demographic or time differences in the two surveys alone and may be suggestive of survey mode effects. Demographic transitions appear to occur less frequently than we might expect which may limit the usefulness of the panel aspect of the data for some applications.

Suggested Citation

  • Andrew Leicester & Zoe Oldfield, 2009. "An analysis of consumer panel data," IFS Working Papers W09/09, Institute for Fiscal Studies.
  • Handle: RePEc:ifs:ifsewp:09/09

    Download full text from publisher

    File URL:
    Download Restriction: no


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Griffith, Rachel & Krol, Michal & Smith, Kate, 2015. "Store Brands and the Role of Advertising," CEPR Discussion Papers 10877, C.E.P.R. Discussion Papers.
    2. Rachel Griffith & Martin O'Connell & Kate Smith, 2016. "Shopping Around: How Households Adjusted Food Spending Over the Great Recession," Economica, London School of Economics and Political Science, vol. 83(330), pages 247-280, April.
    3. repec:bla:econom:v:84:y:2017:i:333:p:34-53 is not listed on IDEAS
    4. Rachel Griffith & Martin O'Connell & Kate Smith, 2017. "The Importance of Product Reformulation Versus Consumer Choice in Improving Diet Quality," Economica, London School of Economics and Political Science, vol. 84(333), pages 34-53, January.

    More about this item


    Household panel data; scanner data; expenditure; food; duration models; attrition;

    JEL classification:

    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models
    • C41 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Duration Analysis; Optimal Timing Strategies

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ifs:ifsewp:09/09. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Emma Hyman). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.