IDEAS home Printed from https://ideas.repec.org/a/gai/recdev/r2295.html
   My bibliography  Save this article

Technical and Methodological Challenges of Collecting Price Data from Online Retailers
[Технические И Методологические Проблемы Сбора Данных О Ценах Онлайн-Ритейлеров]

Author

Listed:
  • Alexey S. Evseev

    (Russian Presidential Academy of National Economy and Public Administration)

  • Rodion R. Latypov

    (JSC Arowana Capital)

  • Egor A. Postolit

    (JSC Arowana Capital)

  • Elena S. Sinelnikova-Muryleva

    (Russian Presidential Academy of National Economy and Public Administration)

Abstract

Price data from online retailers is a valuable source for economics. The use of these data makes it possible to refine inflation forecasts and anticipate future trends in the moment, refine estimates of price rigidity and the conclusions of theoretical pricing models, and test the law of one price. However, there are major difficulties in the data collection process that are not obvious and can threaten both the quality of the data collected and the sustainability of the collection process over time. The article, for the first time in the literature, discusses in detail the technical and methodological problems that impede the continuous collection of data on the network and presents our experience in solving these problems. The pros and cons of solutions to emerging problems are discussed. The article was written on the basis of the RANEPA state assignment research programme for 2022.

Suggested Citation

  • Alexey S. Evseev & Rodion R. Latypov & Egor A. Postolit & Elena S. Sinelnikova-Muryleva, 2022. "Technical and Methodological Challenges of Collecting Price Data from Online Retailers [Технические И Методологические Проблемы Сбора Данных О Ценах Онлайн-Ритейлеров]," Russian Economic Development, Gaidar Institute for Economic Policy, issue 11, pages 36-45, November.
    • Handle: RePEc:gai:recdev:r2295
    as

    Download full text from publisher

    File URL: http://www.iep.ru/files/RePEc/gai/recdev/r2295.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alberto Cavallo, 2017. "Are Online and Offline Prices Similar? Evidence from Large Multi-channel Retailers," American Economic Review, American Economic Association, vol. 107(1), pages 283-303, January.
    2. Alberto Cavallo, 2018. "Scraped Data and Sticky Prices," The Review of Economics and Statistics, MIT Press, vol. 100(1), pages 105-119, March.
    3. Cavallo, Alberto, 2013. "Online and official price indexes: Measuring Argentina's inflation," Journal of Monetary Economics, Elsevier, vol. 60(2), pages 152-165.
    4. Alberto Cavallo & Roberto Rigobon, 2016. "The Billion Prices Project: Using Online Prices for Measurement and Research," Journal of Economic Perspectives, American Economic Association, vol. 30(2), pages 151-178, Spring.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alexey S. Evseev & Rodion R. Latypov & Egor A. Postolit & Elena S. Sinelnikova-Muryleva, 2022. "Техданные О Ценах Онлайн-Ритейлеров Обладают Огромной Ценностью С Точки Зрения Экономической Науки: Их Использование Позволяет Уточнять Прогнозы Инфляции И Предвосхищать Будущие Тенденции В Моменте, К," Russian Economic Development (in Russian), Gaidar Institute for Economic Policy, issue 11, pages 36-45, November.
    2. Hillen, Judith, 2018. "Web Scraping For Food Price Research," 58th Annual Conference, Kiel, Germany, September 12-14, 2018 275840, German Association of Agricultural Economists (GEWISOLA).
    3. Macias, Paweł & Stelmasiak, Damian & Szafranek, Karol, 2023. "Nowcasting food inflation with a massive amount of online prices," International Journal of Forecasting, Elsevier, vol. 39(2), pages 809-826.
    4. David Staines, 2023. "Stochastic Equilibrium the Lucas Critique and Keynesian Economics," Papers 2312.16214, arXiv.org.
    5. W. Erwin Diewert & Kevin J. Fox, 2020. "Measuring Real Consumption and CPI Bias under Lockdown Conditions," NBER Working Papers 27144, National Bureau of Economic Research, Inc.
    6. Resce, Giuliano & Maynard, Diana, 2018. "What matters most to people around the world? Retrieving Better Life Index priorities on Twitter," Technological Forecasting and Social Change, Elsevier, vol. 137(C), pages 61-75.
    7. Bogner Alexandra & Jerger Jürgen, 2023. "Big data in monetary policy analysis—a critical assessment," Economics and Business Review, Sciendo, vol. 9(2), pages 27-40, April.
    8. Aparicio, Diego & Bertolotto, Manuel I., 2020. "Forecasting inflation with online prices," International Journal of Forecasting, Elsevier, vol. 36(2), pages 232-247.
    9. Harchaoui, Tarek M. & Janssen, Robert V., 2018. "How can big data enhance the timeliness of official statistics?," International Journal of Forecasting, Elsevier, vol. 34(2), pages 225-234.
    10. Alberto Cavallo & Roberto Rigobon, 2016. "The Billion Prices Project: Using Online Prices for Measurement and Research," Journal of Economic Perspectives, American Economic Association, vol. 30(2), pages 151-178, Spring.
    11. Ademmer, Martin & Beckmann, Joscha & Bode, Eckhardt & Boysen-Hogrefe, Jens & Funke, Manuel & Hauber, Philipp & Heidland, Tobias & Hinz, Julian & Jannsen, Nils & Kooths, Stefan & Söder, Mareike & Stame, 2021. "Big Data in der makroökonomischen Analyse," Kieler Beiträge zur Wirtschaftspolitik 32, Kiel Institute for the World Economy (IfW Kiel).
    12. Diego Aparicio & Alberto Cavallo, 2021. "Targeted Price Controls on Supermarket Products," The Review of Economics and Statistics, MIT Press, vol. 103(1), pages 60-71, March.
    13. Robert Laskowski, 2022. "Differences between Online Prices and the Consumer Prices Index During Covid-19 in Germany," ACTA VSFS, University of Finance and Administration, vol. 16(1), pages 76-87.
    14. Alberto Cavallo, 2017. "Are Online and Offline Prices Similar? Evidence from Large Multi-channel Retailers," American Economic Review, American Economic Association, vol. 107(1), pages 283-303, January.
    15. Barış Soybilgen & M. Ege Yazgan & Hüseyin Kaya, 2023. "Nowcasting Turkish Food Inflation Using Daily Online Prices," Journal of Business Cycle Research, Springer;Centre for International Research on Economic Tendency Surveys (CIRET), vol. 19(2), pages 171-190, September.
    16. Patrick Bajari & Zhihao Cen & Victor Chernozhukov & Manoj Manukonda & Jin Wang & Ramon Huerta & Junbo Li & Ling Leng & George Monokroussos & Suhas Vijaykunar & Shan Wan, 2023. "Hedonic prices and quality adjusted price indices powered by AI," CeMMAP working papers 08/23, Institute for Fiscal Studies.
    17. Drenik, Andrés & Perez, Diego J., 2020. "Price setting under uncertainty about inflation," Journal of Monetary Economics, Elsevier, vol. 116(C), pages 23-38.
    18. Cavallo, Alberto & Kryvtsov, Oleksiy, 2023. "What can stockouts tell us about inflation? Evidence from online micro data," Journal of International Economics, Elsevier, vol. 146(C).
    19. Fernando Alvarez & Francesco Lippi & Aleksei Oskolkov, 2022. "The Macroeconomics of Sticky Prices with Generalized Hazard Functions [“Optimal Inattention to the Stock Market With Information Costs and Transactions Costs,”]," The Quarterly Journal of Economics, Oxford University Press, vol. 137(2), pages 989-1038.

    More about this item

    Keywords

    prices of online retailers; web-scrapping; inflation; alternative data; big data;
    All these keywords.

    JEL classification:

    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • C82 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Macroeconomic Data; Data Access
    • E31 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Price Level; Inflation; Deflation
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gai:recdev:r2295. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Olga Beloborodova (email available below). General contact details of provider: https://edirc.repec.org/data/gaidaru.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.