IDEAS home Printed from https://ideas.repec.org/p/cwl/cwldpp/2542.html

From Unstructured Data to Demand Counterfactuals: Theory and Practice

Author

Listed:
  • Timothy Christensen

    (Yale University)

  • Giovanni Compiani

    (University of Chicago)

Abstract

Empirical models of multi-product demand rely on low-dimensional product representations to capture substitution patterns, increasingly using proxies built from unstructured data. When proxies are imperfect, standard workflows yield biased counterfactuals and invalid inference. We develop a practical toolkit to address these issues. Our methods apply to market-level and/or individual data, require minimal additional computation, provide simple standard-error formulas, and accommodate proxies from fine-tuned models. Further, we propose diagnostics to assess proxy quality. Our methods yield meaningful improvements in predicting substitution in empirically calibrated simulations and in an application where we assess counterfactual prediction performance against a ground truth.

Suggested Citation

  • Timothy Christensen & Giovanni Compiani, 2026. "From Unstructured Data to Demand Counterfactuals: Theory and Practice," Cowles Foundation Discussion Papers 2542, Cowles Foundation for Research in Economics, Yale University.
    • Handle: RePEc:cwl:cwldpp:2542
    as

    Download full text from publisher

    File URL: https://cowles.yale.edu/sites/default/files/2026-07/d2542.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Hansen, Bruce E, 1996. "Inference When a Nuisance Parameter Is Not Identified under the Null Hypothesis," Econometrica, Econometric Society, vol. 64(2), pages 413-430, March.
    2. Andrews, Donald W K & Ploberger, Werner, 1994. "Optimal Tests When a Nuisance Parameter Is Present Only under the Alternative," Econometrica, Econometric Society, vol. 62(6), pages 1383-1414, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Albert J.F. Yang & William N. Trumbull & Chin Wei Yang & Bwo‐Nung Huang, 2011. "On The Relationship Between Military Expenditure, Threat, And Economic Growth: A Nonlinear Approach," Defence and Peace Economics, Taylor & Francis Journals, vol. 22(4), pages 449-457, April.
    2. Philip Hans Franses & Dick van Dijk, 2006. "A simple test for PPP among traded goods," Applied Financial Economics, Taylor & Francis Journals, vol. 16(1-2), pages 19-27.
    3. Li, Kunpeng & Lin, Wei, 2024. "Threshold spatial autoregressive model," Journal of Econometrics, Elsevier, vol. 244(1).
    4. Alexander Mayer & Davide Raggi, 2025. "Estimation and inference in models with multiple behavioural equilibria," Papers 2512.04541, arXiv.org, revised Sep 2026.
    5. Dang, Viet Anh & Kim, Minjoo & Shin, Yongcheol, 2014. "Asymmetric adjustment toward optimal capital structure: Evidence from a crisis," International Review of Financial Analysis, Elsevier, vol. 33(C), pages 226-242.
    6. Chung‐Hua Shen & Hsing‐Hua Hsu, 2022. "The determinants of Asian banking crises—Application of the panel threshold logit model," International Review of Finance, International Review of Finance Ltd., vol. 22(1), pages 248-277, March.
    7. Lahiani, A. & Scaillet, O., 2009. "Testing for threshold effect in ARFIMA models: Application to US unemployment rate data," International Journal of Forecasting, Elsevier, vol. 25(2), pages 418-428.
    8. Munehisa Kasuya, 2003. "Regime-Switching Approach to Monetary Policy Effects: Empirical Studies using a Smooth Transition Vector Autoregressive Model," Bank of Japan Working Paper Series Research and Statistics D, Bank of Japan.
    9. Che, Chou Ming, 2013. "Panel threshold analysis of Taiwan's outbound visitors," Economic Modelling, Elsevier, vol. 33(C), pages 787-793.
    10. Öğünç, Fethi & Akdoğan, Kurmaş & Başer, Selen & Chadwick, Meltem Gülenay & Ertuğ, Dilara & Hülagü, Timur & Kösem, Sevim & Özmen, Mustafa Utku & Tekatlı, Necati, 2013. "Short-term inflation forecasting models for Turkey and a forecast combination analysis," Economic Modelling, Elsevier, vol. 33(C), pages 312-325.
    11. Franses, Philip Hans, 2006. "Forecasting in Marketing," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 18, pages 983-1012, Elsevier.
    12. Singh, Tarlok, 2014. "On the regime-switching and asymmetric dynamics of economic growth in the OECD countries," Research in Economics, Elsevier, vol. 68(2), pages 169-192.
    13. Kadilli, Anjeza & Krishnakumar, Jaya, 2022. "Smooth Transition Simultaneous Equation Models," Journal of Economic Dynamics and Control, Elsevier, vol. 145(C).
    14. Antonio Diez De Los Rios & René Garcia, 2011. "Assessing and valuing the nonlinear structure of hedge fund returns," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(2), pages 193-212, March.
    15. Donald W.K. Andrews & Werner Ploberger, 1994. "Testing for Serial Correlation Against an ARMA(1,1) Process," Cowles Foundation Discussion Papers 1077, Cowles Foundation for Research in Economics, Yale University.
    16. Pesaran, M. Hashem & Timmermann, Allan, 2009. "Testing Dependence Among Serially Correlated Multicategory Variables," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 325-337.
    17. Buraschi, Andrea & Jiltsov, Alexei, 2005. "Inflation risk premia and the expectations hypothesis," Journal of Financial Economics, Elsevier, vol. 75(2), pages 429-490, February.
    18. Burcu Kapar & William Pouliot, 2013. "Multiple Change-Point Detection in Linear Regression Models via U-Statistic Type Processes," Discussion Papers 13-13, Department of Economics, University of Birmingham.
    19. Olmo, Jose & Sanso-Navarro, Marcos, 2026. "A nonparametric spatial regression model using partitioning estimators," Econometrics and Statistics, Elsevier, vol. 37(C), pages 126-153.
    20. Wei, Lili & Zhang, Chunli & Su, Jen-Je & Yang, Lixiong, 2021. "Panel threshold spatial Durbin models with individual fixed effects," Economics Letters, Elsevier, vol. 201(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cwl:cwldpp:2542. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Brittany Ladd (email available below). General contact details of provider: https://edirc.repec.org/data/cowleus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.