IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2006.05308.html
   My bibliography  Save this paper

Ensemble Learning with Statistical and Structural Models

Author

Listed:
  • Jiaming Mao
  • Jingzhi Xu

Abstract

Statistical and structural modeling represent two distinct approaches to data analysis. In this paper, we propose a set of novel methods for combining statistical and structural models for improved prediction and causal inference. Our first proposed estimator has the doubly robustness property in that it only requires the correct specification of either the statistical or the structural model. Our second proposed estimator is a weighted ensemble that has the ability to outperform both models when they are both misspecified. Experiments demonstrate the potential of our estimators in various settings, including fist-price auctions, dynamic models of entry and exit, and demand estimation with instrumental variables.

Suggested Citation

  • Jiaming Mao & Jingzhi Xu, 2020. "Ensemble Learning with Statistical and Structural Models," Papers 2006.05308, arXiv.org.
  • Handle: RePEc:arx:papers:2006.05308
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2006.05308
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Emmanuel Guerre & Isabelle Perrigne & Quang Vuong, 2000. "Optimal Nonparametric Estimation of First-Price Auctions," Econometrica, Econometric Society, vol. 68(3), pages 525-574, May.
    2. Keane, Michael P., 2010. "Structural vs. atheoretic approaches to econometrics," Journal of Econometrics, Elsevier, vol. 156(1), pages 3-20, May.
    3. Kitagawa, Toru & Muris, Chris, 2016. "Model averaging in semiparametric estimation of treatment effects," Journal of Econometrics, Elsevier, vol. 193(1), pages 271-289.
    4. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    5. D Benkeser & M Carone & M J Van Der Laan & P B Gilbert, 2017. "Doubly robust nonparametric inference on the average treatment effect," Biometrika, Biometrika Trust, vol. 104(4), pages 863-880.
    6. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    7. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    8. Dmitry Arkhangelsky & Guido W. Imbens, 2019. "Doubly Robust Identification for Causal Panel Data Models," Papers 1909.09412, arXiv.org, revised Feb 2022.
    9. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney K. Newey, 2016. "Double machine learning for treatment and causal parameters," CeMMAP working papers 49/16, Institute for Fiscal Studies.
    10. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    11. Zhiqiang Tan, 2010. "Bounded, efficient and doubly robust estimation with inverse weighting," Biometrika, Biometrika Trust, vol. 97(3), pages 661-682.
    12. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    13. Aguirregabiria, Victor & Mira, Pedro, 2010. "Dynamic discrete choice structural models: A survey," Journal of Econometrics, Elsevier, vol. 156(1), pages 38-67, May.
    14. Elodie Guerre & I. Perrigne & Q.H. Vuong, 2000. "Optimal nonparametric estimation of first-price auctions [[Estimation nonparamétrique optimale des enchères au premier prix]]," Post-Print hal-02697497, HAL.
    15. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    16. Joshua D. Angrist & Alan B. Krueger, 1993. "Split Sample Instrumental Variables," Working Papers 699, Princeton University, Department of Economics, Industrial Relations Section..
    17. Scott, Paul, 2014. "Dynamic Discrete Choice Estimation of Agricultural Land Use," TSE Working Papers 14-526, Toulouse School of Economics (TSE).
    18. Masashi Sugiyama & Taiji Suzuki & Shinichi Nakajima & Hisashi Kashima & Paul Bünau & Motoaki Kawanabe, 2008. "Direct importance estimation for covariate shift adaptation," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 60(4), pages 699-746, December.
    19. Maxwell Kellogg & Magne Mogstad & Guillaume Pouliot & Alexander Torgovitsky, 2020. "Combining Matching and Synthetic Control to Trade off Biases from Extrapolation and Interpolation," NBER Working Papers 26624, National Bureau of Economic Research, Inc.
    20. Lewbel, Arthur & Choi, Jin Young & Zhou, Zhuzhu, 2023. "Over-identified Doubly Robust identification and estimation," Journal of Econometrics, Elsevier, vol. 235(1), pages 25-42.
    21. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey, 2017. "Double/Debiased/Neyman Machine Learning of Treatment Effects," American Economic Review, American Economic Association, vol. 107(5), pages 261-265, May.
    22. Patrick Bajari & Ali Hortacsu, 2005. "Are Structural Estimates of Auction Models Reasonable? Evidence from Experimental Data," Journal of Political Economy, University of Chicago Press, vol. 113(4), pages 703-741, August.
    23. Harry J. Paarsch & Han Hong, 2006. "An Introduction to the Structural Econometrics of Auction Data," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262162350, December.
    24. Pirmin Fessler & Maximilian Kasy, 2019. "How to Use Economic Theory to Improve Estimators: Shrinking Toward Theoretical Restrictions," The Review of Economics and Statistics, MIT Press, vol. 101(4), pages 681-698, October.
    25. Angrist, Joshua D & Krueger, Alan B, 1995. "Split-Sample Instrumental Variables Estimates of the Return to Schooling," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 225-235, April.
    26. Wei-Yin Loh, 2014. "Fifty Years of Classification and Regression Trees," International Statistical Review, International Statistical Institute, vol. 82(3), pages 329-348, December.
    27. Hickman Brent R. & Hubbard Timothy P. & Sağlam Yiğit, 2012. "Structural Econometric Methods in Auctions: A Guide to the Literature," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 67-106, August.
    28. Farrell, Max H., 2015. "Robust inference on average treatment effects with possibly more covariates than observations," Journal of Econometrics, Elsevier, vol. 189(1), pages 1-23.
    29. Athey, Susan & Haile, Philip A., 2007. "Nonparametric Approaches to Auctions," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 60, Elsevier.
    30. Hickman Brent R. & Hubbard Timothy P. & Sağlam Yiğit, 2012. "Structural Econometric Methods in Auctions: A Guide to the Literature," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 67-106, August.
    31. Gérard Biau & Erwan Scornet, 2016. "Rejoinder on: A random forest guided tour," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(2), pages 264-268, June.
    32. Victor Aguirregabiria & Arvind Magesan, 2013. "Euler Equations for the Estimation of Dynamic Discrete Choice Structural Models," Advances in Econometrics, in: Structural Econometric Models, volume 31, pages 3-44, Emerald Group Publishing Limited.
    33. Isabelle Perrigne & Quang Vuong, 2019. "Econometrics of Auctions and Nonlinear Pricing," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 27-54, August.
    34. John Rust, 2014. "The Limits of Inference with Theory: A Review of Wolpin (2013)," Journal of Economic Literature, American Economic Association, vol. 52(3), pages 820-850, September.
    35. Erhan Artuç & Shubham Chaudhuri & John McLaren, 2010. "Trade Shocks and Labor Adjustment: A Structural Empirical Approach," American Economic Review, American Economic Association, vol. 100(3), pages 1008-1045, June.
    36. Raj Chetty, 2009. "Sufficient Statistics for Welfare Analysis: A Bridge Between Structural and Reduced-Form Methods," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 451-488, May.
    37. Hansen, Bruce E. & Racine, Jeffrey S., 2012. "Jackknife model averaging," Journal of Econometrics, Elsevier, vol. 167(1), pages 38-46.
    38. Scornet, Erwan, 2016. "On the asymptotics of random forests," Journal of Multivariate Analysis, Elsevier, vol. 146(C), pages 72-83.
    39. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2016. "Double/Debiased Machine Learning for Treatment and Causal Parameters," Papers 1608.00060, arXiv.org, revised Dec 2017.
    40. Peter Arcidiacono & Paul B. Ellickson, 2011. "Practical Methods for Estimation of Dynamic Discrete Choice Models," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 363-394, September.
    41. Wolpin, Kenneth I., 2013. "The Limits of Inference without Theory," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262019086, December.
    42. Gérard Biau & Erwan Scornet, 2016. "A random forest guided tour," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(2), pages 197-227, June.
    43. Hamish Low & Costas Meghir, 2017. "The Use of Structural Models in Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 33-58, Spring.
    44. Enrique Moral-Benito, 2015. "Model Averaging In Economics: An Overview," Journal of Economic Surveys, Wiley Blackwell, vol. 29(1), pages 46-75, February.
    45. James J. Heckman & Vytlacil, Edward J., 2007. "Econometric Evaluation of Social Programs, Part I: Causal Models, Structural Models and Econometric Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 70, Elsevier.
    46. Michael P. Keane, 2010. "A Structural Perspective on the Experimentalist School," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 47-58, Spring.
    47. James J. Heckman, 2000. "Causal Parameters and Policy Analysis in Economics: A Twentieth Century Retrospective," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 115(1), pages 45-97.
    48. Peter Arcidiacono & Robert A. Miller, 2011. "Conditional Choice Probability Estimation of Dynamic Discrete Choice Models With Unobserved Heterogeneity," Econometrica, Econometric Society, vol. 79(6), pages 1823-1867, November.
    49. Hjort N.L. & Claeskens G., 2003. "Frequentist Model Average Estimators," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 879-899, January.
    50. Heejung Bang & James M. Robins, 2005. "Doubly Robust Estimation in Missing Data and Causal Inference Models," Biometrics, The International Biometric Society, vol. 61(4), pages 962-973, December.
    51. Reiss, Peter C. & Wolak, Frank A., 2007. "Structural Econometric Modeling: Rationales and Examples from Industrial Organization," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 64, Elsevier.
    52. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    53. Claeskens G. & Hjort N.L., 2003. "The Focused Information Criterion," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 900-916, January.
    54. Xinyu Zhang & Dalei Yu & Guohua Zou & Hua Liang, 2016. "Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1775-1790, October.
    55. Karel Vermeulen & Stijn Vansteelandt, 2015. "Bias-Reduced Doubly Robust Estimation," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(511), pages 1024-1036, September.
    56. Aviv Nevo & Michael D. Whinston, 2010. "Taking the Dogma out of Econometrics: Structural Modeling and Credible Inference," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 69-82, Spring.
    57. Editors The, 2007. "From the Editors," Basic Income Studies, De Gruyter, vol. 2(1), pages 1-5, June.
    58. Whitney K. Newey, 2013. "Nonparametric Instrumental Variables Estimation," American Economic Review, American Economic Association, vol. 103(3), pages 550-556, May.
    59. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    60. Aguirregabiria, Victor & Magesan, Arvind, 2013. "Euler Equations for the Estimation of Dynamic Discrete Choice Structural," MPRA Paper 46056, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jiaming Mao & Zhesheng Zheng, 2020. "Structural Regularization," Papers 2004.12601, arXiv.org, revised Jun 2020.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiaming Mao & Zhesheng Zheng, 2020. "Structural Regularization," Papers 2004.12601, arXiv.org, revised Jun 2020.
    2. Sebastian Galiani & Juan Pantano, 2021. "Structural Models: Inception and Frontier," NBER Working Papers 28698, National Bureau of Economic Research, Inc.
    3. Dave Donaldson, 2022. "Blending Theory and Data: A Space Odyssey," Journal of Economic Perspectives, American Economic Association, vol. 36(3), pages 185-210, Summer.
    4. Valente, Marica, 2023. "Policy evaluation of waste pricing programs using heterogeneous causal effect estimation," Journal of Environmental Economics and Management, Elsevier, vol. 117(C).
    5. Thoresen, Thor O. & Vattø, Trine E., 2015. "Validation of the discrete choice labor supply model by methods of the new tax responsiveness literature," Labour Economics, Elsevier, vol. 37(C), pages 38-53.
    6. Ruoyao Shi, 2021. "An Averaging Estimator for Two Step M Estimation in Semiparametric Models," Working Papers 202105, University of California at Riverside, Department of Economics.
    7. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    8. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    9. Difang Huang & Jiti Gao & Tatsushi Oka, 2022. "Semiparametric Single-Index Estimation for Average Treatment Effects," Papers 2206.08503, arXiv.org, revised Oct 2022.
    10. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Mar 2024.
    11. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    12. Mochen Yang & Edward McFowland & Gordon Burtch & Gediminas Adomavicius, 2022. "Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem," INFORMS Joural on Data Science, INFORMS, vol. 1(2), pages 138-155, October.
    13. Zhexiao Lin & Fang Han, 2022. "On regression-adjusted imputation estimators of the average treatment effect," Papers 2212.05424, arXiv.org, revised Jan 2023.
    14. Ruoxuan Xiong & Allison Koenecke & Michael Powell & Zhu Shen & Joshua T. Vogelstein & Susan Athey, 2021. "Federated Causal Inference in Heterogeneous Observational Data," Papers 2107.11732, arXiv.org, revised Apr 2023.
    15. Christopher Conlon & Julie Holland Mortimer, 2021. "Empirical properties of diversion ratios," RAND Journal of Economics, RAND Corporation, vol. 52(4), pages 693-726, December.
    16. Yiyi Huo & Yingying Fan & Fang Han, 2023. "On the adaptation of causal forests to manifold data," Papers 2311.16486, arXiv.org, revised Dec 2023.
    17. Elliott Ash & Daniel L. Chen & Sergio Galletta, 2022. "Measuring Judicial Sentiment: Methods and Application to US Circuit Courts," Economica, London School of Economics and Political Science, vol. 89(354), pages 362-376, April.
    18. Nathan Canen & Kristopher Ramsay, 2023. "Quantifying Theory in Politics: Identification, Interpretation and the Role of Structural Methods," Papers 2302.01897, arXiv.org.
    19. Mark Kattenberg & Bas Scheer & Jurre Thiel, 2023. "Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences," CPB Discussion Paper 452, CPB Netherlands Bureau for Economic Policy Analysis.
    20. Antonelli Joseph & Cefalu Matthew, 2020. "Averaging causal estimators in high dimensions," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 92-107, January.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2006.05308. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.