IDEAS home Printed from https://ideas.repec.org/p/hig/wpaper/233-ec-2020.html
   My bibliography  Save this paper

Multilevel Modeling For Economists: Why, When And How

Author

Listed:
  • Aleksey Oshchepkov

    (National Research University Higher School of Economics)

  • Anna Shirokanova

    (National Research University Higher School of Economics)

Abstract

Multilevel modeling (MLM, also known as hierarchical linear modeling, HLM) is a methodological framework widely used in the social sciences to analyze data with a hierarchical structure, where lower units of aggregation are ‘nested’ in higher units, including longitudinal data. In economics, however, MLM is used very rarely. Instead, economists use separate econometric techniques including cluster-robust standard errors and fixed effects models. In this paper, we review the methodological literature and contrast the econometric techniques typically used in economics with the analysis of hierarchical data using MLM. Our review suggests that economic techniques are generally less convenient, flexible, and efficient compared to MLM. The important limitation of MLM, however, is its inability to deal with the omitted variable problem at the lowest level of data, while standard economic techniques may be complemented by quasi-experimental methods mitigating this problem. It is unlikely, though, that this limitation can explain and justify the rare use of MLM in economics. Overall, we conclude that MLM has been unreasonably ignored in economics, and we encourage economists to apply this framework by providing ‘when and how’ guidelines

Suggested Citation

  • Aleksey Oshchepkov & Anna Shirokanova, 2020. "Multilevel Modeling For Economists: Why, When And How," HSE Working papers WP BRP 233/EC/2020, National Research University Higher School of Economics.
  • Handle: RePEc:hig:wpaper:233/ec/2020
    as

    Download full text from publisher

    File URL: https://wp.hse.ru/data/2020/06/29/1610354484/233EC2020.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Arpino, Bruno & Mealli, Fabrizia, 2011. "The specification of the propensity score in multilevel observational studies," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1770-1780, April.
    2. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    3. Katja Möhring & Alexander Schmidt, 2012. "Multilevel tools," German Stata Users' Group Meetings 2012 06, Stata Users Group.
    4. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    5. Jakob Kapeller & Matthias Aistleitner & Stefan Steinerberger, 2017. "Citation Patterns in Economics and Beyond: Assessing the Peculiarities of Economics from Two Scientometric Perspectives," ICAE Working Papers 60, Johannes Kepler University, Institute for Comprehensive Analysis of the Economy.
    6. T. D. Stanley & Stephen B. Jarrell, 2005. "Meta‐Regression Analysis: A Quantitative Method of Literature Surveys," Journal of Economic Surveys, Wiley Blackwell, vol. 19(3), pages 299-308, July.
    7. Jeffrey M. Wooldridge, 2005. "Simple solutions to the initial conditions problem in dynamic, nonlinear panel data models with unobserved heterogeneity," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(1), pages 39-54, January.
    8. Hausman, Jerry A & Taylor, William E, 1981. "Panel Data and Unobservable Individual Effects," Econometrica, Econometric Society, vol. 49(6), pages 1377-1398, November.
    9. Alicia H. Munnell, 1990. "Why has productivity growth declined? Productivity and public investment," New England Economic Review, Federal Reserve Bank of Boston, issue Jan, pages 3-22.
    10. Swamy, P A V B & Tavlas, George S, 1995. "Random Coefficient Models: Theory and Applications," Journal of Economic Surveys, Wiley Blackwell, vol. 9(2), pages 165-196, June.
    11. Lisa Barrow & Cecilia Elena Rouse, 2005. "Causality, causality, causality: the view of education inputs and outputs from economics," Working Paper Series WP-05-15, Federal Reserve Bank of Chicago.
    12. Swamy, P A V B, 1970. "Efficient Inference in a Random Coefficient Regression Model," Econometrica, Econometric Society, vol. 38(2), pages 311-323, March.
    13. Jee-Seon Kim & Edward Frees, 2007. "Multilevel Modeling with Correlated Effects," Psychometrika, Springer;The Psychometric Society, vol. 72(4), pages 505-533, December.
    14. Carmen M. Reinhart & Kenneth S. Rogoff, 2010. "Growth in a Time of Debt," American Economic Review, American Economic Association, vol. 100(2), pages 573-578, May.
    15. Sylvie Goetgeluk & Stijn Vansteelandt & Els Goetghebeur, 2008. "Estimation of controlled direct effects," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(5), pages 1049-1066, November.
    16. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    17. Hausman, Jerry, 2015. "Specification tests in econometrics," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 38(2), pages 112-134.
    18. José R. Zubizarreta & Luke Keele, 2017. "Optimal Multilevel Matching in Clustered Observational Studies: A Case Study of the Effectiveness of Private Schools Under a Large-Scale Voucher System," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 547-560, April.
    19. H. Baltagi, Badi & Heun Song, Seuck & Cheol Jung, Byoung, 2001. "The unbalanced nested error component regression model," Journal of Econometrics, Elsevier, vol. 101(2), pages 357-381, April.
    20. Saxonhouse, Gary R, 1976. "Estimated Parameters as Dependent Variables," American Economic Review, American Economic Association, vol. 66(1), pages 178-183, March.
    21. Reinhard Schunck, 2013. "Within and between estimates in random-effects models: Advantages and drawbacks of correlated random effects and hybrid models," Stata Journal, StataCorp LP, vol. 13(1), pages 65-76, March.
    22. repec:hal:spmain:info:hdl:2441/67ft27s7u58ocangahl1jigu6p is not listed on IDEAS
    23. Tom Snijders, 1996. "Analysis of longitudinal data using the hierarchical linear model," Quality & Quantity: International Journal of Methodology, Springer, vol. 30(4), pages 405-426, November.
    24. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    25. Hansen, Christian B., 2007. "Asymptotic properties of a robust variance matrix estimator for panel data when T is large," Journal of Econometrics, Elsevier, vol. 141(2), pages 597-620, December.
    26. Koop, Gary M & Tobias, Justin, 2004. "Learning About Heterogeneity in Returns to Schooling," Staff General Research Papers Archive 12008, Iowa State University, Department of Economics.
    27. M. Fourcade & E. Ollion & Y. Algan, 2015. "The Superiority of Economists," Voprosy Ekonomiki, NP Voprosy Ekonomiki, issue 7.
    28. Marion Fourcade & Etienne Ollion & Yann Algan, 2015. "La superioridad de los economistas," Revista de Economía Institucional, Universidad Externado de Colombia - Facultad de Economía, vol. 17(33), pages 13-43, July-Dece.
    29. Lewis, Jeffrey B. & Linzer, Drew A., 2005. "Estimating Regression Models in Which the Dependent Variable Is Based on Estimates," Political Analysis, Cambridge University Press, vol. 13(4), pages 345-364.
    30. Maas, Cora J. M. & Hox, J.J.Joop J., 2004. "The influence of violations of assumptions on multilevel parameter estimates and their standard errors," Computational Statistics & Data Analysis, Elsevier, vol. 46(3), pages 427-440, June.
    31. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    32. King, Gary & Roberts, Margaret E., 2015. "How Robust Standard Errors Expose Methodological Problems They Do Not Fix, and What to Do About It," Political Analysis, Cambridge University Press, vol. 23(2), pages 159-179, April.
    33. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    34. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    35. László Mátyás & Patrick Sevestre (ed.), 2008. "The Econometrics of Panel Data," Advanced Studies in Theoretical and Applied Econometrics, Springer, number 978-3-540-75892-1, July-Dece.
    36. Oakes, J. Michael, 2004. "The (mis)estimation of neighborhood effects: causal inference for a practicable social epidemiology," Social Science & Medicine, Elsevier, vol. 58(10), pages 1929-1952, May.
    37. Michael T. French & Ioana Popovici, 2011. "That instrument is lousy! In search of agreement when using instrumental variables estimation in substance use research," Health Economics, John Wiley & Sons, Ltd., vol. 20(2), pages 127-146, February.
    38. Kurt J. Beron & James C. Murdoch & Mark A. Thayer, 1999. "Hierarchical Linear Models With Application to Air Pollution in the South Coast Air Basin," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 81(5), pages 1123-1127.
    39. Heisig, Jan Paul & Schaeffer, Merlin, 2019. "Why You Should Always Include a Random Slope for the Lower-Level Variable Involved in a Cross-Level Interaction," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 35(2), pages 258-279.
    40. Nigel Rice & Andrew Jones, 1997. "Multilevel models and health economics," Health Economics, John Wiley & Sons, Ltd., vol. 6(6), pages 561-575, November.
    41. Bell, Andrew & Jones, Kelvyn, 2015. "Explaining Fixed Effects: Random Effects Modeling of Time-Series Cross-Sectional and Panel Data," Political Science Research and Methods, Cambridge University Press, vol. 3(1), pages 133-153, January.
    42. Rik Pieters & Hans Baumgartner, 2002. "Who Talks to Whom? Intra- and Interdisciplinary Communication of Economics Journals," Journal of Economic Literature, American Economic Association, vol. 40(2), pages 483-509, June.
    43. Hausman, Jerry A & Wise, David A, 1979. "Attrition Bias in Experimental and Panel Data: The Gary Income Maintenance Experiment," Econometrica, Econometric Society, vol. 47(2), pages 455-473, March.
    44. Andrew Bell & Malcolm Fairbrother & Kelvyn Jones, 2019. "Fixed and random effects models: making an informed choice," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(2), pages 1051-1074, March.
    45. Plümper, Thomas & Troeger, Vera E., 2007. "Efficient Estimation of Time-Invariant and Rarely Changing Variables in Finite Sample Panel Analyses with Unit Fixed Effects," Political Analysis, Cambridge University Press, vol. 15(2), pages 124-139, April.
    46. Andrew Bell & Kelvyn Jones & Malcolm Fairbrother, 2018. "Understanding and misunderstanding group mean centering: a commentary on Kelley et al.’s dangerous practice," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(5), pages 2031-2036, September.
    47. Brambor, Thomas & Clark, William Roberts & Golder, Matt, 2006. "Understanding Interaction Models: Improving Empirical Analyses," Political Analysis, Cambridge University Press, vol. 14(1), pages 63-82, January.
    48. Patrick Sevestre & Laszlo Matyas, 2008. "The Econometrics of Panel Data," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-00279977, HAL.
    49. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    50. Jee-Seon Kim & Edward Frees, 2006. "Omitted Variables in Multilevel Models," Psychometrika, Springer;The Psychometric Society, vol. 71(4), pages 659-690, December.
    51. Plümper, Thomas & Troeger, Vera E., 2011. "Fixed-Effects Vector Decomposition: Properties, Reliability, and Instruments," Political Analysis, Cambridge University Press, vol. 19(2), pages 147-164, April.
    52. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    53. Daniel Stegmueller, 2013. "How Many Countries for Multilevel Modeling? A Comparison of Frequentist and Bayesian Approaches," American Journal of Political Science, John Wiley & Sons, vol. 57(3), pages 748-761, July.
    54. Arellano, M, 1987. "Computing Robust Standard Errors for Within-Groups Estimators," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 49(4), pages 431-434, November.
    55. Craig R. Carter & Gavin Meschnig & Lutz Kaufmann, 2015. "Moving to the Next Level: Why Our Discipline Needs More Multilevel Theorization," Journal of Supply Chain Management, Institute for Supply Management, vol. 51(4), pages 94-102, October.
    56. Nees Jan Eck & Ludo Waltman, 2010. "Software survey: VOSviewer, a computer program for bibliometric mapping," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(2), pages 523-538, August.
    57. Andrew Bell & Ron Johnston & Kelvyn Jones, 2015. "Stylised fact or situated messiness? The diverse effects of increasing debt on national economic growth," Journal of Economic Geography, Oxford University Press, vol. 15(2), pages 449-472.
    58. Fairbrother, Malcolm, 2014. "Two Multilevel Modeling Techniques for Analyzing Comparative Longitudinal Survey Datasets," Political Science Research and Methods, Cambridge University Press, vol. 2(1), pages 119-140, April.
    59. Jeffrey M. Wooldridge, 2003. "Cluster-Sample Methods in Applied Econometrics," American Economic Review, American Economic Association, vol. 93(2), pages 133-138, May.
    60. repec:adr:anecst:y:1999:i:55-56:p:05 is not listed on IDEAS
    61. Brown, James N & Light, Audrey, 1992. "Interpreting Panel Data on Job Tenure," Journal of Labor Economics, University of Chicago Press, vol. 10(3), pages 219-257, July.
    62. Kedar, Orit & Shively, W. Phillips, 2005. "Introduction to the Special Issue," Political Analysis, Cambridge University Press, vol. 13(4), pages 297-300.
    63. Sophia Rabe-Hesketh & Anders Skrondal, 2012. "Multilevel and Longitudinal Modeling Using Stata, 3rd Edition," Stata Press books, StataCorp LP, edition 3, number mimus2, March.
    64. Joseph L Dieleman & Tara Templin, 2014. "Random-Effects, Fixed-Effects and the within-between Specification for Clustered Data in Observational Health Studies: A Simulation Study," PLOS ONE, Public Library of Science, vol. 9(10), pages 1-17, October.
    65. Erkan Ozkaya, H. & Dabas, Chitra & Kolev, Kalin & Hult, G. Tomas M. & Dahlquist, Steven H. & Manjeshwar, Sonia Arun, 2013. "An assessment of hierarchical linear modeling in international business, management, and marketing," International Business Review, Elsevier, vol. 22(4), pages 663-677.
    66. Moulton, Brent R, 1990. "An Illustration of a Pitfall in Estimating the Effects of Aggregate Variables on Micro Unit," The Review of Economics and Statistics, MIT Press, vol. 72(2), pages 334-338, May.
    67. Heisig, Jan Paul & Schaeffer, Merlin & Giesecke, Johannes, 2017. "The Costs of Simplicity: Why Multilevel Models May Benefit from Accounting for Cross-Cluster Differences in the Effects of Controls," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 82(4), pages 796-827.
    68. Pagan, Adrian, 1984. "Econometric Issues in the Analysis of Regressions with Generated Regressors," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 25(1), pages 221-247, February.
    69. Marion Fourcade & Etienne Ollion & Yann Algan, 2015. "The Superiority of Economists," Journal of Economic Perspectives, American Economic Association, vol. 29(1), pages 89-114, Winter.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pavol Durana & Lucia Michalkova & Andrej Privara & Josef Marousek & Milos Tumpach, 2021. "Does the life cycle affect earnings management and bankruptcy?," Oeconomia Copernicana, Institute of Economic Research, vol. 12(2), pages 425-461, June.
    2. Kabeya Clement Mulamba, 2022. "Relationship between households’ share of food expenditure and income across South African districts: a multilevel regression analysis," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-11, December.
    3. Cepeda-Francese, Camilo A. & Ramírez-Álvarez, Aurora A., 2023. "Reforming justice under a security crisis: The case of the criminal justice reform in Mexico," World Development, Elsevier, vol. 163(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    2. Andrew Bell & Malcolm Fairbrother & Kelvyn Jones, 2019. "Fixed and random effects models: making an informed choice," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(2), pages 1051-1074, March.
    3. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 318, University of California, Davis, Department of Economics.
    4. Rok Spruk & Mitja Kovac, 2019. "Transaction costs and economic growth under common legal system: State‐level evidence from Mexico," Economics and Politics, Wiley Blackwell, vol. 31(2), pages 240-292, July.
    5. Rok Spruk & Mitja Kovac, 2018. "Inefficient Growth," Review of Economics and Institutions, Università di Perugia, vol. 9(2).
    6. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 106, University of California, Davis, Department of Economics.
    7. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    8. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    9. Mitja Kovac & Salvini Datta & Rok Spruk, 2021. "Pharmaceutical Product Liability, Litigation Regimes, and the Propensity to Patent: An Empirical Firm-Level Investigation," SAGE Open, , vol. 11(2), pages 21582440211, April.
    10. Martin Halla & Friedrich Schneider & Alexander Wagner, 2013. "Satisfaction with democracy and collective action problems: the case of the environment," Public Choice, Springer, vol. 155(1), pages 109-137, April.
    11. Miroslav Verbič & Rok Spruk, 2019. "Political economy of pension reforms: an empirical investigation," European Journal of Law and Economics, Springer, vol. 47(2), pages 171-232, April.
    12. Bruno Ferman & Cristine Pinto, 2019. "Inference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 452-467, July.
    13. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    14. Markku Maula & Wouter Stam, 2020. "Enhancing Rigor in Quantitative Entrepreneurship Research," Entrepreneurship Theory and Practice, , vol. 44(6), pages 1059-1090, November.
    15. Cameron, A. Colin & Gelbach, Jonah B. & Miller, Douglas L., 2011. "Robust Inference With Multiway Clustering," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(2), pages 238-249.
    16. Anders Skrondal & Sophia Rabe-Hesketh, 2022. "The Role of Conditional Likelihoods in Latent Variable Modeling," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 799-834, September.
    17. Entorf, Horst & Sattarova, Liliya, 2016. "The Analysis of Prison-Prisoner Data Using Cluster-Sample Econometrics: Prison Conditions and Prisoners' Assessments of the Future," IZA Discussion Papers 10209, Institute of Labor Economics (IZA).
    18. Heisig, Jan Paul & Schaeffer, Merlin & Giesecke, Johannes, 2017. "The Costs of Simplicity: Why Multilevel Models May Benefit from Accounting for Cross-Cluster Differences in the Effects of Controls," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 82(4), pages 796-827.
    19. Jeffrey D. Michler & Anna Josephson, 2022. "Recent developments in inference: practicalities for applied economics," Chapters, in: A Modern Guide to Food Economics, chapter 11, pages 235-268, Edward Elgar Publishing.
    20. Matthew D. Webb, 2023. "Reworking wild bootstrap‐based inference for clustered errors," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 56(3), pages 839-858, August.

    More about this item

    Keywords

    multilevel modeling; hierarchical linear modeling; mixed effects; random effects; fixed effects; random coefficients; clusterization of errors.;
    All these keywords.

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C50 - Mathematical and Quantitative Methods - - Econometric Modeling - - - General
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models
    • A12 - General Economics and Teaching - - General Economics - - - Relation of Economics to Other Disciplines

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hig:wpaper:233/ec/2020. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Shamil Abdulaev or Shamil Abdulaev (email available below). General contact details of provider: https://edirc.repec.org/data/hsecoru.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.