IDEAS home Printed from https://ideas.repec.org/p/crm/wpaper/2008.html
   My bibliography  Save this paper

Inference for Ranks with Applications to Mobility across Neighborhoods and Academic Achievement across Countries

Author

Listed:
  • Magne Mogstad

    (University of Chicago Statistics Norway & NBER)

  • Joseph P. Romano

    (Stanford University)

  • Azeem M. Shaikh

    (University of Chicago)

  • Daniel Wilhelm

    (University College London)

Abstract

It is often desired to rank different populations according to the value of some feature of each population. For example, it may be desired to rank neighborhoods according to some measure of intergenerational mobility or countries according to some measure of academic achievement. These rankings are invariably computed using estimates rather than the true values of these features. As a result, there may be considerable uncertainty concerning the rank of each population. In this paper, we consider the problem of accounting for such uncertainty by constructing confidence sets for the rank of each population. We consider both the problem of constructing marginal confidence sets for the rank of a particular population as well as simultaneous confidence sets for the ranks of all populations. We show how to construct such confidence sets under weak assumptions. An important feature of all of our constructions is that they remain computationally feasible even when the number of populations is very large. We apply our theoretical results to re-examine the rankings of both neighborhoods in the United States in terms of intergenerational mobility and developed countries in terms of academic achievement. The conclusions about which countries do best and worst at reading, math, and science are fairly robust to accounting for uncertainty. By comparison, several celebrated findings about intergenerational mobility in the United states are not robust to taking uncertainty into account.

Suggested Citation

  • Magne Mogstad & Joseph P. Romano & Azeem M. Shaikh & Daniel Wilhelm, 2020. "Inference for Ranks with Applications to Mobility across Neighborhoods and Academic Achievement across Countries," RF Berlin - CReAM Discussion Paper Series 2008, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
  • Handle: RePEc:crm:wpaper:2008
    as

    Download full text from publisher

    File URL: https://www.cream-migration.org/publ_uploads/CDP_08_20.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, July.
    2. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1163-1228.
    3. Isaiah Andrews & Toru Kitagawa & Adam McCloskey, 2018. "Inference on winners," CeMMAP working papers CWP31/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    4. Dylan Shane Connor & Michael Storper, 2020. "The changing geography of social mobility in the United States," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(48), pages 30309-30317, December.
    5. Martin Klein & Tommy Wright & Jerzy Wieczorek, 2020. "A joint confidence region for an overall ranking of populations," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(3), pages 589-606, June.
    6. P. Bauer & P. Hackl & G. Hommel & E. Sonnemann, 1986. "Multiple testing of pairs of one-sided hypotheses," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 33(1), pages 121-127, December.
    7. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility I: Childhood Exposure Effects," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1107-1162.
    8. Victor Chernozhukov & Denis Chetverikov & Kengo Kato & Yuta Koike, 2019. "Improved Central Limit Theorem and bootstrap approximations in high dimensions," Papers 1912.10529, arXiv.org, revised May 2022.
    9. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    10. Joseph P. Romano & Azeem M. Shaikh & Michael Wolf, 2014. "A Practical Two‐Step Method for Testing Moment Inequalities," Econometrica, Econometric Society, vol. 82, pages 1979-2002, September.
    11. Victor Chernozhukov & Denis Chetverikov & Kengo Kato, 2012. "Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors," Papers 1212.6906, arXiv.org, revised Jan 2018.
    12. M. C. Jones & Arthur Pewsey, 2009. "Sinh-arcsinh distributions," Biometrika, Biometrika Trust, vol. 96(4), pages 761-780.
    13. Xie, Minge & Singh, Kesar & Zhang, Cun-Hui, 2009. "Confidence Intervals for Population Ranks in the Presence of Ties and Near Ties," Journal of the American Statistical Association, American Statistical Association, vol. 104(486), pages 775-788.
    14. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    15. Raj Chetty & Nathaniel Hendren & Patrick Kline & Emmanuel Saez, 2014. "Where is the land of Opportunity? The Geography of Intergenerational Mobility in the United States," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(4), pages 1553-1623.
    16. Raj Chetty & John N. Friedman & Nathaniel Hendren & Maggie R. Jones & Sonya R. Porter, 2018. "The Opportunity Atlas: Mapping the Childhood Roots of Social Mobility," NBER Working Papers 25147, National Bureau of Economic Research, Inc.
    17. Simon Breakspear, 2012. "The Policy Impact of PISA: An Exploration of the Normative Effects of International Benchmarking in School System Performance," OECD Education Working Papers 71, OECD Publishing.
    18. Peter Bergman & Raj Chetty & Stefanie DeLuca & Nathaniel Hendren & Lawrence F. Katz & Christopher Palmer, 2019. "Creating Moves to Opportunity: Experimental Evidence on Barriers to Neighborhood Choice," NBER Working Papers 26164, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andros Kourtellos & Chih Ming Tan & Steven N. Durlauf, 2022. "The Great Gatsby Curve," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 571-605, August.
    2. Jiaying Gu & Roger Koenker, 2021. "Ranking and Selection from Pairwise Comparisons: Empirical Bayes Methods for Citation Analysis," Papers 2112.11064, arXiv.org.
    3. Jiaying Gu & Roger Koenker, 2023. "Invidious Comparisons: Ranking and Selection as Compound Decisions," Econometrica, Econometric Society, vol. 91(1), pages 1-41, January.
    4. Dillon Bowen, 2022. "Simple models predict behavior at least as well as behavioral scientists," Papers 2208.01167, arXiv.org.
    5. Heckman, James J. & Landerso, Rasmus, 2021. "Lessons from Denmark about Inequality and Social Mobility," IZA Discussion Papers 14185, Institute of Labor Economics (IZA).
    6. Dionissi Aliprantis & Kristen Tauber & Hal Martin, 2022. "What Determines the Success of Housing Mobility Programs?," Working Papers 2022-043, Human Capital and Economic Opportunity Working Group.
    7. Johannes König & David I. Stern & Richard S.J. Tol, 2022. "Confidence Intervals for Recursive Journal Impact Factors," Tinbergen Institute Discussion Papers 22-038/III, Tinbergen Institute.
    8. Will Davis & Alexander Gordan & Rusty Tchernis, 2021. "Measuring the spatial distribution of health rankings in the United States," Health Economics, John Wiley & Sons, Ltd., vol. 30(11), pages 2921-2936, November.
    9. Heckman, James & Landersø, Rasmus, 2022. "Lessons for Americans from Denmark about inequality and social mobility," Labour Economics, Elsevier, vol. 77(C).
    10. Isaiah Andrews & Toru Kitagawa & Adam McCloskey, 2018. "Inference on winners," CeMMAP working papers CWP31/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    11. Dingel, Jonathan & Tintelnot, Felix, 2020. "Spatial Economics for Granular Settings," CEPR Discussion Papers 14819, C.E.P.R. Discussion Papers.
    12. Jiaying Gu & Roger Koenker, 2023. "Reply to: Comments on “Invidious Comparisons: Ranking and Selection as Compound Decisions”," Econometrica, Econometric Society, vol. 91(1), pages 61-66, January.
    13. Deutscher, Nathan & Mazumder, Bhashkar, 2020. "Intergenerational mobility across Australia and the stability of regional estimates," Labour Economics, Elsevier, vol. 66(C).
    14. Daniel Wilhelm & Magne Mogstad & Azeem Shaikh, 2021. "Finite- and Large-Sample Inference for Ranks using Multinomial Data with an Application to Ranking Political Parties," RF Berlin - CReAM Discussion Paper Series 2132, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    15. Dionissi Aliprantis & Hal Martin, 2020. "Neighborhood Sorting Obscures Neighborhood Effects in the Opportunity Atlas," Working Papers 20-37, Federal Reserve Bank of Cleveland.
    16. Cody Cook & Pearl Z. Li & Ariel J. Binder, 2023. "Where to Build Affordable Housing? Evaluating the Tradeoffs of Location," Working Papers 23-62, Center for Economic Studies, U.S. Census Bureau.
    17. Magne Mogstad & Joseph P. Romano & Azeem M. Shaikh & Daniel Wilhelm, 2023. "A Comment on: “Invidious Comparisons: Ranking and Selection as Compound Decisions” by Jiaying Gu and Roger Koenker," Econometrica, Econometric Society, vol. 91(1), pages 53-60, January.
    18. Alberto Alesina & Sebastian Hohmann & Stelios Michalopoulos & Elias Papaioannou, 2021. "Intergenerational Mobility in Africa," Econometrica, Econometric Society, vol. 89(1), pages 1-35, January.
    19. Dionissi Aliprantis & Hal Martin & Kristen Tauber, 2020. "What Determines the Success of Housing Mobility Programs?," Working Papers 20-36R, Federal Reserve Bank of Cleveland, revised 19 Oct 2022.
    20. Eckert,Fabian & Kleineberg,Tatjana Karina, 2021. "Saving the American Dream ? Education Policies in Spatial General Equilibrium," Policy Research Working Paper Series 9574, The World Bank.
    21. David M. Kaplan, 2020. "Inference on Consensus Ranking of Distributions," Working Papers 2010, Department of Economics, University of Missouri.
    22. Hartley, Robert Paul & Lamarche, Carlos & Ziliak, James P., 2023. "Bootstrapping quantile correlations with an application for income status across generations," Economics Letters, Elsevier, vol. 228(C).
    23. Nybom, Martin & Stuhler, Jan, 2021. "Intergenerational mobility in a recession: Evidence from Sweden," Working Paper Series 2021:11, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    24. Jiaying Gu & Roger Koenker, 2020. "Invidious Comparisons: Ranking and Selection as Compound Decisions," Papers 2012.12550, arXiv.org, revised Sep 2021.
    25. Dillon Bowen, 2022. "Bayesian ranking and selection with applications to field studies, economic mobility, and forecasting," Papers 2208.02038, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Will Davis & Alexander Gordan & Rusty Tchernis, 2021. "Measuring the spatial distribution of health rankings in the United States," Health Economics, John Wiley & Sons, Ltd., vol. 30(11), pages 2921-2936, November.
    2. Isaiah Andrews & Toru Kitagawa & Adam McCloskey, 2018. "Inference on winners," CeMMAP working papers CWP31/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    3. Daniel Wilhelm & Magne Mogstad & Azeem Shaikh, 2021. "Finite- and Large-Sample Inference for Ranks using Multinomial Data with an Application to Ranking Political Parties," RF Berlin - CReAM Discussion Paper Series 2132, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    4. Chong Lu, 2022. "The effect of migration on rural residents’ intergenerational subjective social status mobility in China," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(5), pages 3279-3308, October.
    5. Ning Jia & Raven Molloy & Christopher Smith & Abigail Wozniak, 2023. "The Economics of Internal Migration: Advances and Policy Questions," Journal of Economic Literature, American Economic Association, vol. 61(1), pages 144-180, March.
    6. Erin Cottle Hunt & Frank N. Caliendo, 2023. "Social security and risk sharing: the role of economic mobility across generations," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 30(5), pages 1374-1407, October.
    7. Kuhn, Andreas, 2022. "The Geography of Occupational Choice: Empirical Evidence from the Swiss Apprenticeship Market," IZA Discussion Papers 15679, Institute of Labor Economics (IZA).
    8. Mariana Laverde, 2022. "Distance to Schools and Equal Access in School Choice Systems," Boston College Working Papers in Economics 1046, Boston College Department of Economics.
    9. Jason M. DeBacker & P. Wesley Routon, 2021. "A culture of despair? Inequality and expectations of educational success," Contemporary Economic Policy, Western Economic Association International, vol. 39(3), pages 573-588, July.
    10. Handy, Christopher & Shester, Katharine L., 2022. "Local changes in intergenerational mobility," Labour Economics, Elsevier, vol. 78(C).
    11. Ihlanfeldt, Keith & Yang, Cynthia Fan, 2021. "Single-family rentals and neighborhood racial integration✰," Journal of Housing Economics, Elsevier, vol. 53(C).
    12. Tamar Ramot-Nyska, 2023. "Changing Residential Mobility Considerations: The Case of Public Housing in Israel," Bank of Israel Working Papers 2023.15, Bank of Israel.
    13. Francisco Meneses, 2021. "Intergenerational Mobility After Expanding Educational Opportunities: A Quasi Experiment," Working Papers 586, ECINEQ, Society for the Study of Economic Inequality.
    14. Ellora Derenoncourt, 2022. "Can You Move to Opportunity? Evidence from the Great Migration," American Economic Review, American Economic Association, vol. 112(2), pages 369-408, February.
    15. Victor Chernozhukov & Denis Chetverikov & Kengo Kato, 2013. "Testing Many Moment Inequalities," CeMMAP working papers 65/13, Institute for Fiscal Studies.
    16. Bratu, Cristina & Bolotnyy, Valentin, 2023. "Immigrant intergenerational mobility: A focus on childhood environment," European Economic Review, Elsevier, vol. 151(C).
    17. Alex Bell & Raj Chetty & Xavier Jaravel & Neviana Petkova & John Van Reenen, 2019. "Who Becomes an Inventor in America? The Importance of Exposure to Innovation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 647-713.
    18. Cobb-Clark, Deborah A. & Dahmann, Sarah C. & Salamanca, Nicolás & Zhu, Anna, 2022. "Intergenerational disadvantage: Learning about equal opportunity from social assistance receipt," Labour Economics, Elsevier, vol. 79(C).
    19. John Gathergood & Fabian Gunzinger & Benedict Guttman-Kenney & Edika Quispe-Torreblanca & Neil Stewart, 2020. "Levelling Down and the COVID-19 Lockdowns: Uneven Regional Recovery in UK Consumer Spending," Papers 2012.09336, arXiv.org, revised Dec 2020.
    20. Kalra, Aarushi, 2021. "A 'Ghetto' of One's Own: Communal Violence, Residential Segregation and Group Education Outcomes in India," SocArXiv rzjct, Center for Open Science.

    More about this item

    Keywords

    Confidence sets; Directional errors; Familywise error rate; Intergenerational mobility; Multiple testing; PISA; Ranks;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • D31 - Microeconomics - - Distribution - - - Personal Income and Wealth Distribution
    • I20 - Health, Education, and Welfare - - Education - - - General
    • J62 - Labor and Demographic Economics - - Mobility, Unemployment, Vacancies, and Immigrant Workers - - - Job, Occupational and Intergenerational Mobility; Promotion

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:crm:wpaper:2008. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CReAM Administrator or Matthew Nibloe (email available below). General contact details of provider: https://edirc.repec.org/data/cmucluk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.