IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i10p2296-2305.html
   My bibliography  Save this article

Confidence intervals for dependent data: Equating non-overlap with statistical significance

Author

Listed:
  • Afshartous, David
  • Preston, Richard A.

Abstract

We revisit the problem of determining confidence interval widths for the comparison of means. For the independent two-sample (two-sided) case, Goldstein and Healy (1995) draw attention to the fact that comparisons based on 95% error bars are not very effective in assessing the statistical significance of the difference in means and derive the correct confidence interval for such a comparison. We provide an extension to Goldstein and Healy (1995) to account for the correlation structure and unequal variances. We use the results to develop rules of thumb for evaluating differences, in an exploratory manner, like Moses (1987) and Cumming (2009), from the independent case. We illustrate the method for the simple comparison of two means in a real data set, provide R code that may be easily implemented in practice, and discuss the extension of the method to other applied problems.

Suggested Citation

  • Afshartous, David & Preston, Richard A., 2010. "Confidence intervals for dependent data: Equating non-overlap with statistical significance," Computational Statistics & Data Analysis, Elsevier, vol. 54(10), pages 2296-2305, October.
    • Handle: RePEc:eee:csdana:v:54:y:2010:i:10:p:2296-2305
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(10)00156-8
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, July.
    2. Schenker N. & Gentleman J. F., 2001. "On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals," The American Statistician, American Statistical Association, vol. 55, pages 182-186, August.
    3. Unknown, 2007. "FAPRI U.S Baseline Briefing Book," FAPRI-MU Report Series 44776, Food and Agricultural Policy Research Institute (FAPRI).
    4. Harvey Goldstein & Michael J. R. Healy, 1995. "The Graphical Presentation of a Collection of Means," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(1), pages 175-177, January.
    5. David Afshartous & Michael Wolf, 2007. "Avoiding ‘data snooping’ in multilevel and mixed effects models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(4), pages 1035-1059, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kimihiro Noguchi & Fernando Marmolejo-Ramos, 2016. "Assessing Equality of Means Using the Overlap of Range-Preserving Confidence Intervals," The American Statistician, Taylor & Francis Journals, vol. 70(4), pages 325-334, October.
    2. Stefan J. Reichelstein & Anshuman Sahoo, 2015. "Cost- and Price Dynamics of Solar PV Modules," CESifo Working Paper Series 5674, CESifo.
    3. Brini, Elisa & Borgen, Solveig Topstad & Borgen, Nicolai T., 2023. "Avoiding the eyeballing fallacy: Visualizing statistical differences between estimates using the pheatplot command," SocArXiv fghcd, Center for Open Science.
    4. Tim Goedemé & Tess Penne & Tine Hufkens & Alexandros Karakitsios & Anikó Bernát & Bori Simonovits & Elena Carillo Alvarez & Eleni Kanavitsa & Irene Cussó Parcerisas & Jordi Riera Romaní & Lauri Mäkine, 2017. "What Does It Mean To Live on the Poverty Threshold? Lessons From Reference Budgets," Working Papers 1707, Herman Deleeck Centre for Social Policy, University of Antwerp.
    5. Columbu, Silvia & Porcu, Mariano & Sulis, Isabella, 2021. "University choice and the attractiveness of the study area: Insights on the differences amongst degree programmes in Italy based on generalised mixed-effect models," Socio-Economic Planning Sciences, Elsevier, vol. 74(C).
    6. Tim Goedemé & Karel Van den Bosch & Lina Salanauskaite & Gerlinde Verbist, 2013. "Testing the Statistical Significance of Microsimulation Results: A Plea," International Journal of Microsimulation, International Microsimulation Association, vol. 6(3), pages 50-77.
    7. Giorgio E. Montanari & Marco Doretti, 2019. "Ranking Nursing Homes’ Performances Through a Latent Markov Model with Fixed and Random Effects," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 146(1), pages 307-326, November.
    8. Tim Goedemé & Karel Van den Bosch & Lina Salanauskaite & Gerlinde Verbist, 2013. "Testing the Statistical Significance of Microsimulation Results: Often Easier than You Think. A Technical Note," ImPRovE Working Papers 13/10, Herman Deleeck Centre for Social Policy, University of Antwerp.
    9. Roman P. Kuster & Wilhelmus J. A. Grooten & Victoria Blom & Daniel Baumgartner & Maria Hagströmer & Örjan Ekblom, 2021. "How Accurate and Precise Can We Measure the Posture and the Energy Expenditure Component of Sedentary Behaviour with One Sensor?," IJERPH, MDPI, vol. 18(11), pages 1-13, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. George Leckie & Harvey Goldstein, 2009. "The limitations of using school league tables to inform school choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 835-851, October.
    2. David Afshartous & Michael Wolf, 2007. "Avoiding ‘data snooping’ in multilevel and mixed effects models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(4), pages 1035-1059, October.
    3. Tom Tregenza & Petri T Niemelä & Rolando Rodríguez-Muñoz & Paul E Hopwood, 2022. "Environment and mate attractiveness in a wild insect [Female pied flycatchers choose territory quality and not male characteristics]," Behavioral Ecology, International Society for Behavioral Ecology, vol. 33(5), pages 999-1006.
    4. David M. Ritzwoller & Joseph P. Romano, 2019. "Uncertainty in the Hot Hand Fallacy: Detecting Streaky Alternatives to Random Bernoulli Sequences," Papers 1908.01406, arXiv.org, revised Apr 2021.
    5. Victor Chernozhukov & Denis Chetverikov & Kengo Kato, 2013. "Testing Many Moment Inequalities," CeMMAP working papers 65/13, Institute for Fiscal Studies.
    6. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    7. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    8. Rute M. Caeiro & Pedro C. Vicente, 2020. "Knowledge of vitamin A deficiency and crop adoption: Evidence from a field experiment in Mozambique," Agricultural Economics, International Association of Agricultural Economists, vol. 51(2), pages 175-190, March.
    9. Orazio Attanasio & Sarah Cattan & Emla Fitzsimons & Costas Meghir & Marta Rubio-Codina, 2020. "Estimating the Production Function for Human Capital: Results from a Randomized Controlled Trial in Colombia," American Economic Review, American Economic Association, vol. 110(1), pages 48-85, January.
    10. Bhalotra, Sonia & Clarke, Damian & Mühlrad, Hanna & Palme, Mårten, 2021. "Health and Labor Market Impacts of Twin Birth : Evidence from a Swedish IVF Policy Mandate," The Warwick Economics Research Paper Series (TWERPS) 1391, University of Warwick, Department of Economics.
    11. Bajgrowicz, Pierre & Scaillet, Olivier, 2012. "Technical trading revisited: False discoveries, persistence tests, and transaction costs," Journal of Financial Economics, Elsevier, vol. 106(3), pages 473-491.
    12. Jaschke Philipp & Sulin Sardoschau & Marco Tabellini, 2021. "Scared Straight? Threat and Assimilation of Refugees in Germany," RF Berlin - CReAM Discussion Paper Series 2136, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    13. Hermes, Henning & Mierisch, Fabian & Peter, Frauke & Wiederhold, Simon & Lergetporer, Philipp, 2023. "Discrimination on the Child Care Market: A Nationwide Field Experiment," IZA Discussion Papers 16082, Institute of Labor Economics (IZA).
    14. Orazio Attanasio & Helen Baker-Henningham & Raquel Bernal & Costas Meghir & Diana Pineda & Marta Rubio-Codina, 2022. "Early Stimulation and Nutrition: The Impacts of a Scalable Intervention," Journal of the European Economic Association, European Economic Association, vol. 20(4), pages 1395-1432.
    15. Grácio, Matilde & Vicente, Pedro C., 2021. "Information, get-out-the-vote messages, and peer influence: Causal effects on political behavior in Mozambique," Journal of Development Economics, Elsevier, vol. 151(C).
    16. Pierfrancesco Rolla & Patricia Justino, 2022. "The social consequences of organized crime in Italy," WIDER Working Paper Series wp-2022-106, World Institute for Development Economic Research (UNU-WIDER).
    17. Chicu, Mark & Masten, Matthew A., 2013. "A specification test for discrete choice models," Economics Letters, Elsevier, vol. 121(2), pages 336-339.
    18. Rossi, Pauline & Villar, Paola, 2020. "Private health investments under competing risks: Evidence from malaria control in Senegal," Journal of Health Economics, Elsevier, vol. 73(C).
    19. Billings, Stephen B. & Johnson, Erik B., 2012. "A non-parametric test for industrial specialization," Journal of Urban Economics, Elsevier, vol. 71(3), pages 312-331.
    20. Cygan-Rehm, Kamila & Karbownik, Krzysztof, 2022. "The effects of incentivizing early prenatal care on infant health," Journal of Health Economics, Elsevier, vol. 83(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:10:p:2296-2305. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.