IDEAS home Printed from https://ideas.repec.org/p/amu/wpaper/2012-11.html
   My bibliography  Save this paper

Combination of "Combinations of P-values

Author

Listed:
  • Xuguang Sheng
  • Lan Cheng

Abstract

We investigate the impact of an uncertain number of false individual null hypotheses on commonly used p-value combination methods and find that the performance of these methods varies substantially under such uncertainty. These variations yield conflicting results in meta-analysis, motivating the development of a new, reconciling test. We consequently develop a combination of "combinations of p-values" (CCP) test that maintains good power properties across such uncertainty. We base the CCP test on a simple union of rejections decision rule that exploits the similarity between any two p-value combination methods. Monte Carlo simulations show that our test controls size and closely tracks the power of the best individual methods.

Suggested Citation

  • Xuguang Sheng & Lan Cheng, 2012. "Combination of "Combinations of P-values," Working Papers 2012-11, American University, Department of Economics.
  • Handle: RePEc:amu:wpaper:2012-11
    DOI: 10.17606/2q5g-qw29
    as

    Download full text from publisher

    File URL: https://doi.org/10.17606/2q5g-qw29
    File Function: First version, 2012
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ana-Maria Dumitru & Giovanni Urga, 2011. "Identifying Jumps in Financial Assets: A Comparison Between Nonparametric Jump Tests," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(2), pages 242-255, October.
    2. Jean-Marie Dufour & Lynda Khalaf & Marcel Voia, 2013. "Finite-sample resampling-based combined hypothesis tests, with applications to serial correlation and predictability," CIRANO Working Papers 2013s-40, CIRANO.
    3. Christian Bayer & Christoph Hanck, 2013. "Combining non-cointegration tests," Journal of Time Series Analysis, Wiley Blackwell, vol. 34(1), pages 83-95, January.
    4. Harvey, David I. & Leybourne, Stephen J. & Taylor, A.M. Robert, 2009. "Unit Root Testing In Practice: Dealing With Uncertainty Over The Trend And Initial Condition," Econometric Theory, Cambridge University Press, vol. 25(3), pages 587-636, June.
    5. Mauro Costantini & Claudio Lupi, 2013. "A Simple Panel-CADF Test for Unit Roots," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 75(2), pages 276-296, April.
    6. N. Gregory Mankiw & Ricardo Reis, 2002. "Sticky Information versus Sticky Prices: A Proposal to Replace the New Keynesian Phillips Curve," The Quarterly Journal of Economics, Oxford University Press, vol. 117(4), pages 1295-1328.
    7. Xuguang Sheng & Jingyun Yang, 2013. "Truncated Product Methods for Panel Unit Root Tests," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 75(4), pages 624-636, August.
    8. Andrade, Philippe & Le Bihan, Hervé, 2013. "Inattentive professional forecasters," Journal of Monetary Economics, Elsevier, vol. 60(8), pages 967-982.
    9. Olivier Coibion & Yuriy Gorodnichenko, 2012. "What Can Survey Forecasts Tell Us about Information Rigidities?," Journal of Political Economy, University of Chicago Press, vol. 120(1), pages 116-159.
    10. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    11. Choi, In, 2001. "Unit root tests for panel data," Journal of International Money and Finance, Elsevier, vol. 20(2), pages 249-272, April.
    12. Olivier Coibion & Yuriy Gorodnichenko, 2015. "Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts," American Economic Review, American Economic Association, vol. 105(8), pages 2644-2678, August.
    13. David I. Harvey & Stephen J. Leybourne & A. M. Robert Taylor, 2011. "Testing for Unit Roots and the Impact of Quadratic Trends, with an Application to Relative Primary Commodity Prices," Econometric Reviews, Taylor & Francis Journals, vol. 30(5), pages 514-547, October.
    14. M. Hashem Pesaran, 2007. "A simple panel unit root test in the presence of cross-section dependence," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(2), pages 265-312.
    15. Lopez, Claude, 2008. "Evidence of purchasing power parity for the floating regime period," Journal of International Money and Finance, Elsevier, vol. 27(1), pages 156-164, February.
    16. Jushan Bai & Serena Ng, 2004. "A PANIC Attack on Unit Roots and Cointegration," Econometrica, Econometric Society, vol. 72(4), pages 1127-1177, July.
    17. repec:taf:jnlbes:v:30:y:2012:i:2:p:242-255 is not listed on IDEAS
    18. Maddala, G S & Wu, Shaowen, 1999. "A Comparative Study of Unit Root Tests with Panel Data and a New Simple Test," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 61(0), pages 631-652, Special I.
    19. Loughin, Thomas M., 2004. "A systematic comparison of methods for combining p-values from independent tests," Computational Statistics & Data Analysis, Elsevier, vol. 47(3), pages 467-485, October.
    20. Lahiri, Kajal & Sheng, Xuguang, 2008. "Evolution of forecast disagreement in a Bayesian learning model," Journal of Econometrics, Elsevier, vol. 144(2), pages 325-340, June.
    21. Wu, Jyh-Lin & Wu, Shaowen, 2001. "Is Purchasing Power Parity Overvalued?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 33(3), pages 804-812, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Algieri, Bernardina & Leccadito, Arturo, 2019. "Price volatility and speculative activities in futures commodity markets: A combination of combinations of p-values test," Journal of Commodity Markets, Elsevier, vol. 13(C), pages 40-54.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul Hubert, 2014. "FOMC Forecasts as a Focal Point for Private Expectations," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 46(7), pages 1381-1420, October.
    2. Andrade, Philippe & Crump, Richard K. & Eusepi, Stefano & Moench, Emanuel, 2016. "Fundamental disagreement," Journal of Monetary Economics, Elsevier, vol. 83(C), pages 106-128.
    3. Manzan, Sebastiano, 2021. "Are professional forecasters Bayesian?," Journal of Economic Dynamics and Control, Elsevier, vol. 123(C).
    4. Xuguang Sheng & Jingyun Yang, 2013. "Truncated Product Methods for Panel Unit Root Tests," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 75(4), pages 624-636, August.
    5. Dovern, Jonas & Fritsche, Ulrich & Loungani, Prakash & Tamirisa, Natalia, 2015. "Information rigidities: Comparing average and individual forecasts for a large international panel," International Journal of Forecasting, Elsevier, vol. 31(1), pages 144-154.
    6. Michael Clements, 2016. "Are Macro-Forecasters Essentially The Same? An Analysis of Disagreement, Accuracy and Efficiency," ICMA Centre Discussion Papers in Finance icma-dp2016-08, Henley Business School, Reading University.
    7. Yara de Almeida Campos Cordeiro & Wagner Piazza Gaglianone & João Victor Issler, 2017. "Inattention in individual expectations," Economia, ANPEC - Associação Nacional dos Centros de Pós-Graduação em Economia [Brazilian Association of Graduate Programs in Economics], vol. 17(1), pages 40-59.
    8. Clements, Michael P., 2012. "Do professional forecasters pay attention to data releases?," International Journal of Forecasting, Elsevier, vol. 28(2), pages 297-308.
    9. Deschamps, Bruno & Ioannidis, Christos & Ka, Kook, 2020. "High-frequency credit spread information and macroeconomic forecast revision," International Journal of Forecasting, Elsevier, vol. 36(2), pages 358-372.
    10. Kim, Insu & Kim, Young Se, 2019. "Inattentive agents and inflation forecast error dynamics: A Bayesian DSGE approach," Journal of Macroeconomics, Elsevier, vol. 62(C).
    11. Hadri, Kaddour & Kurozumi, Eiji, 2008. "A Simple Panel Stationarity Test in the Presence of Cross-Sectional Dependence," CCES Discussion Paper Series 7, Center for Research on Contemporary Economic Systems, Graduate School of Economics, Hitotsubashi University.
    12. Cushman, David O. & Michael, Nils, 2011. "Nonlinear trends in real exchange rates: A panel unit root test approach," Journal of International Money and Finance, Elsevier, vol. 30(8), pages 1619-1637.
    13. Cornand, Camille & Hubert, Paul, 2020. "On the external validity of experimental inflation forecasts: A comparison with five categories of field expectations," Journal of Economic Dynamics and Control, Elsevier, vol. 110(C).
    14. Ricco, Giovanni & Callegari, Giovanni & Cimadomo, Jacopo, 2016. "Signals from the government: Policy disagreement and the transmission of fiscal shocks," Journal of Monetary Economics, Elsevier, vol. 82(C), pages 107-118.
    15. Paul Hubert, 2015. "ECB Projections as a Tool for Understanding Policy Decisions," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 34(7), pages 574-587, November.
    16. M. Ayhan Kose & Hideaki Matsuoka & Ugo Panizza & Dana Vorisek, 2019. "Inflation Expectations: Review and Evidence," Koç University-TUSIAD Economic Research Forum Working Papers 1904, Koc University-TUSIAD Economic Research Forum.
    17. Anthony Garratt & Kevin Lee & Kalvinder Shields, 2018. "The role of uncertainty, sentiment and cross-country interactions in G7 output dynamics," Canadian Journal of Economics, Canadian Economics Association, vol. 51(2), pages 391-418, May.
    18. Beckmann, Joscha & Reitz, Stefan, 2020. "Information rigidities and exchange rate expectations," Journal of International Money and Finance, Elsevier, vol. 105(C).
    19. Jonas Dovern & Matthias Hartmann, 2017. "Forecast performance, disagreement, and heterogeneous signal-to-noise ratios," Empirical Economics, Springer, vol. 53(1), pages 63-77, August.
    20. Lena Dräger & Michael J. Lamla, 2017. "Imperfect Information and Consumer Inflation Expectations: Evidence from Microdata," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 79(6), pages 933-968, December.

    More about this item

    Keywords

    Combination methods; Hypothesis testing; p value; Union of rejections;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C33 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Models with Panel Data; Spatio-temporal Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:amu:wpaper:2012-11. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Thomas Meal). General contact details of provider: http://www.american.edu/cas/economics/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.