IDEAS home Printed from https://ideas.repec.org/p/fip/fedgif/276.html
   My bibliography  Save this paper

Post-simulation analysis of Monte Carlo experiments: interpreting Pesaran's (1974) study of non-nested hypothesis test statistics

Author

Abstract

\"Monte Carlo experimentation in econometrics helps 'solve' deterministic problems by simulating stochastic analogues in which the analytical unknowns are reformulated as parameters to be estimated.\" (Hendry (1980) With that in mind, Monte Carlo studies may be divided operationally into three phases: design, simulation, and post-simulation analysis. This paper provides a guide to the last of those three, post-simulation analysis, given the design and simulation of a Monte Carlo study, and uses Pesaran's (1974) study of statistics for testing non-nested hypotheses to illustrate the techniques described. A statistic is derived for testing for significant deviations between the asymptotic and (observed) finite sample properties. Further, that statistic provides the basis for analyzing discrepancies between the finite sample and asymptotic properties using response surfaces. The results for Pesaran's study indicate the value of asymptotic theory in interpreting finite sample properties and certain limitations for doing so. Finally, a method is proposed for adjusting the finite sample sizes of different test statistics so that comparisons of their power may be made. Extensions to other finite sample properties are indicated.

Suggested Citation

  • Neil R. Ericsson, 1986. "Post-simulation analysis of Monte Carlo experiments: interpreting Pesaran's (1974) study of non-nested hypothesis test statistics," International Finance Discussion Papers 276, Board of Governors of the Federal Reserve System (U.S.).
    • Handle: RePEc:fip:fedgif:276
    as

    Download full text from publisher

    File URL: http://www.federalreserve.gov/pubs/ifdp/1986/276/default.htm
    Download Restriction: no
    File URL: http://www.federalreserve.gov/pubs/ifdp/1986/276/ifdp276.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Maasoumi, Esfandiar & Phillips, Peter C. B., 1982. "On the behavior of inconsistent instrumental variable estimators," Journal of Econometrics, Elsevier, vol. 19(2-3), pages 183-201, August.
    2. Sargan, J D, 1976. "Econometric Estimators and the Edgeworth Approximation," Econometrica, Econometric Society, vol. 44(3), pages 421-448, May.
    3. Neil R. Ericsson, 1986. "Post-simulation Analysis of Monte Carlo Experiments: Interpreting Pesaran's (1974) Study of Non-nested Hypothesis Test Statistics," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 691-707.
    4. Neil R. Ericsson, 1983. "Asymptotic Properties of Instrumental Variables Statistics for Testing Non-Nested Hypotheses," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 50(2), pages 287-304.
    5. Farebrother, R W, 1980. "The Durbin-Watson Test for Serial Correlation When There Is No Intercept in the Regression," Econometrica, Econometric Society, vol. 48(6), pages 1553-1563, September.
    6. Mizon, Grayham E & Richard, Jean-Francois, 1986. "The Encompassing Principle and Its Application to Testing Non-nested Hypotheses," Econometrica, Econometric Society, vol. 54(3), pages 657-678, May.
    7. Sargan, J D, 1980. "Some Approximations to the Distribution of Econometric Criteria Which are Asymptotically Distributed as Chi-Squared," Econometrica, Econometric Society, vol. 48(5), pages 1107-1138, July.
    8. Grayham E. Mizon & David F. Hendry, 1980. "An Empirical Application and Monte Carlo Analysis of Tests of Dynamic Specification," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 21-45.
    9. Hendry, David F., 1982. "A reply to Professors Maasoumi and Phillips," Journal of Econometrics, Elsevier, vol. 19(2-3), pages 203-213, August.
    10. Jarque, Carlos M. & Bera, Anil K., 1980. "Efficient tests for normality, homoscedasticity and serial independence of regression residuals," Economics Letters, Elsevier, vol. 6(3), pages 255-259.
    11. Robert F. Engle & David F. Hendry & David Trumble, 1985. "Small-Sample Properties of ARCH Estimators and Tests," Canadian Journal of Economics, Canadian Economics Association, vol. 18(1), pages 66-93, February.
    12. James G. MacKinnon, 1983. "Model Specification Tests Against Non-Nested Alternatives," Working Paper 573, Economics Department, Queen's University.
    13. Engle, Robert F, 1982. "Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation," Econometrica, Econometric Society, vol. 50(4), pages 987-1007, July.
    14. Phillips, Peter C B, 1977. "Approximations to Some Finite Sample Distributions Associated with a First-Order Stochastic Difference Equation," Econometrica, Econometric Society, vol. 45(2), pages 463-485, March.
    15. Campos, Julia, 1986. "Finite-sample properties of the instrumental-variables estimator for dynamic simultaneous-equation subsystems with ARMA disturbances," Journal of Econometrics, Elsevier, vol. 32(3), pages 333-366, August.
    16. Godfrey, L. G. & Pesaran, M. H., 1983. "Tests of non-nested regression models: Small sample adjustments and Monte Carlo evidence," Journal of Econometrics, Elsevier, vol. 21(1), pages 133-154, January.
    17. repec:bla:econom:v:40:y:1973:i:158:p:210-17 is not listed on IDEAS
    18. Godfrey, Leslie G, 1978. "Testing against General Autoregressive and Moving Average Error Models When the Regressors Include Lagged Dependent Variables," Econometrica, Econometric Society, vol. 46(6), pages 1293-1301, November.
    19. Pesaran, M H, 1982. "Comparison of Local Power of Alternative Tests of Non-Nested Regression Models," Econometrica, Econometric Society, vol. 50(5), pages 1287-1305, September.
    20. Nicholls, D F & Pagan, A R, 1983. "Heteroscedasticity in Models with Lagged Dependent Variables," Econometrica, Econometric Society, vol. 51(4), pages 1233-1242, July.
    21. Hendry, David F. & Srba, Frank, 1980. "Autoreg: a computer program library for dynamic econometric models with autoregressive errors," Journal of Econometrics, Elsevier, vol. 12(1), pages 85-102, January.
    22. Gwyn Aneuryn-Evans & Angus Deaton, 1980. "Testing Linear versus Logarithmic Regression Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 275-291.
    23. Sowey, Eric R., 1973. "A classified bibliography of Monte Carlo studies in econometrics," Journal of Econometrics, Elsevier, vol. 1(4), pages 377-395, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Harvey, David I. & van Dijk, Dick, 2006. "Sample size, lag order and critical values of seasonal unit root tests," Computational Statistics & Data Analysis, Elsevier, vol. 50(10), pages 2734-2751, June.
    2. Neil R. Ericsson, 1986. "Post-simulation Analysis of Monte Carlo Experiments: Interpreting Pesaran's (1974) Study of Non-nested Hypothesis Test Statistics," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 691-707.
    3. de Brouwer, Gordon & Ericsson, Neil R, 1998. "Modeling Inflation in Australia," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(4), pages 433-449, October.
    4. repec:jss:jstsof:32:i02 is not listed on IDEAS
    5. Levine, Ross, 1989. "The pricing of forward exchange rates," Journal of International Money and Finance, Elsevier, vol. 8(2), pages 163-179, June.
    6. Neil R. Ericsson, 1987. "Monte Carlo methodology and the finite sample properties of statistics for testing nested and non-nested hypotheses," International Finance Discussion Papers 317, Board of Governors of the Federal Reserve System (U.S.).
    7. BHATTI, M.Ishaq & BODLA, Mahmud, A., 2008. "Empirical Power Comparison Of Non-Nested Tests For The Evm: Some Monte Carlo Evidence," International Journal of Applied Econometrics and Quantitative Studies, Euro-American Association of Economic Development, vol. 5(2).
    8. McAleer, Michael, 1995. "The significance of testing empirical non-nested models," Journal of Econometrics, Elsevier, vol. 67(1), pages 149-171, May.
    9. Lupi, Claudio, 2009. "Unit Root CADF Testing with R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 32(i02).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Neil R. Ericsson, 1987. "Monte Carlo methodology and the finite sample properties of statistics for testing nested and non-nested hypotheses," International Finance Discussion Papers 317, Board of Governors of the Federal Reserve System (U.S.).
    2. McAleer, Michael, 1995. "The significance of testing empirical non-nested models," Journal of Econometrics, Elsevier, vol. 67(1), pages 149-171, May.
    3. de Brouwer, Gordon & Ericsson, Neil R, 1998. "Modeling Inflation in Australia," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(4), pages 433-449, October.
    4. Luc Anselin, 1988. "Model Validation in Spatial Econometrics: A Review and Evaluation of Alternative Approaches," International Regional Science Review, , vol. 11(3), pages 279-316, December.
    5. Ericsson, Neil R., 1992. "Parameter constancy, mean square forecast errors, and measuring forecast performance: An exposition, extensions, and illustration," Journal of Policy Modeling, Elsevier, vol. 14(4), pages 465-495, August.
    6. Adrian C. Darnell, 1994. "A Dictionary Of Econometrics," Books, Edward Elgar Publishing, number 118.
    7. David F. Hendry & Peter C.B. Phillips, 2017. "John Denis Sargan at the London School of Economics," Cowles Foundation Discussion Papers 2082, Cowles Foundation for Research in Economics, Yale University.
    8. Hendry, David F. & Ericsson, Neil R., 1991. "Modeling the demand for narrow money in the United Kingdom and the United States," European Economic Review, Elsevier, vol. 35(4), pages 833-881, May.
    9. MacKinnon, James G, 1992. "Model Specification Tests and Artificial Regressions," Journal of Economic Literature, American Economic Association, vol. 30(1), pages 102-146, March.
    10. Yock Y. Chong & David F. Hendry, 1986. "Econometric Evaluation of Linear Macro-Economic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 671-690.
    11. Mizon, Grayham E & Richard, Jean-Francois, 1986. "The Encompassing Principle and Its Application to Testing Non-nested Hypotheses," Econometrica, Econometric Society, vol. 54(3), pages 657-678, May.
    12. Ericsson, Neil R. & Campos, Julia & Tran, Hong-Anh, 1990. "Pc-Give and David Hendry'S Econometric Methodology," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 10(1), April.
    13. Hendry, David F., 1984. "Monte carlo experimentation in econometrics," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 16, pages 937-976, Elsevier.
    14. Neil R. Ericsson & David F. Hendry, 1985. "Conditional econometric modelling : an application to new house prices in the United Kingdom," International Finance Discussion Papers 254, Board of Governors of the Federal Reserve System (U.S.).
    15. David F. Hendry, 2013. "Econometric Modelling: The ‘Consumption Function’ In Retrospect," Scottish Journal of Political Economy, Scottish Economic Society, vol. 60(5), pages 495-522, November.
    16. Maozu Lu & Grayham E. Mizon & Chiara Monfardini, 2008. "Simulation Encompassing: Testing Non‐nested Hypotheses," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 70(s1), pages 781-806, December.
    17. Charles G. Renfro, 2009. "The Practice of Econometric Theory," Advanced Studies in Theoretical and Applied Econometrics, Springer, number 978-3-540-75571-5.
    18. Sunil Sharma & Neil R. Ericsson, 1998. "Broad money demand and financial liberalization in Greece," Empirical Economics, Springer, vol. 23(3), pages 417-436.
    19. Hendry, David F & Ericsson, Neil R, 1991. "An Econometric Analysis of U.K. Money Demand in 'Monetary Trends in the United States and the United Kingdom' by Milton Friedman and Anna Schwartz," American Economic Review, American Economic Association, vol. 81(1), pages 8-38, March.
    20. McAleer, Michael, 1994. "Sherlock Holmes and the Search for Truth: A Diagnostic Tale," Journal of Economic Surveys, Wiley Blackwell, vol. 8(4), pages 317-370, December.

    More about this item

    Keywords

    Econometrics; Statistics;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedgif:276. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ryan Wolfslayer ; Keisha Fournillier (email available below). General contact details of provider: https://edirc.repec.org/data/frbgvus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.