IDEAS home Printed from
   My bibliography  Save this paper

Post-simulation analysis of Monte Carlo experiments: interpreting Pesaran's (1974) study of non-nested hypothesis test statistics


  • Neil R. Ericsson


"Monte Carlo experimentation in econometrics helps 'solve' deterministic problems by simulating stochastic analogues in which the analytical unknowns are reformulated as parameters to be estimated." (Hendry (1980) With that in mind, Monte Carlo studies may be divided operationally into three phases: design, simulation, and post-simulation analysis. This paper provides a guide to the last of those three, post-simulation analysis, given the design and simulation of a Monte Carlo study, and uses Pesaran's (1974) study of statistics for testing non-nested hypotheses to illustrate the techniques described. A statistic is derived for testing for significant deviations between the asymptotic and (observed) finite sample properties. Further, that statistic provides the basis for analyzing discrepancies between the finite sample and asymptotic properties using response surfaces. The results for Pesaran's study indicate the value of asymptotic theory in interpreting finite sample properties and certain limitations for doing so. Finally, a method is proposed for adjusting the finite sample sizes of different test statistics so that comparisons of their power may be made. Extensions to other finite sample properties are indicated.

Suggested Citation

  • Neil R. Ericsson, 1986. "Post-simulation analysis of Monte Carlo experiments: interpreting Pesaran's (1974) study of non-nested hypothesis test statistics," International Finance Discussion Papers 276, Board of Governors of the Federal Reserve System (U.S.).
  • Handle: RePEc:fip:fedgif:276

    Download full text from publisher

    File URL:
    Download Restriction: no

    File URL:
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    1. Maasoumi, Esfandiar & Phillips, Peter C. B., 1982. "On the behavior of inconsistent instrumental variable estimators," Journal of Econometrics, Elsevier, vol. 19(2-3), pages 183-201, August.
    2. Sargan, J D, 1976. "Econometric Estimators and the Edgeworth Approximation," Econometrica, Econometric Society, vol. 44(3), pages 421-448, May.
    3. Neil R. Ericsson, 1983. "Asymptotic Properties of Instrumental Variables Statistics for Testing Non-Nested Hypotheses," Review of Economic Studies, Oxford University Press, vol. 50(2), pages 287-304.
    4. Godfrey, L. G. & Pesaran, M. H., 1983. "Tests of non-nested regression models: Small sample adjustments and Monte Carlo evidence," Journal of Econometrics, Elsevier, vol. 21(1), pages 133-154, January.
    5. Hendry, D F, 1973. "On Asymptotic Theory and Finite Sample Experiments," Economica, London School of Economics and Political Science, vol. 40(158), pages 210-217, May.
    6. Godfrey, Leslie G, 1978. "Testing against General Autoregressive and Moving Average Error Models When the Regressors Include Lagged Dependent Variables," Econometrica, Econometric Society, vol. 46(6), pages 1293-1301, November.
    7. Sargan, J D, 1980. "Some Approximations to the Distribution of Econometric Criteria Which are Asymptotically Distributed as Chi-Squared," Econometrica, Econometric Society, vol. 48(5), pages 1107-1138, July.
    8. Pesaran, M H, 1982. "Comparison of Local Power of Alternative Tests of Non-Nested Regression Models," Econometrica, Econometric Society, vol. 50(5), pages 1287-1305, September.
    9. Nicholls, D F & Pagan, A R, 1983. "Heteroscedasticity in Models with Lagged Dependent Variables," Econometrica, Econometric Society, vol. 51(4), pages 1233-1242, July.
    10. Hendry, David F. & Srba, Frank, 1980. "Autoreg: a computer program library for dynamic econometric models with autoregressive errors," Journal of Econometrics, Elsevier, vol. 12(1), pages 85-102, January.
    11. Grayham E. Mizon & David F. Hendry, 1980. "An Empirical Application and Monte Carlo Analysis of Tests of Dynamic Specification," Review of Economic Studies, Oxford University Press, vol. 47(1), pages 21-45.
    12. Hendry, David F., 1982. "A reply to Professors Maasoumi and Phillips," Journal of Econometrics, Elsevier, vol. 19(2-3), pages 203-213, August.
    13. Gwyn Aneuryn-Evans & Angus Deaton, 1980. "Testing Linear versus Logarithmic Regression Models," Review of Economic Studies, Oxford University Press, vol. 47(1), pages 275-291.
    14. Jarque, Carlos M. & Bera, Anil K., 1980. "Efficient tests for normality, homoscedasticity and serial independence of regression residuals," Economics Letters, Elsevier, vol. 6(3), pages 255-259.
    15. Robert F. Engle & David F. Hendry & David Trumble, 1985. "Small-Sample Properties of ARCH Estimators and Tests," Canadian Journal of Economics, Canadian Economics Association, vol. 18(1), pages 66-93, February.
    16. Sowey, Eric R., 1973. "A classified bibliography of Monte Carlo studies in econometrics," Journal of Econometrics, Elsevier, vol. 1(4), pages 377-395, December.
    17. Engle, Robert F, 1982. "Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation," Econometrica, Econometric Society, vol. 50(4), pages 987-1007, July.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Harvey, David I. & van Dijk, Dick, 2006. "Sample size, lag order and critical values of seasonal unit root tests," Computational Statistics & Data Analysis, Elsevier, vol. 50(10), pages 2734-2751, June.
    2. Neil R. Ericsson, 1987. "Monte Carlo methodology and the finite sample properties of statistics for testing nested and non-nested hypotheses," International Finance Discussion Papers 317, Board of Governors of the Federal Reserve System (U.S.).
    3. BHATTI, M.Ishaq & BODLA, Mahmud, A., 2008. "Empirical Power Comparison Of Non-Nested Tests For The Evm: Some Monte Carlo Evidence," International Journal of Applied Econometrics and Quantitative Studies, Euro-American Association of Economic Development, vol. 5(2).
    4. McAleer, Michael, 1995. "The significance of testing empirical non-nested models," Journal of Econometrics, Elsevier, vol. 67(1), pages 149-171, May.
    5. Lupi, Claudio, 2009. "Unit Root CADF Testing with R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 32(i02).

    More about this item


    Econometrics ; Statistics;


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedgif:276. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Franz Osorio). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.