Testing for regime shifts in short-sample macroeconomic data: a survey of some Monte Carlo evidence

The fundamental relevance of econometrics to applied policy analysis lies in the ability to use inference drawn from past events as a basis for projecting the future impact of policy changes. Robert Lucas (1976), however, has argued that in situations where agents act rationally, then their behaviour will necessarily change systematically with changes in the policy regime. Therefore any inference on economic behaviour based on models which do not explicitly model the change in expectations (ie models which condition on fixed expectations) will be prone to systematic predictive error. The direct implication of this critique is that inference is sought on the structural or "deep" parameters of the economic model, then the process of expectations requires to be estimated directly. However, econometrics, like all branches of economics, is based on the simplification, and we generally seek to reduce a complex data generation process to a small set of conditional models. All econometric models are conditional models derived from the simplification of the joint distribution. The fundamental issue in model design is whether the chosen simplification ensures that the conditional model satisfies the exogeneity conditions required to sustain inference and policy evaluation. Three cases are typically identified: weak exogeneity which is required for valid inference within the sample period; strong exogeneity, which is required for forecasting; and superexogeneity which is a necessary condition for policy evaluation [Engle, Hendry and Richard (1982)]. Consequently, whether the Lucas critique is relevant depends on.whether the super exogeneity of the variables for the parameters of the conditional model are accepted. Exogeneity is thus an empirical issue, depending on the nature of the specific data generation process, and the nature of the regime changes being confronted. There is an extensive literature on defining and testing for weak, strong, and superexogeneity [see, for example, Hendry (1979), Hendry and Mizon (1980), Engle et al (1982), Kiviet (1985) Favero and Hendry (1990) and Engle and Hendry (1993)]. While much of the literature concerns the theoretical issues associated with testing for exogeneity, the applied example tend to concentrate on UK macroeconomic relationships. One strong result emerging from this work, however, is the weakness of statistical tests used to examine invariance questions. In particular these tests generally show low power in rejecting the false null, in other words accepting that variables are invariant more often than they should. In the light of this. an issue that is of particular concern to econometricians working on African macroeconomic data concerns the question of super-exogeneity and tests for invariance of structural parameters in the face of the specific regime shifts brought about through the programmes of liberalization which have been undertaken in most sub-Saharan economies since the mid 1980s. Two specific issues confront the researcher, the first being whether we can identify the effects of the regime shifts given the data and sample constraints faced, and secondly, what implications are raised for the econometric modelling of macroeconomic relationships if invariance does not seem to exist. This paper is designed to provide a brief survey of the issues which arise in trying to answer the first of these questions, namely given the concerns about low power, whether we can identify regime shifts in African macroeconomic data.