Application of machine learning to short-term equity return prediction

Cooper showed how a filter method could be used to predict equity returns for the next week by using information about returns and volume for the two previous weeks. Cooper's method may be regarded as a crude method of Machine Learning. Over the last 20 years Machine Learning has been successfully applied to the modeling of large data sets, often containing a lot of noise, in many different fields. When applying the technique it is important to fit it to the specific problem under consideration. We have designed and applied to Cooper's problem a practical new method of Machine Learning, appropriate to the problem, that is based on a modification of the well-known kernel regression method. We call it the Prototype Kernel Regression method (PKR). In both the period 1978-1993 studied by Cooper, and the period 1994-2004, the PKR method leads to a clear profit improvement compared to Cooper's approach. In all of 48 different cases studied, the period pre-cost average return is larger for the PKR method than Cooper's method, on average 37% higher, and that margin would increase as costs were taken into account. Our method aims to minimize the danger of data snooping, and it could plausibly have been applied in 1994 or earlier. There may be a lesson here for proponents of the Efficient Market Hypothesis in the form that states that profitable prediction of equity returns is impossible except by chance. It is not enough for them to show that the profits from an anomaly-based trading scheme disappear after costs. The proponents should also consider what would have been plausible applications of more sophisticated Machine Learning techniques before dismissing evidence against the EMH.