A hybrid artificial bee colony algorithmic approach for classification using neural networks

Artificial neural networks are an integral component of most corporate and research functions across different platforms. However, depending upon the nature of the problem and quality of initialisation values, the usage of standard stochastic gradient descent always risks the possibility of getting trapped in local minima and saddle points for smaller neural networks in particular. One way to overcome this is by using algorithms with proven global search capabilities to train the network. This allows the neural net to reach the optimum values for weights regardless of the initialisation parameters used during training. Two algorithms are proposed based on modifications to the original artificial bee colony algorithm and their performances are analysed extensively on three benchmark datasets of increasing complexity. The first (NMABC), employs neural network appropriate initialisation and linear search space expansion. This is integrated into the second (LHABC), and incorporates stochastic gradient descent into the employed phase of the bees for faster convergence. It is found that the proposed algorithms consistently outperform standard approaches in all cases.