On the theory of flexible neural networks – Part I: a survey paper

Although flexible neural networks (FNNs) have been used more successfully than classical neural networks (CNNs) in many industrial applications, nothing is rigorously known about their properties. In fact they are not even well known to the systems and control community. In the first part of this paper, existing structures of and results on FNNs are surveyed. In the second part FNNs are examined in a theoretical framework. As a result, theoretical evidence is given for the superiority of FNNs over CNNs and further properties of the former are developed. More precisely, several fundamental properties of feedforward and recurrent FNNs are established. This includes the universal approximation capability, minimality, controllability, observability, and identifiability. In the broad sense, the results of this paper help that general use of FNNs in systems and control theory and applications be based on firm theoretical foundations. Theoretical analysis and synthesis of FNN-based systems thus become possible. The paper is concluded by a collection of topics for future work.