Design of reduced-order and pinning controllers for probabilistic Boolean networks using reinforcement learning

In this paper, we study a stabilization method for probabilistic Boolean networks (PBNs) using Q-learning, which is one of the typical methods in reinforcement learning. A PBN is a class of discrete-time stochastic logical systems in which update functions are randomly chosen from the set of the candidates of Boolean functions. In the existing methods using reinforcement learning, a design method of structured controllers has not been studied. In this paper, we propose reward design methods to derive reduced-order controllers and pinning controllers. The key idea is to adjust the structure of controllers by the reward for the structure of the Q-table. The advantage of the proposed method is that implementation is easy, because the proposed method can be embedded in the existing Q-learning-based stabilization algorithm. In design of pinning controllers, we can calculate not only controllers but also pinning nodes in which the control input is assigned. We demonstrate through numerical examples that the controller obtained from our proposed method is simpler than that from the existing method.