A diffusion wavelets-based multiscale framework for inverse optimal control of stochastic systems

This work presents a multiscale framework to solve a class of inverse optimal control (IOC) problems in the context of robot motion planning and control in a complex environment. In order to handle complications resulting from a large decision space and complex environmental geometry, two key concepts are adopted: (a) a diffusion wavelet representation of the Markov chain for hierarchical abstraction of the state space; and (b) a desirability function-based representation of the Markov decision process (MDP) to efficiently calculate the optimal policy. An IOC problem constructed on a ‘abstract state’ is solved, which is much more tractable than using the original bases set; moreover, the solution can be obtained recursively in the ‘coarse to fine’ direction by utilizing the hierarchical structure of basis functions. The resulting multiscale plan is utilized to finally compute a continuous-time optimal control policy within a receding horizon implementation. Illustrative numerical experiments on a robot path control in a complex environment and on a quadrotor ball-catching task are presented to verify the proposed method.