Design of multi-modal feedback channel of human–robot cognitive interface for teleoperation in manufacturing

Teleoperation, which is a specific mode of human–robot collaboration enabling a human operator to provide instructions and monitor the actions of the robot remotely, has proved beneficial for application to hazardous and unstructured manufacturing environments. Despite the design of a command channel from human operators to robots, most existing studies on teleoperation fail to focus on the design of the feedback channel from the robot to the human operator, which plays a crucial role in reducing the cognitive load, particularly in precise and concentrated manufacturing tasks. This paper focuses on designing a feedback channel for the cognitive interface between a human operator and a robot considering human cognition. Current studies on human–robot cognitive interfaces in robot teleoperation are extensively surveyed. Further, the modalities of human cognition that foster understanding and transparency during teleoperation are identified. In addition, the human–robot cognitive interface, which utilizes the proposed multi-modal feedback channel, is developed on a teleoperated robotic grasping system as a case study. Finally, a series of experiments based on different modal feedback channels are conducted to demonstrate the effectiveness of enhancing the performance of the teleoperated grasping of fragile products and reducing the cognitive load via the objective aspects of experimental results and the subjective aspects of operator feedback.