IDEAS home Printed from https://ideas.repec.org/a/spr/joinma/v36y2025i6d10.1007_s10845-024-02451-x.html

Design of multi-modal feedback channel of human–robot cognitive interface for teleoperation in manufacturing

Author

Listed:
  • Chen Zheng

    (Northwestern Polytechnical University
    CSSC Huangpu Wenchong Shipbuilding Company Limited)

  • Kangning Wang

    (Northwestern Polytechnical University)

  • Shiqi Gao

    (Northwestern Polytechnical University)

  • Yang Yu

    (Northwestern Polytechnical University)

  • Zhanxi Wang

    (Northwestern Polytechnical University)

  • Yunlong Tang

    (Monash University
    Monash University)

Abstract

Teleoperation, which is a specific mode of human–robot collaboration enabling a human operator to provide instructions and monitor the actions of the robot remotely, has proved beneficial for application to hazardous and unstructured manufacturing environments. Despite the design of a command channel from human operators to robots, most existing studies on teleoperation fail to focus on the design of the feedback channel from the robot to the human operator, which plays a crucial role in reducing the cognitive load, particularly in precise and concentrated manufacturing tasks. This paper focuses on designing a feedback channel for the cognitive interface between a human operator and a robot considering human cognition. Current studies on human–robot cognitive interfaces in robot teleoperation are extensively surveyed. Further, the modalities of human cognition that foster understanding and transparency during teleoperation are identified. In addition, the human–robot cognitive interface, which utilizes the proposed multi-modal feedback channel, is developed on a teleoperated robotic grasping system as a case study. Finally, a series of experiments based on different modal feedback channels are conducted to demonstrate the effectiveness of enhancing the performance of the teleoperated grasping of fragile products and reducing the cognitive load via the objective aspects of experimental results and the subjective aspects of operator feedback.

Suggested Citation

  • Chen Zheng & Kangning Wang & Shiqi Gao & Yang Yu & Zhanxi Wang & Yunlong Tang, 2025. "Design of multi-modal feedback channel of human–robot cognitive interface for teleoperation in manufacturing," Journal of Intelligent Manufacturing, Springer, vol. 36(6), pages 4283-4303, August.
    • Handle: RePEc:spr:joinma:v:36:y:2025:i:6:d:10.1007_s10845-024-02451-x
    DOI: 10.1007/s10845-024-02451-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10845-024-02451-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10845-024-02451-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Xiaoqian Huang & Mohamad Halwani & Rajkumar Muthusamy & Abdulla Ayyad & Dewald Swart & Lakmal Seneviratne & Dongming Gan & Yahya Zweiri, 2022. "Real-time grasping strategies using event camera," Journal of Intelligent Manufacturing, Springer, vol. 33(2), pages 593-615, February.
    2. Chen Zhao & Shichang Du & Jun Lv & Yafei Deng & Guilong Li, 2023. "A novel parallel classification network for classifying three-dimensional surface with point cloud data," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 515-527, February.
    3. Thanh An Nguyen & Yong Zeng, 2017. "Effects of stress and effort on self-rated reports in experimental study of design activities," Journal of Intelligent Manufacturing, Springer, vol. 28(7), pages 1609-1622, October.
    4. Hien Nguyen Ngoc & Ganix Lasa & Ion Iriarte, 2022. "Human-centred design in industry 4.0: case study review and opportunities for future research," Journal of Intelligent Manufacturing, Springer, vol. 33(1), pages 35-76, January.
    5. Xiang T. R. Kong & Hao Luo & George Q. Huang & Xuan Yang, 2019. "Industrial wearable system: the human-centric empowering technology in Industry 4.0," Journal of Intelligent Manufacturing, Springer, vol. 30(8), pages 2853-2869, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wei Fang & Tienong Zhang & Lixi Chen & Hao Hu, 2025. "A survey on HoloLens AR in support of human-centric intelligent manufacturing," Journal of Intelligent Manufacturing, Springer, vol. 36(1), pages 35-59, January.
    2. Chiara Cimini & David Romero & Roberto Pinto & Sergio Cavalieri, 2023. "Task Classification Framework and Job-Task Analysis Method for Understanding the Impact of Smart and Digital Technologies on the Operators 4.0 Job Profiles," Sustainability, MDPI, vol. 15(5), pages 1-28, February.
    3. Paulo Sergio Gonçalves de Oliveira & Luciano Ferreira da Silva & Pedro Teixeira de Araujo & Guilherme Fernandes Gomes Reis & Marco Antônios Soares Gomes Otero, 2025. "Proposal of a Classification Method for Brazilian Automotive Companies Using the Principal Components Analysis," SN Operations Research Forum, Springer, vol. 6(4), pages 1-26, December.
    4. Xiaoheng Jiang & Jian Feng & Feng Yan & Yang Lu & Quanhai Fa & Wenjie Zhang & Mingliang Xu, 2025. "Foreground–background separation transformer for weakly supervised surface defect detection," Journal of Intelligent Manufacturing, Springer, vol. 36(6), pages 4217-4232, August.
    5. Ibrahim Yitmen & Amjad Almusaed & Sepehr Alizadehsalehi, 2023. "Investigating the Causal Relationships among Enablers of the Construction 5.0 Paradigm: Integration of Operator 5.0 and Society 5.0 with Human-Centricity, Sustainability, and Resilience," Sustainability, MDPI, vol. 15(11), pages 1-25, June.
    6. Neeraj Gupta & Saurabh Gupta & Mahdi Khosravy & Nilanjan Dey & Nisheeth Joshi & Rubén González Crespo & Nilesh Patel, 2021. "RETRACTED ARTICLE: Economic IoT strategy: the future technology for health monitoring and diagnostic of agriculture vehicles," Journal of Intelligent Manufacturing, Springer, vol. 32(4), pages 1117-1128, April.
    7. Lorenzo Vianello & Serena Ivaldi & Alexis Aubry & Luka Peternel, 2024. "The effects of role transitions and adaptation in human–cobot collaboration," Journal of Intelligent Manufacturing, Springer, vol. 35(5), pages 2005-2019, June.
    8. Massimiliano Pirani & Alessandro Carbonari & Alessandro Cucchiarelli & Alberto Giretti & Luca Spalazzi, 2025. "The Meta Holonic Management Tree: review, steps, and roadmap to industrial Cybernetics 5.0," Journal of Intelligent Manufacturing, Springer, vol. 36(8), pages 5285-5326, December.
    9. Marina Crnjac Zizic & Marko Mladineo & Nikola Gjeldum & Luka Celent, 2022. "From Industry 4.0 towards Industry 5.0: A Review and Analysis of Paradigm Shift for the People, Organization and Technology," Energies, MDPI, vol. 15(14), pages 1-20, July.
    10. Zhijie Yuan & Binjie Xin & Jing Zhang & Yingqi Xu, 2025. "Three-dimensional fabric smoothness evaluation using point cloud data for enhanced quality control," Journal of Intelligent Manufacturing, Springer, vol. 36(5), pages 3327-3343, June.
    11. Hien Nguyen Ngoc & Ganix Lasa & Ion Iriarte, 2022. "Human-centred design in industry 4.0: case study review and opportunities for future research," Journal of Intelligent Manufacturing, Springer, vol. 33(1), pages 35-76, January.
    12. Liang Hou & Roger J. Jiao, 2020. "Data-informed inverse design by product usage information: a review, framework and outlook," Journal of Intelligent Manufacturing, Springer, vol. 31(3), pages 529-552, March.
    13. Mingxing Li & Ray Y. Zhong & Ting Qu & George Q. Huang, 2022. "Spatial–temporal out-of-order execution for advanced planning and scheduling in cyber-physical factories," Journal of Intelligent Manufacturing, Springer, vol. 33(5), pages 1355-1372, June.
    14. Francesca Serravalle & Milena Viassone & Giacomo Chiappa, 2022. "Sensory disclosure in an augmented environment: memory of touch and willingness to buy," Italian Journal of Marketing, Springer, vol. 2022(4), pages 401-417, December.
    15. Zichen Zhan & Congcong Han & Lai Zou & Jianmin Dong & Shengbo Yan & Wenxi Wang, 2026. "A new characterization method for rapid prediction of recrystallization damage in single crystal superalloys considering plastic deformation," Journal of Intelligent Manufacturing, Springer, vol. 37(3), pages 1145-1161, March.
    16. Simon Micheler & Yee Mey Goh & Niels Lohse, 2021. "A transformation of human operation approach to inform system design for automation," Journal of Intelligent Manufacturing, Springer, vol. 32(1), pages 201-220, January.
    17. Mohammed Salah & Abdulla Ayyad & Mohammed Ramadan & Yusra Abdulrahman & Dewald Swart & Abdelqader Abusafieh & Lakmal Seneviratne & Yahya Zweiri, 2024. "High speed neuromorphic vision-based inspection of countersinks in automated manufacturing processes," Journal of Intelligent Manufacturing, Springer, vol. 35(7), pages 3067-3081, October.
    18. Md. Al-Amin & Ruwen Qin & Md Moniruzzaman & Zhaozheng Yin & Wenjin Tao & Ming C. Leu, 2023. "An individualized system of skeletal data-based CNN classifiers for action recognition in manufacturing assembly," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 633-649, February.
    19. Wei Qin & Qing Hu & Zilong Zhuang & Haozhe Huang & Xiaodan Zhu & Lin Han, 2023. "IPPE-PCR: a novel 6D pose estimation method based on point cloud repair for texture-less and occluded industrial parts," Journal of Intelligent Manufacturing, Springer, vol. 34(6), pages 2797-2807, August.
    20. Min Shi & Jingzhao Hou & Zhaoxin Li & Dengming Zhu, 2025. "NG-Net: No-Grasp annotation grasp detection network for stacked scenes," Journal of Intelligent Manufacturing, Springer, vol. 36(2), pages 1477-1490, February.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joinma:v:36:y:2025:i:6:d:10.1007_s10845-024-02451-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.