IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v13y2022i1d10.1038_s41467-022-32745-8.html
   My bibliography  Save this article

Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions

Author

Listed:
  • Zhongda Sun

    (National University of Singapore
    National University of Singapore
    Singapore Institute of Manufacturing Technology and National University of Singapore (SIMTech-NUS) Joint Lab on Large-area Flexible Hybrid Electronics, National University of Singapore
    National University of Singapore Suzhou Research Institute (NUSRI), Suzhou Industrial Park)

  • Minglu Zhu

    (National University of Singapore
    National University of Singapore
    National University of Singapore Suzhou Research Institute (NUSRI), Suzhou Industrial Park)

  • Xuechuan Shan

    (Singapore Institute of Manufacturing Technology and National University of Singapore (SIMTech-NUS) Joint Lab on Large-area Flexible Hybrid Electronics, National University of Singapore
    Technology and Research (A*STAR))

  • Chengkuo Lee

    (National University of Singapore
    National University of Singapore
    Singapore Institute of Manufacturing Technology and National University of Singapore (SIMTech-NUS) Joint Lab on Large-area Flexible Hybrid Electronics, National University of Singapore
    National University of Singapore Suzhou Research Institute (NUSRI), Suzhou Industrial Park)

Abstract

Advancements of virtual reality technology pave the way for developing wearable devices to enable somatosensory sensation, which can bring more comprehensive perception and feedback in the metaverse-based virtual society. Here, we propose augmented tactile-perception and haptic-feedback rings with multimodal sensing and feedback capabilities. This highly integrated ring consists of triboelectric and pyroelectric sensors for tactile and temperature perception, and vibrators and nichrome heaters for vibro- and thermo-haptic feedback. All these components integrated on the ring can be directly driven by a custom wireless platform of low power consumption for wearable/portable scenarios. With voltage integration processing, high-resolution continuous finger motion tracking is achieved via the triboelectric tactile sensor, which also contributes to superior performance in gesture/object recognition with artificial intelligence analysis. By fusing the multimodal sensing and feedback functions, an interactive metaverse platform with cross-space perception capability is successfully achieved, giving people a face-to-face like immersive virtual social experience.

Suggested Citation

  • Zhongda Sun & Minglu Zhu & Xuechuan Shan & Chengkuo Lee, 2022. "Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    • Handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-32745-8
    DOI: 10.1038/s41467-022-32745-8
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-022-32745-8
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1038/s41467-022-32745-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Qiongfeng Shi & Zixuan Zhang & Tianyiyi He & Zhongda Sun & Bingjie Wang & Yuqin Feng & Xuechuan Shan & Budiman Salam & Chengkuo Lee, 2020. "Deep learning enabled smart mats as a scalable floor monitoring system," Nature Communications, Nature, vol. 11(1), pages 1-11, December.
    2. Nozariasbmarz, Amin & Collins, Henry & Dsouza, Kelvin & Polash, Mobarak Hossain & Hosseini, Mahshid & Hyland, Melissa & Liu, Jie & Malhotra, Abhishek & Ortiz, Francisco Matos & Mohaddes, Farzad & Rame, 2020. "Review of wearable thermoelectric energy harvesting: From body temperature to electronic systems," Applied Energy, Elsevier, vol. 258(C).
    3. Chengyu Li & Di Liu & Chaoqun Xu & Ziming Wang & Sheng Shu & Zhuoran Sun & Wei Tang & Zhong Lin Wang, 2021. "Sensing of joint and spinal bending or stretching via a retractable and wearable badge reel," Nature Communications, Nature, vol. 12(1), pages 1-11, December.
    4. Xinge Yu & Zhaoqian Xie & Yang Yu & Jungyup Lee & Abraham Vazquez-Guardado & Haiwen Luan & Jasper Ruban & Xin Ning & Aadeel Akhtar & Dengfeng Li & Bowen Ji & Yiming Liu & Rujie Sun & Jingyue Cao & Qin, 2019. "Skin-integrated wireless haptic interfaces for virtual and augmented reality," Nature, Nature, vol. 575(7783), pages 473-479, November.
    5. Aldo Faisal, 2017. "Computer science: Visionary of virtual reality," Nature, Nature, vol. 551(7680), pages 298-299, November.
    6. Minglu Zhu & Zhongda Sun & Tao Chen & Chengkuo Lee, 2021. "Low cost exoskeleton manipulator using bidirectional triboelectric sensors enhanced multiple degree of freedom sensory system," Nature Communications, Nature, vol. 12(1), pages 1-16, December.
    7. Subramanian Sundaram & Petr Kellnhofer & Yunzhu Li & Jun-Yan Zhu & Antonio Torralba & Wojciech Matusik, 2019. "Learning the signatures of the human grasp using a scalable tactile glove," Nature, Nature, vol. 569(7758), pages 698-702, May.
    8. Kyun Kyu Kim & InHo Ha & Min Kim & Joonhwa Choi & Phillip Won & Sungho Jo & Seung Hwan Ko, 2020. "A deep-learned skin sensor decoding the epicentral human motions," Nature Communications, Nature, vol. 11(1), pages 1-8, December.
    9. Feng Wen & Zixuan Zhang & Tianyiyi He & Chengkuo Lee, 2021. "AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove," Nature Communications, Nature, vol. 12(1), pages 1-13, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ningning Bai & Yiheng Xue & Shuiqing Chen & Lin Shi & Junli Shi & Yuan Zhang & Xingyu Hou & Yu Cheng & Kaixi Huang & Weidong Wang & Jin Zhang & Yuan Liu & Chuan Fei Guo, 2023. "A robotic sensory system with high spatiotemporal resolution for texture recognition," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    2. Yiming Liu & Chun Ki Yiu & Zhao Zhao & Wooyoung Park & Rui Shi & Xingcan Huang & Yuyang Zeng & Kuan Wang & Tsz Hung Wong & Shengxin Jia & Jingkun Zhou & Zhan Gao & Ling Zhao & Kuanming Yao & Jian Li &, 2023. "Soft, miniaturized, wireless olfactory interface for virtual reality," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    3. Liuting Shan & Qizhen Chen & Rengjian Yu & Changsong Gao & Lujian Liu & Tailiang Guo & Huipeng Chen, 2023. "A sensory memory processing system with multi-wavelength synaptic-polychromatic light emission for multi-modal information recognition," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    4. Jiayue Zhang & Yikui Gao & Di Liu & Jing-Shan Zhao & Jie Wang, 2023. "Discharge domains regulation and dynamic processes of direct-current triboelectric nanogenerator," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    5. Haisheng Xia & Yuchong Zhang & Nona Rajabi & Farzaneh Taleb & Qunting Yang & Danica Kragic & Zhijun Li, 2024. "Shaping high-performance wearable robots for human motor and sensory reconstruction and enhancement," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    6. Chungryeol Lee & Changhyeon Lee & Seungmin Lee & Junhwan Choi & Hocheon Yoo & Sung Gap Im, 2023. "A reconfigurable binary/ternary logic conversion-in-memory based on drain-aligned floating-gate heterojunction transistors," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    7. Yuxiang Shi & Peng Yang & Rui Lei & Zhaoqi Liu & Xuanyi Dong & Xinglin Tao & Xiangcheng Chu & Zhong Lin Wang & Xiangyu Chen, 2023. "Eye tracking and eye expression decoding based on transparent, flexible and ultra-persistent electrostatic interface," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    8. Yiyue Luo & Chao Liu & Young Joong Lee & Joseph DelPreto & Kui Wu & Michael Foshey & Daniela Rus & Tomás Palacios & Yunzhu Li & Antonio Torralba & Wojciech Matusik, 2024. "Adaptive tactile interaction transfer via digitally embroidered smart gloves," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    9. Shaomei Lin & Weifeng Yang & Xubin Zhu & Yubin Lan & Kerui Li & Qinghong Zhang & Yaogang Li & Chengyi Hou & Hongzhi Wang, 2024. "Triboelectric micro-flexure-sensitive fiber electronics," Nature Communications, Nature, vol. 15(1), pages 1-10, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yijia Lu & Han Tian & Jia Cheng & Fei Zhu & Bin Liu & Shanshan Wei & Linhong Ji & Zhong Lin Wang, 2022. "Decoding lip language using triboelectric sensors with deep learning," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    2. Min Chen & Jingyu Ouyang & Aijia Jian & Jia Liu & Pan Li & Yixue Hao & Yuchen Gong & Jiayu Hu & Jing Zhou & Rui Wang & Jiaxi Wang & Long Hu & Yuwei Wang & Ju Ouyang & Jing Zhang & Chong Hou & Lei Wei , 2022. "Imperceptible, designable, and scalable braided electronic cord," Nature Communications, Nature, vol. 13(1), pages 1-10, December.
    3. Shun An & Hanrui Zhu & Chunzhi Guo & Benwei Fu & Chengyi Song & Peng Tao & Wen Shang & Tao Deng, 2022. "Noncontact human-machine interaction based on hand-responsive infrared structural color," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    4. Jinhong Park & Duhwan Seong & Yong Jun Park & Sang Hyeok Park & Hyunjin Jung & Yewon Kim & Hyoung Won Baac & Mikyung Shin & Seunghyun Lee & Minbaek Lee & Donghee Son, 2022. "Reversible electrical percolation in a stretchable and self-healable silver-gradient nanocomposite bilayer," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    5. Yiyue Luo & Chao Liu & Young Joong Lee & Joseph DelPreto & Kui Wu & Michael Foshey & Daniela Rus & Tomás Palacios & Yunzhu Li & Antonio Torralba & Wojciech Matusik, 2024. "Adaptive tactile interaction transfer via digitally embroidered smart gloves," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    6. Shijing Zhang & Yingxiang Liu & Jie Deng & Xiang Gao & Jing Li & Weiyi Wang & Mingxin Xun & Xuefeng Ma & Qingbing Chang & Junkao Liu & Weishan Chen & Jie Zhao, 2023. "Piezo robotic hand for motion manipulation from micro to macro," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    7. Wang, Xue & Wang, Hongchao & Su, Wenbing & Chen, Tingting & Tan, Chang & Madre, María A. & Sotelo, Andres & Wang, Chunlei, 2022. "U-type unileg thermoelectric module: A novel structure for high-temperature application with long lifespan," Energy, Elsevier, vol. 238(PB).
    8. Haisheng Xia & Yuchong Zhang & Nona Rajabi & Farzaneh Taleb & Qunting Yang & Danica Kragic & Zhijun Li, 2024. "Shaping high-performance wearable robots for human motor and sensory reconstruction and enhancement," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    9. Zhao, Lin-Chuan & Zhou, Teng & Chang, Si-Deng & Zou, Hong-Xiang & Gao, Qiu-Hua & Wu, Zhi-Yuan & Yan, Ge & Wei, Ke-Xiang & Yeatman, Eric M. & Meng, Guang & Zhang, Wen-Ming, 2024. "A disposable cup inspired smart floor for trajectory recognition and human-interactive sensing," Applied Energy, Elsevier, vol. 357(C).
    10. Won Bae Han & Gwan-Jin Ko & Kang-Gon Lee & Donghak Kim & Joong Hoon Lee & Seung Min Yang & Dong-Je Kim & Jeong-Woong Shin & Tae-Min Jang & Sungkeun Han & Honglei Zhou & Heeseok Kang & Jun Hyeon Lim & , 2023. "Ultra-stretchable and biodegradable elastomers for soft, transient electronics," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    11. Mai, Van-Phung & Yang, Ruey-Jen, 2020. "Boosting power generation from salinity gradient on high-density nanoporous membrane using thermal effect," Applied Energy, Elsevier, vol. 274(C).
    12. Liu, H.R. & Li, B.J. & Hua, L.J. & Wang, R.Z., 2022. "Designing thermoelectric self-cooling system for electronic devices: Experimental investigation and model validation," Energy, Elsevier, vol. 243(C).
    13. Pengwei Wang & Xiaohao Ma & Zhiqiang Lin & Fan Chen & Zijian Chen & Hong Hu & Hailong Xu & Xinyi Zhang & Yuqing Shi & Qiyao Huang & Yuanjing Lin & Zijian Zheng, 2024. "Well-defined in-textile photolithography towards permeable textile electronics," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    14. Yu, Yuedong & Zhu, Wei & Wang, Yaling & Zhu, Pengcheng & Peng, Kang & Deng, Yuan, 2020. "Towards high integration and power density: Zigzag-type thin-film thermoelectric generator assisted by rapid pulse laser patterning technique," Applied Energy, Elsevier, vol. 275(C).
    15. Pang, Dandan & Zhang, Aibing & Guo, Yage & Wu, Junfeng, 2023. "Energy harvesting analysis of wearable thermoelectric generators integrated with human skin," Energy, Elsevier, vol. 282(C).
    16. Guorui Li & Tuck-Whye Wong & Benjamin Shih & Chunyu Guo & Luwen Wang & Jiaqi Liu & Tao Wang & Xiaobo Liu & Jiayao Yan & Baosheng Wu & Fajun Yu & Yunsai Chen & Yiming Liang & Yaoting Xue & Chengjun Wan, 2023. "Bioinspired soft robots for deep-sea exploration," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    17. Roberto De Fazio & Roberta Proto & Carolina Del-Valle-Soto & Ramiro Velázquez & Paolo Visconti, 2022. "New Wearable Technologies and Devices to Efficiently Scavenge Energy from the Human Body: State of the Art and Future Trends," Energies, MDPI, vol. 15(18), pages 1-37, September.
    18. Rui Chen & Tao Luo & Jincheng Wang & Renpeng Wang & Chen Zhang & Yu Xie & Lifeng Qin & Haimin Yao & Wei Zhou, 2023. "Nonlinearity synergy: An elegant strategy for realizing high-sensitivity and wide-linear-range pressure sensing," Nature Communications, Nature, vol. 14(1), pages 1-9, December.
    19. Taemin Kim & Yejee Shin & Kyowon Kang & Kiho Kim & Gwanho Kim & Yunsu Byeon & Hwayeon Kim & Yuyan Gao & Jeong Ryong Lee & Geonhui Son & Taeseong Kim & Yohan Jun & Jihyun Kim & Jinyoung Lee & Seyun Um , 2022. "Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    20. Sargolzaeiaval, Yasaman & Ramesh, Viswanath Padmanabhan & Ozturk, Mehmet C., 2022. "A comprehensive analytical model for thermoelectric body heat harvesting incorporating the impact of human metabolism and physical activity," Applied Energy, Elsevier, vol. 324(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:13:y:2022:i:1:d:10.1038_s41467-022-32745-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.