IDEAS home Printed from https://ideas.repec.org/a/gam/jftint/v15y2022i1p14-d1016903.html
   My bibliography  Save this article

Human–Machine Interaction through Advanced Haptic Sensors: A Piezoelectric Sensory Glove with Edge Machine Learning for Gesture and Object Recognition

Author

Listed:
  • Roberto De Fazio

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
    Facultad de Ingeniería, Universidad Panamericana, Aguascalientes 20290, Mexico)

  • Vincenzo Mariano Mastronardi

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
    Center for Biomolecular Nanotechnologies, Italian Technology Institute IIT, 73010 Arnesano, Italy)

  • Matteo Petruzzi

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy)

  • Massimo De Vittorio

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
    Center for Biomolecular Nanotechnologies, Italian Technology Institute IIT, 73010 Arnesano, Italy)

  • Paolo Visconti

    (Department of Innovation Engineering, University of Salento, 73100 Lecce, Italy
    Center for Biomolecular Nanotechnologies, Italian Technology Institute IIT, 73010 Arnesano, Italy)

Abstract

Human–machine interaction (HMI) refers to systems enabling communication between machines and humans. Systems for human–machine interfaces have advanced significantly in terms of materials, device design, and production methods. Energy supply units, logic circuits, sensors, and data storage units must be flexible, stretchable, undetectable, biocompatible, and self-healing to act as human–machine interfaces. This paper discusses the technologies for providing different haptic feedback of different natures. Notably, the physiological mechanisms behind touch perception are reported, along with a classification of the main haptic interfaces. Afterward, a comprehensive overview of wearable haptic interfaces is presented, comparing them in terms of cost, the number of integrated actuators and sensors, their main haptic feedback typology, and their future application. Additionally, a review of sensing systems that use haptic feedback technologies—specifically, smart gloves—is given by going through their fundamental technological specifications and key design requirements. Furthermore, useful insights related to the design of the next-generation HMI devices are reported. Lastly, a novel smart glove based on thin and conformable AlN (aluminum nitride) piezoelectric sensors is demonstrated. Specifically, the device acquires and processes the signal from the piezo sensors to classify performed gestures through an onboard machine learning (ML) algorithm. Then, the design and testing of the electronic conditioning section of AlN-based sensors integrated into the smart glove are shown. Finally, the architecture of a wearable visual-tactile recognition system is presented, combining visual data acquired by a micro-camera mounted on the user’s glass with the haptic ones provided by the piezoelectric sensors.

Suggested Citation

  • Roberto De Fazio & Vincenzo Mariano Mastronardi & Matteo Petruzzi & Massimo De Vittorio & Paolo Visconti, 2022. "Human–Machine Interaction through Advanced Haptic Sensors: A Piezoelectric Sensory Glove with Edge Machine Learning for Gesture and Object Recognition," Future Internet, MDPI, vol. 15(1), pages 1-42, December.
  • Handle: RePEc:gam:jftint:v:15:y:2022:i:1:p:14-:d:1016903
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1999-5903/15/1/14/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1999-5903/15/1/14/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christian Janiesch & Patrick Zschech & Kai Heinrich, 2021. "Machine learning and deep learning," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 685-695, September.
    2. Roberto De Fazio & Massimo De Vittorio & Paolo Visconti, 2022. "A BLE-Connected Piezoresistive and Inertial Chest Band for Remote Monitoring of the Respiratory Activity by an Android Application: Hardware Design and Software Optimization," Future Internet, MDPI, vol. 14(6), pages 1-27, June.
    3. Chad E. Bouton & Ammar Shaikhouni & Nicholas V. Annetta & Marcia A. Bockbrader & David A. Friedenberg & Dylan M. Nielson & Gaurav Sharma & Per B. Sederberg & Bradley C. Glenn & W. Jerry Mysiw & Austin, 2016. "Restoring cortical control of functional movement in a human with quadriplegia," Nature, Nature, vol. 533(7602), pages 247-250, May.
    4. Roberto de Fazio & Donato Cafagna & Giorgio Marcuccio & Alessandro Minerba & Paolo Visconti, 2020. "A Multi-Source Harvesting System Applied to Sensor-Based Smart Garments for Monitoring Workers’ Bio-Physical Parameters in Harsh Environments," Energies, MDPI, vol. 13(9), pages 1-33, May.
    5. Vito Cacucciolo & Jun Shintake & Yu Kuwajima & Shingo Maeda & Dario Floreano & Herbert Shea, 2019. "Stretchable pumps for soft machines," Nature, Nature, vol. 572(7770), pages 516-519, August.
    6. Bernardo Calabrese & Ramiro Velázquez & Carolina Del-Valle-Soto & Roberto de Fazio & Nicola Ivan Giannoccaro & Paolo Visconti, 2020. "Solar-Powered Deep Learning-Based Recognition System of Daily Used Objects and Human Faces for Assistance of the Visually Impaired," Energies, MDPI, vol. 13(22), pages 1-30, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jen-Yu Lee & Tien-Thinh Nguyen & Hong-Giang Nguyen & Jen-Yao Lee, 2022. "Towards Predictive Crude Oil Purchase: A Case Study in the USA and Europe," Energies, MDPI, vol. 15(11), pages 1-15, May.
    2. Eduard Hartwich & Alexander Rieger & Johannes Sedlmeir & Dominik Jurek & Gilbert Fridgen, 2023. "Machine economies," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-13, December.
    3. Rui Ma & Jia Wang & Wei Zhao & Hongjie Guo & Dongnan Dai & Yuliang Yun & Li Li & Fengqi Hao & Jinqiang Bai & Dexin Ma, 2022. "Identification of Maize Seed Varieties Using MobileNetV2 with Improved Attention Mechanism CBAM," Agriculture, MDPI, vol. 13(1), pages 1-16, December.
    4. Dylan Norbert Gono & Herlina Napitupulu & Firdaniza, 2023. "Silver Price Forecasting Using Extreme Gradient Boosting (XGBoost) Method," Mathematics, MDPI, vol. 11(18), pages 1-15, September.
    5. Cheng Yang & Fuhao Sun & Yujie Zou & Zhipeng Lv & Liang Xue & Chao Jiang & Shuangyu Liu & Bochao Zhao & Haoyang Cui, 2024. "A Survey of Photovoltaic Panel Overlay and Fault Detection Methods," Energies, MDPI, vol. 17(4), pages 1-37, February.
    6. Shuai Sang & Lu Li, 2024. "A Novel Variant of LSTM Stock Prediction Method Incorporating Attention Mechanism," Mathematics, MDPI, vol. 12(7), pages 1-20, March.
    7. Vladimir Franki & Darin Majnarić & Alfredo Višković, 2023. "A Comprehensive Review of Artificial Intelligence (AI) Companies in the Power Sector," Energies, MDPI, vol. 16(3), pages 1-35, January.
    8. Joshua Holstein & Max Schemmer & Johannes Jakubik & Michael Vössing & Gerhard Satzger, 2023. "Sanitizing data for analysis: Designing systems for data understanding," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-18, December.
    9. Zhouheng Wang & Nanlin Shi & Yingchao Zhang & Ning Zheng & Haicheng Li & Yang Jiao & Jiahui Cheng & Yutong Wang & Xiaoqing Zhang & Ying Chen & Yihao Chen & Heling Wang & Tao Xie & Yijun Wang & Yinji M, 2023. "Conformal in-ear bioelectronics for visual and auditory brain-computer interfaces," Nature Communications, Nature, vol. 14(1), pages 1-11, December.
    10. Patrick Zschech, 2023. "Beyond descriptive taxonomies in data analytics: a systematic evaluation approach for data-driven method pipelines," Information Systems and e-Business Management, Springer, vol. 21(1), pages 193-227, March.
    11. Julius Peter Landwehr & Niklas Kühl & Jannis Walk & Mario Gnädig, 2022. "Design Knowledge for Deep-Learning-Enabled Image-Based Decision Support Systems," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(6), pages 707-728, December.
    12. Michael Weber & Martin Engert & Norman Schaffer & Jörg Weking & Helmut Krcmar, 2023. "Organizational Capabilities for AI Implementation—Coping with Inscrutability and Data Dependency in AI," Information Systems Frontiers, Springer, vol. 25(4), pages 1549-1569, August.
    13. Rashid Amin & Muzammal Majeed & Farrukh Shoukat Ali & Adeel Ahmed & Mudassar Hussain, 2022. "Reliability Awareness Multiple Path Installation in Software Defined Networking using Machine Learning Algorithm," International Journal of Innovations in Science & Technology, 50sea, vol. 4(5), pages 158-172, July.
    14. Irene Cappelli & Stefano Parrino & Alessandro Pozzebon & Alessio Salta, 2021. "Providing Energy Self-Sufficiency to LoRaWAN Nodes by Means of Thermoelectric Generators (TEGs)-Based Energy Harvesting," Energies, MDPI, vol. 14(21), pages 1-17, November.
    15. Kalliopi Kanaki & Michail Kalogiannakis & Emmanouil Poulakis & Panagiotis Politis, 2022. "Investigating the Association between Algorithmic Thinking and Performance in Environmental Study," Sustainability, MDPI, vol. 14(17), pages 1-16, August.
    16. Guorui Li & Tuck-Whye Wong & Benjamin Shih & Chunyu Guo & Luwen Wang & Jiaqi Liu & Tao Wang & Xiaobo Liu & Jiayao Yan & Baosheng Wu & Fajun Yu & Yunsai Chen & Yiming Liang & Yaoting Xue & Chengjun Wan, 2023. "Bioinspired soft robots for deep-sea exploration," Nature Communications, Nature, vol. 14(1), pages 1-10, December.
    17. Roberto De Fazio & Roberta Proto & Carolina Del-Valle-Soto & Ramiro Velázquez & Paolo Visconti, 2022. "New Wearable Technologies and Devices to Efficiently Scavenge Energy from the Human Body: State of the Art and Future Trends," Energies, MDPI, vol. 15(18), pages 1-37, September.
    18. Rafael Magdalena-Benedicto & Sonia Pérez-Díaz & Adrià Costa-Roig, 2023. "Challenges and Opportunities in Machine Learning for Geometry," Mathematics, MDPI, vol. 11(11), pages 1-24, June.
    19. Ruiz-Moreno, Sara & Gallego, Antonio J. & Sanchez, Adolfo J. & Camacho, Eduardo F., 2023. "A cascade neural network methodology for fault detection and diagnosis in solar thermal plants," Renewable Energy, Elsevier, vol. 211(C), pages 76-86.
    20. Bangfeng Wang & Yiwei Li & Mengfan Zhou & Yulong Han & Mingyu Zhang & Zhaolong Gao & Zetai Liu & Peng Chen & Wei Du & Xingcai Zhang & Xiaojun Feng & Bi-Feng Liu, 2023. "Smartphone-based platforms implementing microfluidic detection with image-based artificial intelligence," Nature Communications, Nature, vol. 14(1), pages 1-18, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jftint:v:15:y:2022:i:1:p:14-:d:1016903. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.