IDEAS home Printed from https://ideas.repec.org/a/spr/ijsaem/v13y2022i1d10.1007_s13198-021-01540-x.html
   My bibliography  Save this article

Research on digital media animation control technology based on recurrent neural network using speech technology

Author

Listed:
  • Hui Wang

    (JiaoZuo University)

  • Ashutosh Sharma

    (Southern Federal University)

  • Mohammad Shabaz

    (Chandigarh University)

Abstract

A vivid and lifelike virtual speaker can attract the user's attention, and the construction of a lifelike virtual speaker not only requires a beautiful static appearance, but also has mouth movements, facial expressions and body movements that are truly synchronized with the voice. Virtual speaker refers to a technology in which a computer generates an animated facial image that can speak. In order to add special effects such as image editing and beautification in the broadcast screen. This paper proposes a voice-driven facial animation synthesis method based on deep BLSTM. A Neural Network BLSTM-RNN Using Audio-Visual Dual Modal Information Training of Speakers, uses the active appearance model to model the face image, and uses the AAM model parameters as Network output, to study the influence of network structure and input of different voice features on the effect of animation synthesis. The experimental results based on the LIPS2008 standard evaluation library show that the network effect with BLSTM layer is obviously better than that of forward network, and the three-layer model structure based on BLSTM—forward- BLSTM 256 node (BFB256) is the best. FBank, fundamental frequency and energy combination can further improve animation synthesis effect. The main aim of this paper is to study the method of speech-driven facial animation synthesis based on deep BLSTM-RNN, and tries the synthesis effect of different neural network structures and different speech features.

Suggested Citation

  • Hui Wang & Ashutosh Sharma & Mohammad Shabaz, 2022. "Research on digital media animation control technology based on recurrent neural network using speech technology," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 13(1), pages 564-575, March.
    • Handle: RePEc:spr:ijsaem:v:13:y:2022:i:1:d:10.1007_s13198-021-01540-x
    DOI: 10.1007/s13198-021-01540-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s13198-021-01540-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s13198-021-01540-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Courtney J Spoerer & Tim C Kietzmann & Johannes Mehrer & Ian Charest & Nikolaus Kriegeskorte, 2020. "Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision," PLOS Computational Biology, Public Library of Science, vol. 16(10), pages 1-27, October.
    2. Wee Chin Wong & Ewan Chee & Jiali Li & Xiaonan Wang, 2018. "Recurrent Neural Network-Based Model Predictive Control for Continuous Pharmaceutical Manufacturing," Mathematics, MDPI, vol. 6(11), pages 1-20, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kai-Chao Yao & Wei-Tzer Huang & Teng-Yu Chen & Cheng-Chun Wu & Wei-Sho Ho, 2022. "Establishing an Intelligent Emotion Analysis System for Long-Term Care Application Based on LabVIEW," Sustainability, MDPI, vol. 14(14), pages 1-18, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Allen Axelrod, 2021. "On the Obsolescence of Long-Run Rationality," RAIS Conference Proceedings 2021 0139, Research Association for Interdisciplinary Studies.
    2. Aleksey I. Shinkevich & Irina G. Ershova & Farida F. Galimulina, 2022. "Forecasting the Efficiency of Innovative Industrial Systems Based on Neural Networks," Mathematics, MDPI, vol. 11(1), pages 1-25, December.
    3. Monika Graumann & Caterina Ciuffi & Kshitij Dwivedi & Gemma Roig & Radoslaw M. Cichy, 2022. "The spatiotemporal neural dynamics of object location representations in the human brain," Nature Human Behaviour, Nature, vol. 6(6), pages 796-811, June.
    4. Tian Zhu & Wei Zhu, 2022. "Quantitative Trading through Random Perturbation Q-Network with Nonlinear Transaction Costs," Stats, MDPI, vol. 5(2), pages 1-15, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:ijsaem:v:13:y:2022:i:1:d:10.1007_s13198-021-01540-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.