IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i6p3569-d773961.html
   My bibliography  Save this article

Design and Implementation of Attention Depression Detection Model Based on Multimodal Analysis

Author

Listed:
  • Junhee Park

    (Department of Computer Science and Engineering, Hoseo University, Asan 31499, Korea)

  • Nammee Moon

    (Department of Computer Science and Engineering, Hoseo University, Asan 31499, Korea)

Abstract

Depression is becoming a social problem as the number of sufferers steadily increases. In this regard, this paper proposes a multimodal analysis-based attention depression detection model that simultaneously uses voice and text data obtained from users. The proposed models consist of Bidirectional Encoders from Transformers-Convolutional Neural Network (BERT-CNN) for natural language analysis, CNN-Bidirectional Long Short-Term Memory (CNN-BiLSTM) for voice signal processing, and multimodal analysis and fusion models for depression detection. The experiments in this paper are conducted using the DAIC-WOZ dataset, a clinical interview designed to support psychological distress states such as anxiety and post-traumatic stress. The voice data were set to 4 seconds in length and the number of mel filters was set to 128 in the preprocessing process. For text data, we used the subject text data of the interview and derived the embedding vector using a transformers tokenizer. Based on each data set, the BERT-CNN and CNN-BiLSTM proposed in this paper were applied and combined to classify depression. Through experiments, the accuracy and loss degree were compared for the cases of using multimodal data and using single data, and it was confirmed that the existing low accuracy was improved.

Suggested Citation

  • Junhee Park & Nammee Moon, 2022. "Design and Implementation of Attention Depression Detection Model Based on Multimodal Analysis," Sustainability, MDPI, vol. 14(6), pages 1-15, March.
  • Handle: RePEc:gam:jsusta:v:14:y:2022:i:6:p:3569-:d:773961
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/6/3569/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/14/6/3569/
    Download Restriction: no
    ---><---

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:6:p:3569-:d:773961. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.