IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v15y2023i14p10828-d1190887.html
   My bibliography  Save this article

Transformer Architecture-Based Transfer Learning for Politeness Prediction in Conversation

Author

Listed:
  • Shakir Khan

    (College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University, Riyadh 11432, Saudi Arabia
    University Center for Research and Development, Department of Computer Science and Engineering, Chandigarh University, Mohali 140413, India)

  • Mohd Fazil

    (Center for Transformative Learning, University of Limerick, V94 T9PX Limerick, Ireland)

  • Agbotiname Lucky Imoize

    (Department of Electrical and Electronics Engineering, Faculty of Engineering, University of Lagos, Akoka, Lagos 100213, Nigeria)

  • Bayan Ibrahimm Alabduallah

    (Department of Information System, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, Riyadh 11564, Saudi Arabia)

  • Bader M. Albahlal

    (College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University, Riyadh 11432, Saudi Arabia)

  • Saad Abdullah Alajlan

    (College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University, Riyadh 11432, Saudi Arabia)

  • Abrar Almjally

    (College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University, Riyadh 11432, Saudi Arabia)

  • Tamanna Siddiqui

    (Department of Computer Science, Aligarh Muslim University, Aligarh 202002, India)

Abstract

Politeness is an essential part of a conversation. Like verbal communication, politeness in textual conversation and social media posts is also stimulating. Therefore, the automatic detection of politeness is a significant and relevant problem. The existing literature generally employs classical machine learning-based models like naive Bayes and Support Vector-based trained models for politeness prediction. This paper exploits the state-of-the-art (SOTA) transformer architecture and transfer learning for respectability prediction. The proposed model employs the strengths of context-incorporating large language models, a feed-forward neural network, and an attention mechanism for representation learning of natural language requests. The trained representation is further classified using a softmax function into polite, impolite, and neutral classes. We evaluate the presented model employing two SOTA pre-trained large language models on two benchmark datasets. Our model outperformed the two SOTA and six baseline models, including two domain-specific transformer-based models using both the BERT and RoBERTa language models. The ablation investigation shows that the exclusion of the feed-forward layer displays the highest impact on the presented model. The analysis reveals the batch size and optimization algorithms as effective parameters affecting the model performance.

Suggested Citation

  • Shakir Khan & Mohd Fazil & Agbotiname Lucky Imoize & Bayan Ibrahimm Alabduallah & Bader M. Albahlal & Saad Abdullah Alajlan & Abrar Almjally & Tamanna Siddiqui, 2023. "Transformer Architecture-Based Transfer Learning for Politeness Prediction in Conversation," Sustainability, MDPI, vol. 15(14), pages 1-11, July.
  • Handle: RePEc:gam:jsusta:v:15:y:2023:i:14:p:10828-:d:1190887
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/15/14/10828/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/15/14/10828/
    Download Restriction: no
    ---><---

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:15:y:2023:i:14:p:10828-:d:1190887. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.