Metaphor interpretation in Jordanian Arabic, Emirati Arabic and Classical Arabic: artificial intelligence vs. humans

This study examines how well humans, both Jordanians and Emiratis, and four AI tools—ChatGPT-4, ChatGPT-3.5, Google Gemini, and Ask PDF—can understand metaphors in Classical Arabic (CA) and its everyday forms in Jordanian Arabic (JA) and Emirati Arabic (EA). We tested fifty participants from Jordan and the UAE on their grasp of various colloquial and CA metaphorical expressions. Two distinct tests were employed, each comprising 40 items. Test 1 was administered to Jordanian participants and included 20 metaphorical expressions in Jordanian Arabic and 20 metaphorical expressions in Classical Arabic. Similarly, Test 2 was administered to Emirati participants and contained 20 expressions in Emirati Arabic and 20 expressions in Classical Arabic. The Mann–Whitney U test was employed to evaluate differences in accuracy and interpretation between AI tools and human participants from both regions in the contexts of colloquial and Classical Arabic. The results showed that participants from Jordan had a better understanding than the AI tools, likely due to their strong cultural background. In contrast, the Emirati participants performed similarly to the AI. The AI tools were more effective at interpreting CA metaphors compared to Emirati participants; AI tools are typically trained on diverse datasets and that usually leads to strong performance in interpreting formal or Classical Arabic expressions. These findings emphasize the need for improvements in AI models to boost their language processing abilities, as they often miss the cultural aspects required for accurately interpreting figurative language. This study adds to the ongoing discussion about AI and language interpretation, revealing both the potential and the obstacles AI faces when dealing with culturally rich and context-sensitive language.