Overview of EmoThreat: Emotions and Threat Detection in Urdu at FIRE 2022

Sabur Butt, Maaz Amjad, Fazlourrahman Balouchzahi, Noman Ashraf, Rajesh Sharma, Grigori Sidorov, Alexander Gelbukh

Research output: Contribution to journalConference articlepeer-review

10 Scopus citations

Abstract

Emotion and targeted abuse detection i.e threat, are problems that have been studied in many rich resource languages. However, when it comes to low-resource languages such as Urdu, we find a dearth of resources and methodologies. Our paper presents the findings of the shared task "EmoThreat: Emotions and Threat detection in Urdu", where we focused on presenting resources for multi-label emotion classification (Task A) and binary threat detection (Task B) in Urdu. Task B was further divided into group and individual threat detection, making it a multi-class problem. The paper presents a summary of the methodologies and findings of the ten different participating teams. Each team also presented a thorough error analysis for the best model. The best performing system in Task A achieved a macroF1 score of 0.687, whereas, Task B subtask 1 and subtask 2 achieved 0.716 and 0.539 macro-F1 scores respectively.

Original languageEnglish
Pages (from-to)220-230
Number of pages11
JournalCEUR Workshop Proceedings
Volume3395
StatePublished - 2022
Event14th Forum for Information Retrieval Evaluation, FIRE 2022 - Kolkata, India
Duration: 9 Dec 202213 Dec 2022

Keywords

  • Emotion Detection
  • Group Threats
  • Individual Threats
  • Natural Language Processing
  • Threatening language Detection
  • Urdu language

Fingerprint

Dive into the research topics of 'Overview of EmoThreat: Emotions and Threat Detection in Urdu at FIRE 2022'. Together they form a unique fingerprint.

Cite this