TY - GEN
T1 - EmoThreat@FIRE2022
T2 - 14th Annual Forum for Information Retrieval Evaluation
AU - Butt, Sabur
AU - Amjad, Maaz
AU - Balouchzah, Fazlourrahman
AU - Ashraf, Noman
AU - Sharma, Rajesh
AU - Sidorov, Grigori
AU - Gelbukh, Alexander
N1 - Publisher Copyright:
© 2022 Owner/Author.
PY - 2022/12/9
Y1 - 2022/12/9
N2 - Many languages with a wealth of resources have been researched to solve the challenges of emotion and targeted abuse detection, i.e.Threat. But when it comes to languages, such as Urdu, it is noted that there is a severe lack of both resources and approaches in terms of Urdu language processing. Therefore, this study concentrated on offering resources for Urdu by organizing a shared task called "EmoThreat: Emotions and Threat detection in Urdu". The task offered two tasks: (i) multi-label emotion classification (Task A), and (ii) binary threat detection (Task B). Task B was a multi-class problem since it was further subdivided into the identification of threats posed by groups and individuals. This paper provides an overview of the methodology and results obtained by each of the 10 distinct teams who participated in the shared task. In addition, each group presented a detailed error analysis as part of their submission for the best model. The top-performing system in Task A received a macro-F1 score of 0.687. In contrast, subtask 1 of Task B received a score of 0.716 macro-F1 while subtask 2 of Task B obtained a 0.539 macro-F1 score.
AB - Many languages with a wealth of resources have been researched to solve the challenges of emotion and targeted abuse detection, i.e.Threat. But when it comes to languages, such as Urdu, it is noted that there is a severe lack of both resources and approaches in terms of Urdu language processing. Therefore, this study concentrated on offering resources for Urdu by organizing a shared task called "EmoThreat: Emotions and Threat detection in Urdu". The task offered two tasks: (i) multi-label emotion classification (Task A), and (ii) binary threat detection (Task B). Task B was a multi-class problem since it was further subdivided into the identification of threats posed by groups and individuals. This paper provides an overview of the methodology and results obtained by each of the 10 distinct teams who participated in the shared task. In addition, each group presented a detailed error analysis as part of their submission for the best model. The top-performing system in Task A received a macro-F1 score of 0.687. In contrast, subtask 1 of Task B received a score of 0.716 macro-F1 while subtask 2 of Task B obtained a 0.539 macro-F1 score.
KW - Threatening languages detection
KW - Urdu language
KW - emotion detection
KW - low resource languages
KW - text classification
UR - http://www.scopus.com/inward/record.url?scp=85146658212&partnerID=8YFLogxK
U2 - 10.1145/3574318.3574327
DO - 10.1145/3574318.3574327
M3 - Contribución a la conferencia
AN - SCOPUS:85146658212
T3 - ACM International Conference Proceeding Series
SP - 1
EP - 3
BT - FIRE 2022 - Proceedings of the 14th Annual Meeting of the Forum for Information Retrieval Evaluation
A2 - Ganguly, Debasis
A2 - Gangopadhyay, Surupendu
A2 - Mitra, Mandar
A2 - Majumder, Prasenjit
PB - Association for Computing Machinery
Y2 - 9 December 2022 through 13 December 2022
ER -