TY - JOUR
T1 - PolyHope
T2 - Two-level hope speech detection from tweets
AU - Balouchzahi, Fazlourrahman
AU - Sidorov, Grigori
AU - Gelbukh, Alexander
N1 - Publisher Copyright:
© 2023 Elsevier Ltd
PY - 2023/9/1
Y1 - 2023/9/1
N2 - Hope is characterized as openness of spirit towards the future, a desire, expectation, and wish for something to happen or to be true that remarkably affects human's state of mind, emotions, behaviors, and decisions. Hope is usually associated with concepts of desired expectations and possibility/probability concerning the future. Despite its importance, hope has rarely been studied as a social media analysis task. This paper presents a hope speech dataset that classifies each tweet first into “Hope” and “Not Hope”, then into three fine-grained hope categories: “Generalized Hope”, “Realistic Hope”, and “Unrealistic Hope” (along with “Not Hope”). English tweets in the first half of 2022 were collected to build this dataset. Furthermore, we describe our annotation process and guidelines in detail and discuss the challenges of classifying hope and the limitations of the existing hope speech detection corpora. In addition, we reported several baselines based on different learning approaches, such as traditional machine learning, deep learning, and transformers, to benchmark our dataset. We evaluated our baselines using averaged-weighted and averaged-macro F1-scores. Observations show that a strict process for annotator selection and detailed annotation guidelines enhanced the dataset's quality. This strict annotation process yielded promising performance for simple machine learning classifiers with only uni-grams; however, binary and multiclass hope speech detection results reveal that contextual embedding models have higher performance in this dataset.
AB - Hope is characterized as openness of spirit towards the future, a desire, expectation, and wish for something to happen or to be true that remarkably affects human's state of mind, emotions, behaviors, and decisions. Hope is usually associated with concepts of desired expectations and possibility/probability concerning the future. Despite its importance, hope has rarely been studied as a social media analysis task. This paper presents a hope speech dataset that classifies each tweet first into “Hope” and “Not Hope”, then into three fine-grained hope categories: “Generalized Hope”, “Realistic Hope”, and “Unrealistic Hope” (along with “Not Hope”). English tweets in the first half of 2022 were collected to build this dataset. Furthermore, we describe our annotation process and guidelines in detail and discuss the challenges of classifying hope and the limitations of the existing hope speech detection corpora. In addition, we reported several baselines based on different learning approaches, such as traditional machine learning, deep learning, and transformers, to benchmark our dataset. We evaluated our baselines using averaged-weighted and averaged-macro F1-scores. Observations show that a strict process for annotator selection and detailed annotation guidelines enhanced the dataset's quality. This strict annotation process yielded promising performance for simple machine learning classifiers with only uni-grams; however, binary and multiclass hope speech detection results reveal that contextual embedding models have higher performance in this dataset.
KW - Deep learning
KW - Desire
KW - Expectation
KW - Hope
KW - Machine learning
KW - Natural Language Processing
KW - Transformers
KW - Wish
UR - http://www.scopus.com/inward/record.url?scp=85152436287&partnerID=8YFLogxK
U2 - 10.1016/j.eswa.2023.120078
DO - 10.1016/j.eswa.2023.120078
M3 - Artículo
AN - SCOPUS:85152436287
SN - 0957-4174
VL - 225
JO - Expert Systems with Applications
JF - Expert Systems with Applications
M1 - 120078
ER -