TY - JOUR
T1 - GAN-BERT, an Adversarial Learning Architecture for Paraphrase Identification
AU - Ta, Hoang Thang
AU - Rahman, Abu Bakar Siddiqur
AU - Najjar, Lotfollah
AU - Gelbukh, Alexander
N1 - Publisher Copyright:
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
PY - 2022
Y1 - 2022
N2 - In this paper, we address the task of Paraphrase Identification in Mexican Spanish (PAR-MEX) at sentence-level. We introduced our method, using text embeddings from pre-trained transformer models for the training process by GAN-BERT, an adversarial learning. We modified noises for the generator, which have a random rate and the same size of the hidden layer of transformers. To improve the model performance, a rule of thumb based on the pair similarity is used to remove possible wrong sentence pairs in positive examples; parallel with the addition of unlabelled data in the same domain. The best obtained F1 is 90.22%, ranked third in the final result table, also outperformed the organizers' baseline.
AB - In this paper, we address the task of Paraphrase Identification in Mexican Spanish (PAR-MEX) at sentence-level. We introduced our method, using text embeddings from pre-trained transformer models for the training process by GAN-BERT, an adversarial learning. We modified noises for the generator, which have a random rate and the same size of the hidden layer of transformers. To improve the model performance, a rule of thumb based on the pair similarity is used to remove possible wrong sentence pairs in positive examples; parallel with the addition of unlabelled data in the same domain. The best obtained F1 is 90.22%, ranked third in the final result table, also outperformed the organizers' baseline.
KW - GAN-BERT
KW - IberLEF
KW - PAR-MEX
KW - Paraphrase Identification
KW - Text Classification
UR - http://www.scopus.com/inward/record.url?scp=85137346387&partnerID=8YFLogxK
M3 - Artículo de la conferencia
AN - SCOPUS:85137346387
SN - 1613-0073
VL - 3202
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
T2 - 2022 Iberian Languages Evaluation Forum, IberLEF 2022
Y2 - 20 September 2022
ER -