TY - GEN
T1 - Dependency Parser based textual entailment system
AU - Pakray, Partha
AU - Bandyopadhyay, Sivaji
AU - Gelbukh, Alexander
PY - 2010
Y1 - 2010
N2 - The development of a parser based textual entailment system that is based on comparing the dependency relations in both the text and the hypothesis has been reported. The textual entailment system uses the CCG Parser and the Stanford Parser. The Dependency Parser has been run on the 2-way Parser Training and Evaluation (PETE) (SemEval-2010 Evaluation Exercises on Semantic Evaluation Task 12 Parser Evaluation using Textual Entailment) trial set and the dependency relations obtained for a text and hypothesis pair has been compared. Some of the important comparisons are: subject-verb comparison, subject-subject comparison, object-verb comparison and cross subject-verb comparison. Each of the matches is assigned some weight learned from the PETE trial set corpus. A threshold has been set on the fraction of matching hypothesis relations for YES entailment decision based on the PETE trial set. The threshold score has been applied on the PETE test set using the same methods of dependency parsing followed by comparisons. Evaluation scores for Run 1 (CCG Parser output), obtained on the test set show 58.19% precision and 45.51% recall for YES decisions and 52.51% precision and 64.82% recall for NO decisions. Evaluation scores for Run 2 (Stanford Parser output), obtained on the test set show 55.68% precision and 59.61% recall for YES decisions and 52.23% precision and 48.61% recall for NO decisions. Evaluation scores for Run 3 (combining the output from CCG parser and Stanford Parser), obtained on the test set show 57.14% precision and 74.35% recall for YES decisions and 59.18% precision and 40% recall for NO decisions.
AB - The development of a parser based textual entailment system that is based on comparing the dependency relations in both the text and the hypothesis has been reported. The textual entailment system uses the CCG Parser and the Stanford Parser. The Dependency Parser has been run on the 2-way Parser Training and Evaluation (PETE) (SemEval-2010 Evaluation Exercises on Semantic Evaluation Task 12 Parser Evaluation using Textual Entailment) trial set and the dependency relations obtained for a text and hypothesis pair has been compared. Some of the important comparisons are: subject-verb comparison, subject-subject comparison, object-verb comparison and cross subject-verb comparison. Each of the matches is assigned some weight learned from the PETE trial set corpus. A threshold has been set on the fraction of matching hypothesis relations for YES entailment decision based on the PETE trial set. The threshold score has been applied on the PETE test set using the same methods of dependency parsing followed by comparisons. Evaluation scores for Run 1 (CCG Parser output), obtained on the test set show 58.19% precision and 45.51% recall for YES decisions and 52.51% precision and 64.82% recall for NO decisions. Evaluation scores for Run 2 (Stanford Parser output), obtained on the test set show 55.68% precision and 59.61% recall for YES decisions and 52.23% precision and 48.61% recall for NO decisions. Evaluation scores for Run 3 (combining the output from CCG parser and Stanford Parser), obtained on the test set show 57.14% precision and 74.35% recall for YES decisions and 59.18% precision and 40% recall for NO decisions.
KW - CCG parser
KW - PETE test set
KW - PETE trial set
KW - Stanford parser
KW - Textual entailment
UR - http://www.scopus.com/inward/record.url?scp=78651442792&partnerID=8YFLogxK
U2 - 10.1109/AICI.2010.89
DO - 10.1109/AICI.2010.89
M3 - Contribución a la conferencia
AN - SCOPUS:78651442792
SN - 9780769542256
T3 - Proceedings - International Conference on Artificial Intelligence and Computational Intelligence, AICI 2010
SP - 393
EP - 397
BT - Proceedings - International Conference on Artificial Intelligence and Computational Intelligence, AICI 2010
T2 - 2010 International Conference on Artificial Intelligence and Computational Intelligence, AICI 2010
Y2 - 23 October 2010 through 24 October 2010
ER -