A syntactico-semantic analysis of non-factive predicates for the construction of natural language inference datasets

Abstract

This study presents a syntactico-semantic analysis of non-factive predicates for constructing Natural Language Inference (NLI) datasets. NLI datasets are essential for enhancing the performance of current language models, which are expected to understand human intentions and engage in dialogue effectively. Given that existing Korean NLI datasets are not developed with careful consideration of syntactic and semantic linguistic features, several limitations arise in building linguistically robust language models. In this study, we classify six types of non-factive predicates, which, unlike factive predicates, do not presuppose the truth value of their embedded clauses. Using 600 Korean NLI pairs based on these non-factive predicates, we evaluated a state-of-the-art language model, GPT-4o, and found that it achieved a relatively low F1 score of 52.65%. This result highlights the need for fine-tuning language models using linguistically informed NLI datasets, such as those proposed in this study.