The use of context vectors in determining Thai compounds
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In this paper we first discuss the problem of identifying compound words in Thai. It will be shown that the structures of compound words are often identical to the structure of phrases or sentences. Determining whether a sequence of words is a compound or a phrase or a sentence has to be determined within the context in which it occurs. Therefore, there is no clear cut way of determining the boundary of a compound. The longer the word sequence, the less likely it is that it will be considered a compound. In this study, we focus on extracting compound words consisting of two words from a large corpus using a vector space model. The basic assumption is that the context in which a compound occurs should be different from the context in which its parts occur. Two experiments were conducted on known compounds and on general bigram words. The test on known compounds was to verify that the cosine similarity of the context vector clearly indicates the differences of context vectors between the compound and its parts. The test on general bigram words was to further verify that the cosine similarity of context vectors between a non-compound bigram and its parts is different from that found in known compounds. When applying the cosine similarity of context vectors to compound candidates which have been extracted from a large corpus and ranked by statistics of collocation, we can determine a compound correctly with the F-measure at 0.81. The results indicate that the cosine similarity of context vectors is useful for determining a compound in Thai.
Abstract
In this paper we first discuss the problem of identifying compound words in Thai. It will be shown that the structures of compound words are often identical to the structure of phrases or sentences. Determining whether a sequence of words is a compound or a phrase or a sentence has to be determined within the context in which it occurs. Therefore, there is no clear cut way of determining the boundary of a compound. The longer the word sequence, the less likely it is that it will be considered a compound. In this study, we focus on extracting compound words consisting of two words from a large corpus using a vector space model. The basic assumption is that the context in which a compound occurs should be different from the context in which its parts occur. Two experiments were conducted on known compounds and on general bigram words. The test on known compounds was to verify that the cosine similarity of the context vector clearly indicates the differences of context vectors between the compound and its parts. The test on general bigram words was to further verify that the cosine similarity of context vectors between a non-compound bigram and its parts is different from that found in known compounds. When applying the cosine similarity of context vectors to compound candidates which have been extracted from a large corpus and ranked by statistics of collocation, we can determine a compound correctly with the F-measure at 0.81. The results indicate that the cosine similarity of context vectors is useful for determining a compound in Thai.
참고문헌 (30)
- Aronoff, Mark, "What is morphology?" (2011), Wiley-Blackwell
- Aroonmanakun, Wirote, "Collocation and Thai word segmentation", Proceedings of the Fifth Symposium on Natural Language Processing & The Fifth Oriental COCOSDA Workshop, pp.68~75 (2002), Sirindhorn International Institute of Technology
- Aroonmanakun, Wirote, "Creating the Thai national corpus", pp.4~17 (2007)
- Aroonmanakun, Wirote, "Extracting Thai compounds using collocations and POS Bigram probabilities without a POS tagger", Paper presented to the Asian Language Processing, 2009. IALP '09. International Conference on, 2009 (2009)
- Attia, Mohammed, "Automatic extraction of Arabic multiword expressions", Proceedings of the Workshop on Multiword Expressions: from Theory to Applications (MWE 2010) (2010), Association for Computational Linguistics
- Bloomfield, Leonard, "Language" (1933), H. Holt and Company
- Booij, G. E, "The Grammar of Words : An Introduction to Linguistic Morphology" (2007), Oxford University Press
- Church, Kenneth Ward, "Word association norms, mutual information, and lexicography", Computational Linguistics, 16(1), pp.22~29 (1990)
- Downing, Pamela., "On the creation and use of English compound nouns", Language, 53, pp.810~842 (1977)
- Dressler, W.U., "The Representation and Processing of Compound Words 15", pp.242~ (2006), Oxford University Press
- Girju, Roxana, "On the semantics of noun compounds", Computer Speech and Language, 19, pp.479~496 (2005)
- Jian, Zhang, "Extraction of Chinese compound words:an experimental study on a very large corpus", Proceedings of the second workshop on Chinese language processing: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics, 12 (2000), Association for Computational Linguistics
- Kunchukuttan, Anoop, "A system for compound noun multiword expression extraction for Hindi", Paper presented to the 6th International Conference on Natural Language Processing (2008)
- Lauer, Mark, "Corpus statistics meet the noun compound: Some empirical results", Paper presented at the The 33rd Annual Meeting of the Association for Computational Linguistics (1995)
- Lees, Robert B, "The Grammar of English Nominalizations" (1960)
- Liu, Jianzhou, "Extracting Chinese multi-word units from large-scale balanced corpus", Paper presented to The 17th Pacific Asia Conference on Language, Information and Computation (2003)
- Nakagawa, Hiroshi, "A simple but powerful automatic term extraction method", Paper presented at the COLING-02 on COMPUTERM 2002: Second International Workshop on Computational Terminology, 14 (2002)
- Pedersen, Ted, "Computational Approaches to Measuring the Similarity of Short Contexts : A Review of Applications and Methods", University of Minnesota Super Computuing Institute UMSI 2010, 2010
- Prasithrathsint, Amara, "Lexicalization of syntactic constructions in Thai", Paper presented to The 20th Anniversary Meeting of the Southeast Asian Linguistics Society (2010)
- Rosario, Barbara, "Classifying the semantic relations in noun compounds via a domain-specific lexical hierarchy", Paper presented at the Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing (2001)
- Sag, Ivan A, "Computational Linguistics and Intelligent Text Processing", pp.1~15 (2002), Springer
- Salton, Gerard, "A vector space model for automatic indexing", Commun. ACM, 18, pp.613~620 (1975)
- Singnoi, Unchalee, "Nominal Constructions in Thai", Ph.D, pp.2000
- Smadja, Frank, "Translating collocations for bilingual lexicons: a statistical approach", Computational Linguistics, 22(1), pp.1~38 (1996)
- Sornlertlamvanich, Virach, "Automatic corpus-based Thai eord extraction with the c4.5 learning algorithm", Paper presented at the Proceedings of the 18th conference on Computational linguistics, 2 (2000)
- Sornlertlamvanich, Virach, "The automatic extraction of open compounds from text corpora", Paper presented to the COLING '96 (1996)
- Spencer, Andrew, "What’s in a compound?", Journal of Linguistics, 47, pp.481~507 (2011)
- Su, Keh-Yih, "A corpus-based approach to automatic compound extraction", Paper presented at the Proceedings of the 32nd annual meeting on Association for Computational Linguistics (1994)
- Suwanno, N., "Extracting Thai compound nouns for para graph extraction in Thai text", Paper presented to the Natural Language Processing and Knowledge Engineering, 2005. IEEE NLP-KE '05. Proceedings of 2005 IEEE International Conference on, 2005 (2005)
- Turney, Peter D., "From Frequency to Meaning: Vector Space Models of Semantics", CoRR abs, 2010