A Manchu speller: With a practical introduction to the natural language processing of minority languages
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This article presents a Manchu speller developed using open-source proofingtools for use in a large-scale digitization of Manchu lexical and textualresources published during 17th and 18th centuries. A speller requiresthe list of words and affix rules. Morphological analysis is inevitable todevelop a spell checker especially for morphologically rich languages. Thedictionaries compiled for a speller would be a primary resource for thedevelopment of a morphological analyzer. The present Manchu speller,therefore, is introduced as a preparation for further development ofmorphological analyzer. I intended to propose possibly best practices for non-computationallinguists who want to take an initial step of natural language processing ofminority languages without serious programming knowledge. Developinga spell checker and morphological analyzer is the initial and criticalstep for natural language processing. This article introduces the way inwhich researchers of minority languages only need to organize the listof lexical items and morphological rules of the target language into theformat specified by the spell-checking engine. The main sections arededicated to introducing popular spell-checking engines, briefly reviewingmajor issues and concepts of computational morphology with concreteexamples, and finally demonstrating how to implement a working spellerand morphological analyzer with the explanation of how to translatingefficiently linguistic knowledge into the technical description.
Abstract
This article presents a Manchu speller developed using open-source proofingtools for use in a large-scale digitization of Manchu lexical and textualresources published during 17th and 18th centuries. A speller requiresthe list of words and affix rules. Morphological analysis is inevitable todevelop a spell checker especially for morphologically rich languages. Thedictionaries compiled for a speller would be a primary resource for thedevelopment of a morphological analyzer. The present Manchu speller,therefore, is introduced as a preparation for further development ofmorphological analyzer. I intended to propose possibly best practices for non-computationallinguists who want to take an initial step of natural language processing ofminority languages without serious programming knowledge. Developinga spell checker and morphological analyzer is the initial and criticalstep for natural language processing. This article introduces the way inwhich researchers of minority languages only need to organize the listof lexical items and morphological rules of the target language into theformat specified by the spell-checking engine. The main sections arededicated to introducing popular spell-checking engines, briefly reviewingmajor issues and concepts of computational morphology with concreteexamples, and finally demonstrating how to implement a working spellerand morphological analyzer with the explanation of how to translatingefficiently linguistic knowledge into the technical description.
참고문헌 (21)
- Alegria, Iñaki, "Spelling correction: From two-level morphology to open source", Proceedings of LREC 2008, pp.1051~1054 (2008), ELRA
- Aronoff, Mark, "Morphology by itself: Stems and inflectional classes" (1994), MIT Press
- What is morphology?, "Aronoff, Mark", Blackwell, ~ (2011)
- Avrorin, Valentin A, "Grammar of written Manchu language" (2000), Nauka
- Barton, G. Edward, "The computational complexity of two-level morphology", Proceedings of the 24th annual meeting on association for computational linguistics, pp.53~59 (1985), ACL
- Beesley, Kennech R, "Morphological analysis and generation: A first-step in natural lanugage processing", Proceedings of the SALTMIL Workshop at LREC 2004:First steps in language documentation for minority languages, pp.1~8 (2004), SALTMIL
- Gorelova, Liliya M, "Manchu grammar" (2002), Brill
- Halácsy, Péter, "Creating open language resources for Hungarian", Proceedings of LREC 2004, pp.1201~1204 (2004), ERLA
- Kawachi, Y., "Introduction to literary Manchu" (2002), Kyoto Daigaku gakuzyutsu Shuppankai
- Koskenniemi, K., "Two-level morphology: A general computational model for wordform recognition and production", Department of General Linguistics, pp.1983
- Miɫkowski, Marcin, "Developing an open-source, rule-base proofreading tool", Software: Practice and Experience, 40(7), pp.543~566 (2010), Wiley InterScience
- Möllendorff, P. G. von, "A Manchu grammar with analyzed texts" (1892), American Presbyterian Mission Press
- Németh, László, "Leveraging the open source Ispell codebase for minority language analysis", Proceedings of the SALTMIL workshop at LREC 2004: First steps in language documentation for minority languages, pp.56~59 (2004), SATMIL
- Payne, Thomas E, "Describing morphosyntax: A guide for field linguists" (1997), Cambridge University Press
- Pirinen, T. A., "Creating and weighting Hunspell dictionaries as finite-state automata", Investigationes Linguisticae, 21, pp.1~16 (2010), Institute of Linguistics, Adam Mickiewicz University
- Porter, M. F, "An algorithm for suffix stripping", Program: Electronic library and information systems, 14(3), pp.130~137 (1980), Emerald
- Pustejovsky, J, "Computational lexicons", pp.160~162 (2014), MIT Press
- Ritchie, Graeme D., "Computational morphology: Practical mechanisms for the English lexicon" (1992), MIT Press
- Smrž, Otakar, "Mutilingual natural language processing applications", pp.3~28 (2010), IBM Press
- Trón, Viktor, "Hunmorph: open source word analysis", Proceedings of the ACL 2005 workshop on software, pp.77~85 (2005), Association for Computational Linguistics
- Zakharov, Ivan I., "Grammar of the Manchu Language" (1879), Nauka