Improved Character-Based Neural Network for POS Tagging on Morphologically Rich Languages
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Since the widespread adoption of deep-learning and related distributed representation, there have beensubstantial advancements in part-of-speech (POS) tagging for many languages. When training wordrepresentations, morphology and shape are typically ignored, as these representations rely primarily oncollecting syntactic and semantic aspects of words. However, for tasks like POS tagging, notably inmorphologically rich and resource-limited language environments, the intra-word information is essential. Inthis study, we introduce a deep neural network (DNN) for POS tagging that learns character-level wordrepresentations and combines them with general word representations. Using the proposed approach andomitting hand-crafted features, we achieve 90.47%, 80.16%, and 79.32% accuracy on our own dataset for threemorphologically rich languages: Uyghur, Uzbek, and Kyrgyz. The experimental results reveal that thepresented character-based strategy greatly improves POS tagging performance for several morphologically richlanguages (MRL) where character information is significant. Furthermore, when compared to the previouslyreported state-of-the-art POS tagging results for Turkish on the METU Turkish Treebank dataset, the proposedapproach improved on the prior work slightly. As a result, the experimental results indicate that character-basedrepresentations outperform word-level representations for MRL performance. Our technique is also robusttowards the-out-of-vocabulary issues and performs better on manually edited text.
Abstract
Since the widespread adoption of deep-learning and related distributed representation, there have beensubstantial advancements in part-of-speech (POS) tagging for many languages. When training wordrepresentations, morphology and shape are typically ignored, as these representations rely primarily oncollecting syntactic and semantic aspects of words. However, for tasks like POS tagging, notably inmorphologically rich and resource-limited language environments, the intra-word information is essential. Inthis study, we introduce a deep neural network (DNN) for POS tagging that learns character-level wordrepresentations and combines them with general word representations. Using the proposed approach andomitting hand-crafted features, we achieve 90.47%, 80.16%, and 79.32% accuracy on our own dataset for threemorphologically rich languages: Uyghur, Uzbek, and Kyrgyz. The experimental results reveal that thepresented character-based strategy greatly improves POS tagging performance for several morphologically richlanguages (MRL) where character information is significant. Furthermore, when compared to the previouslyreported state-of-the-art POS tagging results for Turkish on the METU Turkish Treebank dataset, the proposedapproach improved on the prior work slightly. As a result, the experimental results indicate that character-basedrepresentations outperform word-level representations for MRL performance. Our technique is also robusttowards the-out-of-vocabulary issues and performs better on manually edited text.
참고문헌 (18)
- X. Yu, "Character composition model with convolutional neural networks for dependency parsing on morphologically rich languages", Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp.672~678 (2017)
- "Overcoming the rare word problem for low-resource language pairs in neural machine translation" (2019)[URL]
- M. Grohe, "word2vec, node2vec, graph2vec, x2vec: towards a theory of vector embeddings of structured data", Proceedings of the 39th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems, pp.1~16 (2020)
- C. I. Eke, "The significance of global vectors representation in sarcasm analysis", Proceedings of 2020 International Conference in Mathematics, Computer Engineering and Computer Science (ICMCECS), pp.1~7 (2020)
- P. Bojanowski, "Enriching word vectors with subword information", Transactions of the Association for Computational Linguistics, 5, pp.135~146 (2017)
- Alebachew Chiche, "Part of speech tagging: a systematic review of deep learning and machine learning approaches", Journal of Big Data, 9(1) (2022), Springer Science and Business Media LLC[DOI]
- R. D. Deshmukh, "Deep learning techniques for part of speech tagging by natural language processing", Proceedings of 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), pp.76~81 (2020)
- K. K. Akhil, "Parts-of-Speech tagging for Malayalam using deep learning techniques", International Journal of Information Technology, 12, pp.741~748 (2020)
- W. Khan, "Part of speech tagging in Urdu : comparison of machine and deep learning approaches", IEEE Access, 7, pp.38918~38936 (2019)
- C. A. Bahcevan, "Deep neural network architecture for part-of-speech tagging for Turkish language", Proceedings of 2018 3rd International Conference on Computer Science and Engineering (UBMK), pp.235~238 (2018)
- "Building a Kannada POS tagger using machine learning and neural network models" (2018)[URL]
- S. Kumar, "Deep learning based part-of-speech tagging for Malayalam Twitter data (Special issue: deep learning techniques for natural language processing)", Journal of Intelligent Systems, 28(3), pp.423~435 (2019)
- M. Maimaiti, "Bidirectional long short-term memory network with a conditional random field layer for Uyghur part-of-speech tagging", Information, 8(4) (2017)[DOI]
- S. Fouladi, "Efficient deep neural networks for classification of COVID-19 based on CT images: virtualization via software defined radio", Computer Communications, 176, pp.234~248 (2021)
- A. Javaheri, "Public vs media opinion on robots and their evolution over recent years", CCF Transactions on Pervasive Computing and Interaction, pp.189~205 (2020)
- S. Fouladi, "Efficient deep neural networks for classification of Alzheimer’s disease and mild cognitive impairment from scalp EEG recordings", Cognitive Computation, 14, pp.1247~1268 (2022)
- R. Collobert, "Deep learning for efficient discriminative parsing", Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), pp.224~232 (2011)
- Y. Esref, "Using morpheme-level attention mechanism for Turkish sequence labelling", Proceedings of 2019 27th Signal Processing and Communications Applications Conference (SIU), pp.1~4 (2019)