단어그룹 확장 기법을 활용한 순환신경망 알고리즘 성능개선 연구
A Study on Performance Improvement of Recurrent Neural Networks Algorithm using Word Group Expansion Technique
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
최근 인공지능(AI)과 딥러닝 발전으로 대화형 인공지능 챗봇의 중요성이 부각되고 있으며 다양한 분야에서 연구가 진행되고 있다. 챗봇을 만들기 위해서 직접 개발해 사용하기도 하지만 개발의 용이성을 위해 오픈소스 플랫폼이나 상업용 플랫폼을 활용하여 개발한다. 이러한 챗봇 플랫폼은 주로 RNN (Recurrent Neural Network)과 응용 알고리즘을 사용하며, 빠른 학습속도와 모니터링 및 검증의 용이성 그리고 좋은 추론 성능의 장점을 가지고 있다. 본 논문에서는 RNN과 응용 알고리즘의 추론 성능 향상방법을 연구하였다. 제안 방법은 RNN과 응용 알고리즘 적용 시 각 문장에 대한 핵심단어의 단어그룹에 대해 확장학습을 통해 데이터에 내재된 의미를 넓히는 기법을 사용하였다. 본 연구의 결과는 순환구조를 갖는 RNN, GRU (Gated Recurrent Unit), LSTM (Long-short Term Memory) 세 알고리즘에서 최소 0.37%에서 최대 1.25% 추론 성능향상을 달성하였다. 본 연구를 통해 얻은 연구결과는 관련 산업에서 인공지능 챗봇 도입을 가속하고 다양한 RNN 응용 알고리즘을 활용하도록 하는데 기여할 수 있다. 향후 연구에서는 다양한 활성 함수들이 인공신경망 알고리즘의 성능 향상에 미치는 영향에 관한 연구가 필요할 것이다.
Abstract
Recently, with the development of artificial intelligence (AI) and deep learning, the importance of conversational artificial intelligence chatbots is being highlighted. In addition, chatbot research is being conducted in various fields. To build a chatbot, it is developed using an open source platform or a commercial platform for ease of development. These chatbot platforms mainly use RNN and application algorithms. The RNN algorithm has the advantages of fast learning speed, ease of monitoring and verification, and good inference performance. In this paper, a method for improving the inference performance of RNNs and applied algorithms was studied. The proposed method used the word group expansion learning technique of key words for each sentence when RNN and applied algorithm were applied. As a result of this study, the RNN, GRU, and LSTM three algorithms with a cyclic structure achieved a minimum of 0.37% and a maximum of 1.25% inference performance improvement. The research results obtained through this study can accelerate the adoption of artificial intelligence chatbots in related industries. In addition, it can contribute to utilizing various RNN application algorithms. In future research, it will be necessary to study the effect of various activation functions on the performance improvement of artificial neural network algorithms.
참고문헌 (31)
- Rahman, A. M., "Programming challenges of chatbot: Current and future prospective", 5th IEEE Region 10Humanitarian Technology Conference 2017, 2017, pp.75~78 (2018)
- Hristidis, V., "Chatbot Technologies and Challenges", 2018 First International Conference on Artificial Intelligence for Industries (AI4I), pp.126~126 (2019)
- Lijia Chen, "Artificial Intelligence in Education: A Review", IEEE Access, 8, pp.75264~75278 (2020), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Daniel, G., "The Software Challenges of Building Smart Chatbots", Proceedings - International Conference on Software Engineering, pp.324~325 (2021)
- Nguyen, P. A. H., "Evaluating AI chatbot platforms by a fuzzy AHP approach", Proceedings of 2021 International Conference on System Science and Engineering, ICSSE 2021, pp.206~209 (2021)
- Prasetyo, A., "Intents Categorization for Chatbot Development Using Recurrent Neural Network (RNN) Learning", 20217th International Conference on Advanced Computing and Communication Systems, ICACCS 2021, pp.551~556 (2021)
- Majid, R., "Conversations Sentiment and Intent Categorization Using Context RNN for Emotion Recognition", 2021 7th International Conference on Advanced Computing and Communication Systems, ICACCS 2021, pp.46~50 (2021)
- Anki, P., "High Accuracy Conversational AI Chatbot Using Deep Recurrent Neural Networks Based on BiLSTM Model", 2020 3rd International Conference on Information and Communications Technology, ICOIACT 2020, pp.382~387 (2020)
- Mou, "Deep recurrent neural networks for hyperspectral image classification", IEEE Transactions on Geoscience and Remote Sensing, 55(7), pp.3639~3655 (2017)[DOI]
- Shuai Li, "Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN", 2018 IEEE, 2018, 5457~5466[DOI]
- Ronald J. Williams, "A Learning Algorithm for Continually Running Fully Recurrent Neural Networks", Neural Computation, 1(2), pp.270~280 (1989), MIT Press - Journals[DOI]
- Jeff Donahue, "Long-term recurrent convolutional networks for visual recognition and description", 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.2625~2634 (2015), IEEE
- Carlene Lebeuf, "Software Bots", IEEE Software, 35(1), pp.18~23 (2018), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Gwendal Daniel, "Xatkit: A Multimodal Low-Code Chatbot Development Framework", IEEE Access, 8, pp.15332~15346 (2020), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Abbas Saliimi Lokman, "Modern Chatbot Systems: A Technical Review", Advances in Intelligent Systems and Computing, 881, pp.1012~1023 (2018), Springer International Publishing[DOI]
- Shafquat Hussain, "A Survey on Conversational Agents", Chatbots Classification and Design Techniques, 2019, 946~956[DOI]
- Rafael Mellado-Silva, "Learning tax regulations through rules-based chatbots using decision trees: a case study at the time of COVID-19", 2020 39th International Conference of the Chilean Computer Science Society (SCCC) (2020), IEEE
- Vinod Kumar Shukla, "Enhancing LMS Experience through AIML Base and Retrieval Base Chatbot using R Language", 2019 International Conference on Automation, Computational and Technology Management (ICACTM), pp.561~567 (2019), IEEE
- Vishal Dutt, "Dynamic Information Retrieval With Chatbots: A Review of Artificial Intelligence Methodology", 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA), pp.1299~1303 (2020), IEEE
- Enza Varghese, "A Standalone Generative Conversational Interface Using Deep Learning", 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), pp.1915~1920 (2018), IEEE
- Amber Nigam, "Intent Detection and Slots Prompt in a Closed-Domain Chatbot", 2019 IEEE 13th International Conference on Semantic Computing (ICSC), pp.340~343 (2019), IEEE
- Stephen Roller, "Recipes for Building an Open-Domain Chatbot", Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp.300~325 (2021), Association for Computational Linguistics
- "IEEE Computational Intelligence Magazine", 2018, 55~75, pp.Institute of Electrical and Electronics Engineers (IEEE), 10.1109[DOI]
- Ruineihart, D. E., "Learning Internal Representations Error Propagation", Cognitive Science, 1(V) (1986)
- Hochreiter, S., "Long Short-Term Memory", Neural Computation, 9(8), pp.1735~1780 (1997)[DOI]
- Chung, J., "Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling", Presented in NIPS 2014 Deep Learning and Representation Learning Workshop (2014)
- Ying Shen, "Improving Medical Short Text Classification with Semantic Expansion Using Word-Cluster Embedding", Information Science and Applications 2018, pp.401~411 (2018), Springer Singapore
- Cao, W., "Stacked residual recurrent neural network with word weight for text classification", IAENG International Journal of Computer Science, 44(3), pp.277~284 (2017)
- 김선미, "단어 연관성 가중치를 적용한 연관 문서 추천 방법", 멀티미디어학회논문지, 22(2), pp.250~259 (2019), 한국멀티미디어학회[DOI]
- Sepp Hochreiter, "The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions", International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 06(02), pp.107~116 (1998), World Scientific Pub Co Pte Lt[DOI]
- Talathi, S. S., "Improving performance of recurrent neural network with relu nonlinearity", arXiv preprint arXiv:1511.03771, ~ (2015)