초등 인공지능 프로그래밍 교육 환경을 위한 Sentence-BERT 기반 단문 분류
Sentence-BERT-based Short Sentence Classification for Elementary AI Programming Educational Environments
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 논문에서는 Sentence-BERT 바이-인코더에 기반한 단문 분류 방법을 제안하고, 이를 초등 인공지능 프로그래 밍 교육 환경에서 텍스트 학습 및 추론 시 효과적으로 적용할 수 있음을 실험적으로 보인다. 본 단문 분류 방법은 신경망을 실제로 처음부터 학습시키거나 미세 조정을 하지 않기 때문에, ‘가상-학습’ 및 ‘가상-추론’ 방법이라고 명 명하였다. 가상-학습 단계에서는 학습할 단문들을 경량 바이-인코더를 이용하여 임베딩하고, 이를 효과적으로 검색 할 수 있도록 인덱스를 생성한다. 가상-추론 단계에서는 추론할 단문들을 동일한 바이-인코더를 이용하여 임베딩하 고, FAISS, HNSW 등과 같은 근사 인접 이웃 알고리즘을 활용하여 빠르게 유사한 임베딩 벡터를 찾아 레이블 예 측을 수행한다. 이 방식을 초등 인공지능 교육에서 많이 활용되는 LSTM과 비교한 결과, 클래스 별 학습 샘플의 수 가 많지 않은 상황에서 본 방식은 LSTM보다 정확도가 크게 앞서는 것으로 나타났다. 그리고 학생들이 저성능의 컴퓨터를 활용하는 환경에서 충분히 활용할 수 있는 수준으로 학습 시간과 추론 시간이 소요됨을 확인하였다.
Abstract
In this paper, we present a short sentence classification approach based on a Sentence-BERT bi-encoder and experimentally demonstrate its effectiveness for training text data in elementary AI programming educational environments. Because this approach does not train from scratch or fine-tune a neural network, we have termed it the "pseudo-training" and "pseudo-inference" method. In the pseudo-training phase, the short sentences to be trained are embedded using a lightweight bi-encoder, and an index is generated for efficient retrieval. In the pseudo-inference phase, the short sentences to be inferred are embedded using the same bi-encoder, and approximate nearest neighbor algorithms such as FAISS and HNSW are utilized to quickly find similar embedding vectors, followed by label prediction. Compared to LSTM, which is widely used in elementary AI education, the proposed method showed higher accuracy in situations where the number of training samples per class is limited. Additionally, we confirmed that the training and inference time for our approach is sufficient for practical use in programming environments where students use low-performance computers.
참고문헌 (28)
- Han, A., "Entry: visual programming to enhance children's computational thinking", Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp.73~76 (2015)
- Lane, D., "Machine learning for kids: A project-based introduction to artificial intelligence" (2021), No Starch Press
- Druga, S., "Growing up with AI: Cognimates:from coding to teaching machines", Doctor, pp.2018
- Park, Y., "Tooee : A novel scratch extension for K-12 big data and artificial intelligence education using text-based visual blocks", IEEE Access, 9, pp.149630~149646 (2021)
- Reimers, N., "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", Proceedings of the 2019Conference on Empirical Methods in Natural Language Processing (2019)
- Thakur, N., "Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks", Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.296~310 (2021)
- Zhu, K., "An educational approach to machine learning with mobile applications", Doctor, pp.2019
- Van Brummelen, J. J. R., "Tools to create and democratize conversational artificial intelligence", Doctor, pp.2019
- Maloney, J., "The Scratch programming language and environment", ACM Transactions on Computing Education, 10(4), pp.1~15 (2010)
- Carney, M., "Teachable machine: Approachable web-based tool for exploring machine learning classification", Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, pp.1~8 (2020)
- Wolber, D., "App Inventor" (2011), O'Reilly Media, Inc
- Park, Y., "Comparing the effectiveness of Scratch and App Inventor with regard to learning computational thinking concepts", Electronics, 8(11), pp.1269~ (2019)
- Park, Y., "Novel Scratch programming blocks for web scraping", Electronics, 11(16), pp.2584~ (2022)
- Park, Y., "A block-based interactive programming environment for large-scale machine learning education", Applied Sciences, 12(24), pp.13008~ (2022)
- Park, Y., "Text processing education using a block-based programming language", IEEE Access, 10, pp.128484~128497 (2022)
- Park, Y., "Adaptive Bi-Encoder Model Selection and Ensemble for Text Classification", Mathematics, 12(19), pp.3090~ (2024)
- Vaswani, A., "Attention is all you need", Advances in Neural Information Processing Systems (2017)
- Improving language understanding by generative pre-training, "Radford, A.", ~ (2018)
- Bekamiri, H., "PatentsBERTa : A deep NLP based hybrid model for patent distance and classification using augmented SBERT", Technological Forecasting and Social Change, 206, pp.123536~ (2024)
- "All-MiniLM-L12-v2 model"[URL]
- Malkov, Y. A., "Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs", IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4), pp.824~836 (2018)
- Socher, R., "Recursive deep models for semantic compositionality over a sentiment treebank", Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp.1631~1642 (2013)
- Zhang, X., "Character-level convolutional networks for text classification", Advances in Neural Information Processing Systems, 28, pp.649~657 (2015)
- Park, Y., "Using multiple monolingual models for efficiently embedding Korean and English conversational sentences", Applied Sciences, 13(9), pp.5771~ (2023)