나라장터 입찰공고 인공지능 검색모델 개발에 관한 연구
A Study on the Development of AI Search Model for G2B Tender Notice
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
나라장터(g2b.go.kr)의 검색 기능은 매우 부정확하여 많은 사용자들이 불편을 느끼고 있다. 입찰에 참여하려는 조달 기업은 입찰공고의 검색에 많은 시간을 소비하여 업무시간의 낭비를 초래하고있다. 본 논문에서는 나라장터 입찰공고의 불편하고 부정확한 검색기능을 대체할 신경망 기반의 정보검색 모델을 도출하는 방안을 연구하였다. 자연어 검색에 획기적인 발전을 가져온 BERT를 기반으로검색결과를 리랭킹함으로써 사용자의 만족도를 향상시키고자 하였다. 신경망 기반 IR모델 학습에 필요한 데이터를 전처리하기 위하여 기존의 연구를 토대로 문서를 구절로 분할하고, 분할된 구절에서 키워드를 추출하여 의사-질의를 생성하였다. 생성된 의사-질의를 입력으로 엘라스틱 서치의 BM25모델을 사용하여 의사-레이블인 질의-문서쌍(q-d pair)을 생성하였다. 학습용 데이터셋의 양과 유형, 배치크기, 러닝 레이트, positive 라벨링 비율 등의 하이퍼파라미터 값 변화에 따른 다양한 실험을 통하여입찰공고 데이터에 적합한 IR모델을 개발하였다. 개발된 IR모델로 실제 입찰공고 조회시 현행 나라장터보다는 훨씬 정확한 검색결과를 보여주었다.
Abstract
The searching service of Nara Market(g2b.go.kr) is so inaccurate that many users feel uncomfortable. Procurement companies that want to participate in bidding are spending a lot of time searching for tender notices, resulting in waste of business hours. In this paper, we study the method of deriving a neural network-based information retrieval model to replace the uncomfortable and inaccurate search service of the Nara Market tender notice. We tried to improve user satisfaction by reranked search results based on BERT, which brought remarkable development in information retrieval. In order to preprocess the data required for neural network-based IR model training, document was divided into passages based on existing studies, and keywords were extracted from the passages to generate pseudo-queries. Q-d pairs were generated as pseudo-label using BM25 model of Elastic Search with the generated pseudo-queries. An IR model suitable for tender notice data developed through various experiments according to changes in hyperparameter values such as the amount and type of learning dataset, batch size, learning rate, and positive labeling ratio. The developed IR model showed much more accurate search results than the current Nara Market.
참고문헌 (44)
- "나라장터운영실적"[URL]
- 조달청, "차세대국가종합전자조달시스템(나라장터)구축전략 및 이행방안수립", 제안요청서, ~ (2020)
- Devlin, J., "Bert: Pre-training of deep bidirectional transformers for language understanding", arXiv preprint arXiv:1810.04805, ~ (2018)
- Zamani, H., "Neural query performance prediction using weak supervision from multiple signals", The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (2018)
- Lin, J., "Pretrained transformers for text ranking : Bert and beyond", Synthesis Lectures on Human Language Technologies, 14(4), pp.1~325 (2021)
- Hofstätter, S., "Let's measure run time! Extending the IR replicability infrastructure to include performance aspects", arXiv preprint arXiv:1907.04614, ~ (2019)
- Asadi, N., "Pseudo test collections for learning web search ranking functions", Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval (2011)
- Berendsen, R., "Pseudo test collections for training and tuning microblog rankers", Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval (2013)
- MacAvaney, S., "An approach for weakly-supervised deep information retrieval", arXiv preprint arXiv:1707.00189, ~ (2017)
- Ma, J., "Zero-shot neural passage retrieval via domain-targeted synthetic question generation", arXiv preprint arXiv:2004.14503, ~ (2020)
- Dai, Z., "Deeper text understanding for IR with contextual neural language modeling", Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (2019)
- Broder, A. Z., "Efficient query evaluation using a two-level retrieval process", Proceedings of the twelfth international conference on Information and knowledge management (2003)
- Mothe, J., "Analytics Methods to Understand Information Retrieval Effectiveness—A Survey", Mathematics, 10(12), pp.2135~ (2022)
- Potthast, M., "The dilemma of the direct answer", ACM SIGIR Forum (2021), ACM New York
- Izacard, G., "Towards unsupervised dense information retrieval with contrastive learning", arXiv preprint arXiv:2112.09118, ~ (2021)
- Robertson, S. E., "Okapi at TREC-3", pp.109~ (1995), Nist Special Publication Sp
- Joachims, T., "Optimizing search engines using clickthrough data", Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining (2002)
- Radlinski, F., "Query chains: learning to rank from implicit feedback", Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining (2005)
- Burges, C., "Learning to rank using gradient descent", Proceedings of the 22nd international conference on Machine learning (2005)
- Huang, P. -S., "Learning deep structured semantic models for web search using clickthrough data", Proceedings of the 22nd ACM international conference on Information & Knowledge Management (2013)
- Shen, Y., "Learning semantic representations using convolutional neural networks for web search", Proceedings of the 23rd international conference on world wide web (2014)
- Palangi, H., "Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval.IEEE", ACM Transactions on Audio, 2016, pp.24(4), 694~707
- Nogueira, R., "Passage Re-ranking with BERT", arXiv preprint arXiv:1901.04085, ~ (2019)
- Nguyen, T., "MS MARCO: A human generated machine reading comprehension dataset", CoCo@NIPs, ~ (2016)
- Gillick, D., "End-to-end retrieval in continuous space", arXiv preprint arXiv:1811.08008, ~ (2018)
- Karpukhin, V., "Dense passage retrieval for open-domain question answering", arXiv preprint arXiv:2004.04906, ~ (2020)
- Xiong, L., "Approximate nearest neighbor negative contrastive learning for dense text retrieval", arXiv preprint arXiv:2007.00808, ~ (2020)
- Johnson, J., "Billion-scale similarity search with gpus", IEEE Transactions on Big Data, 7(3), pp.535~547 (2019)
- Humeau, S., "Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring", arXiv preprint arXiv:1905.01969, ~ (2019)
- Khattab, O., "Relevance-guided supervision for openqa with colbert", Transactions of the Association for Computational Linguistics, 9, pp.929~944 (2021)
- Luan, Y., "Sparse, dense, and attentional representations for text retrieval", Transactions of the Association for Computational Linguistics, 9, pp.329~345 (2021)
- Gautier, I., "A memory efficient baseline for open domain question answering", arXiv preprint arXiv: 2012.15156, ~ (2020)
- Mikolov, T., "Distributed representations of words and phrases and their compositionality", Advances in neural information processing systems, 26 (2013)
- Jernite, Y., "Discourse-based objectives for fast unsupervised sentence representation learning", arXiv preprint arXiv:1705. 00557, ~ (2017)
- Lan, Z., "Albert: A lite bert for self-supervised learning of language representations", arXiv preprint arXiv:1909.11942, ~ (2019)
- Guu, K., "Retrieval augmented language model pre-training", International Conference on Machine Learning (2020), PMLR
- Lewis, M., "Pre-training via paraphrasing", Advances in Neural Information Processing Systems, 33, pp.18470~18481 (2020)
- Dehghani, M., "Neural ranking models with weak supervision", Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval (2017)
- Xu, P., "Passage ranking with weak supervision", arXiv preprint arXiv:1905.05910, ~ (2019)
- Liang, D., "Embedding-based zero-shot retrieval through query generation", arXiv preprint arXiv:2009. 10270, ~ (2020)
- "How to Extract Relevant Keywords with KeyBERT"[URL]
- MacAvaney, S., "CEDR: Contextualized embeddings for document ranking", Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval (2019)
- Eunyoung Choi, "논문 및 특허 데이터를 활용한 자연어 처리 기반 ICT 동향 분석 연구", 정보화연구, 17(2), pp.127~140 (2020), 한국엔터프라이즈아키텍처학회[DOI]
- Chang, S., "Improving effectiveness of neural information retrieval model using weak-supervision based on keyword extraction", doctor, pp.2022