정보 검색 과제별 동적 검색 랭킹 모델 구현 및 검증: 사용자 중심 적합성 판단 모형 평가를 중심으로
Implementation and Verification of Dynamic Search Ranking Model for Information Search Tasks: The Evaluation of Users' Relevance Judgement Model
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 연구는 정보 검색 과제별 주요 적합성 판단 기준을 실제 정보 검색 시스템으로 구현해 보고 사용자 평가를 통해 그 효과를 검증해 보고자 하였다. 이를 위해, 사용자 적합성 판단 기준들을 정보 검색 시스템에서 적합성을 결정하는 검색 랭킹 모델의 랭킹 요소들로 적용하였다. 그리고 정보 검색 과제별 차이가 있는 동적 검색 랭킹 모델과 차이가 없는 정적 검색 랭킹 모델을 시스템으로 구현하였고, 이에 대한 사용자 평가를 진행하여 비교해 보았다. 총 45명의 참가자가 실험에 참여하였고, 정보 검색 과제별 차이가 있는 동적 검색 랭킹 모델과 차이가 없는 정적 검색 랭킹 모델이 적용된 각각의 검색 시스템에서 3개의 검색 과제를 수행하였다. 3개의 정보 검색 과제로는 사실 검색 과제, 문제 해결 검색 과제, 의사 결정 검색 과제가 사용되었다. 각 참가자는 검색 결과 첫 페이지 상위 5 개의 검색 결과에 대해 적합성 정도를 7 점 척도로 평가하였다. 그 결과, 사용자는 전반적으로 모든 검색어에 동일하게 반응하는 정적 검색 랭킹 모델을 적용한 시스템보다 정보 검색 과제별로 사용자 적합성 판단 기준의 변화에 따라 랭킹 요소 가중치를 달리한 동적 검색 랭킹 모델을 더 높이 평가하는 것을 확인할 수 있었다. 본 연구는 이를 통해, 정보 검색 과제를 고려한 정보 검색 시스템 디자인의 필요성과 함께, 사용자 중심 적합성 판단 모형 연구 결과를 실제 정보 검색 시스템으로 구현하여 평가함으로써 사용자 중심 적합성 연구 결과의 타당성을 검증하였다는 점, 그리고 사용자 연구 접목을 통한 시스템 개선의 중요성을 강조하였다는 점에서 의의를 가진다.
Abstract
The purpose of this research was to implement and verify an information retrieval(IR) system based on users'relevance criteria for information search tasks. For this purpose, we implemented an IR system with a dynamic ranking model using users' relevance criteria varying with the types of information search task and evaluated this system through user experiment. 45 participants performed three information search tasks on both IR systems with a static and a dynamic ranking model. Three Information search tasks are fact finding search task, problem solving search task and decision making search task. Participants evaluated top five search results on 7 likert scales of relevance. We observed that the IR system with a dynamic ranking model provided more relevant search results compared to the system with a static ranking model. This research has significance in designing IR system for information search tasks, in testing the validity of user-oriented relevance judgement model by implementing an IR system for actual information search tasks and in relating user research to the improvement of an IR system.
참고문헌 (41)
- Agichtein, E., "Improving Web search ranking by incorporating user behavior", Proceedings of the 29th Annual International ACM SIGIR, pp.19~26 (2006)
- Baeza-Yates. R, "Modern information retrieval" (1999), Addison-Wesley
- Barry, C. L, "User-defined relevance criteria: An exploratory study", Journal of the American Society for Information Science, 45, pp.149~159 (1994)
- Barry, C. L, "User-defined relevance criteria: A comparison of two studies", Proceedings of the American Society for Information Science, pp.103~111 (1998), InformationToday, Inc
- Bateman, J, "Changes in relevance criteria: A longitudinal study", ASIS Proceedings, 1998, pp.23~32 (1998)
- Borlund, P, "The concept of relevance in IR", Journal of the American Society for Information Science and Technology, 54(10), pp.913~925 (2003)
- Choi, Y, "Users' relevance criteria in image retrieval in American history", Information Processing & Management, 38, pp.695~726 (2002)
- Clarke, C.L.A, "Novelty and diversity in information retrieval evaluation", Proceedings of the 31th Annual International ACM SIGIR, pp.659~666 (2008)
- Cosijn, E, "Dimensions of relevance", Information Processing & Management, 36, pp.533~550 (2000)
- Cuadra, C. A, "Opening the black box of relevance", Journal of Documentation, 23, pp.291~303 (1967)
- Cutler, M., "Using the structures of html documents to improveretrieval", Proceedings of the USENIX Symposiumon Internet Technologies and Systems, pp.241~251 (1997)
- Drori, O, "Algorithm for documents ranking: Idea and simulation results", Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering. ACM International Conference Proceeding Series, 27, pp.99~102 (2002)
- Fitzgerald, M. A, "Relevance judging, evaluation, and decision making in virtual library: A descriptive study", Journal of the American Society for Information Science and Technology, 52, pp.989~1010 (2001)
- Geng, X, "Query dependent ranking using K-nearest neighbor", Proceedings of the 31th Annual International ACM SIGIR, pp.115~122 (2008)
- Hirsh, S. G, "Children's relevance criteria and information seeking on electronic resources", Journal of the American Society for Information Science, 50, pp.1265~1283 (1999)
- Ingwersen, P, "The turn: Integration of information seeking and retrieval in context" (2005), Springer
- Järvelin, K, "Cumulated gain-based evaluation of ir techniques", ACM Transactions on Information Systems, 20, pp.422~446 (2002)
- Joachims, T, "Optimizing search engines using clickthrough data", Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pp.133~142 (2002)
- Kang, I. H, "Query type classification for web document retrieval", Proceedings of the 26th annual international ACM SIGIR, pp.64~71 (2003)
- Kekäläinen, J, "Evaluating information retrieval systems under the challenges of interaction and multidimensional dynamic relevance, In Emerging Frameworks and Methods", pp.253~270 (2002), Libraries Unlimited
- Kelly, D, "Features of documents relevant to task and fact-oriented questions", Proceedings of the Eleventh International Conference on Information and Knowledge Management, pp.645~647 (2002), ACM
- Limberg, L, "Experiencing information seeking and learning: a study of the interaction between two phenomena", Information Research, 5(1) (1999)[DOI]
- Maglaughlin, K. L, "User perspective on relevance criteria: A comparison among relevant, partially relevant, and not-relevant", Journal of the American Society for Information Science and Technology, 53, pp.327~342 (2002)
- Mizzaro, S, "Relevance: The whole history", Journal of the American Society for Information Science, 48, pp.810~832 (1997)
- 박정아, "정보검색에서의 사용자 중심 적합성 판단 모형", 감성과학, 12(4), pp.489~500 (2009), 한국감성과학회
- Park, T. K, "The nature of relevance in information retrieval: An empirical study", Library Quarterly, 63, pp.318~351 (1993)
- Robertson, S, "Overview of the okapi projects", Journal of Documentation, 53(1), pp.3~7 (1997)
- A field experiment approach to the study of relevance assessments in relation to document searching, 2 vols, "Rees, A. M", Center for Documentation and Communication Research, School of Library Science, Case Western Reserve University, ~ (1967)
- Salton, G, "The SMART Retrieval System-Experiments in Automatic Document Processing" (1971), Prentice-Hall, Inc
- Salton, G, "evaluation of indexing and text processing", Journal of the Association for Computing Machinery, 15, pp.8~36 (1968)
- Saracevic, T, "concept of relevance in information science: A historical review, In Introduction to information science", pp.111~151 (1970), R.R. Bowker
- Saracevic, T, "Relevance: A review of and a framework for the thinking on the notion in information science", Journal of the American Society for Information Science, 26, pp.321~343 (1975)
- Saracevic, T, "Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: Nature and manifestations of relevance", Journal of the American Society for Information Science and Technology, 58, pp.1915~1933 (2007)
- Savolainen, R, "User-defined relevance criteria in web searching", Journal of Documentation, 62(6), pp.685~707 (2006)
- Schamber, L, "Users’ criteria for evaluation in a multimedia environment", Proceedings of the 54th ASIS Annual Meeting, 28, pp.126~133 (1991)
- Spink, A., "From highly relevant to not relevant : Examining different regions of relevance", Information Processing & Management, 34, pp.599~621 (1998)
- Spink, A, "Web search: Public searching of the Web" (2004), Kluwer Academic Publishers
- Su, L. T, "A comprehensive and systematic model of user evaluation of Web search engines II: An evaluation by undergraduates", Journal of the American Society for Information and Technology, 54, pp.1193~1223 (2003)
- Tang, R, "Towards an understanding of the dynamics of relevance judgments: An analysis of one person's search behavior", Information Processing & Management, 34, pp.237~256 (1998)
- Vaughan, L, "New measurements for search engine evaluation proposed and tested", Information Processing & Management, 40, pp.677~691 (2004)
- Xu, Y, "Relevance judgment - What do information consumers consider beyond topicality?", Journal of the American Society for Information Science and Technology, 57(7), pp.961~973 (2006)