Evaluation of Term Ranking Algorithms for Pseudo- Relevance Feedback in MEDLINE Retrieval
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Objectives: The purpose of this study was to investigate the effects of query expansion algorithms for MEDLINE retrieval within a pseudo-relevance feedback framework. Methods: A number of query expansion algorithms were tested using various term ranking formulas, focusing on query expansion based on pseudo-relevance feedback. The OHSUMED test collection,which is a subset of the MEDLINE database, was used as a test corpus. Various ranking algorithms were tested in combination with different term re-weighting algorithms. Results: Our comprehensive evaluation showed that the local context analysis ranking algorithm, when used in combination with one of the reweighting algorithms – Rocchio, the probabilistic model, and our variants – significantly outperformed other algorithm combinations by up to 12% (paired t-test; p < 0.05). In a pseudo-relevance feedback framework, effective query expansion would be achieved by the careful consideration of term ranking and re-weighting algorithm pairs, at least in the context of the OHSUMED corpus. Conclusions: Comparative experiments on term ranking algorithms were performed in the context of a subset of MEDLINE documents. With medical documents, local context analysis, which uses co-occurrence with all query terms, significantly outperformed various term ranking methods based on both frequency and distribution analyses. Furthermore, the results of the experiments demonstrated that the term rank-based re-weighting method contributed to a remarkable improvement in mean average precision.
Abstract
Objectives: The purpose of this study was to investigate the effects of query expansion algorithms for MEDLINE retrieval within a pseudo-relevance feedback framework. Methods: A number of query expansion algorithms were tested using various term ranking formulas, focusing on query expansion based on pseudo-relevance feedback. The OHSUMED test collection,which is a subset of the MEDLINE database, was used as a test corpus. Various ranking algorithms were tested in combination with different term re-weighting algorithms. Results: Our comprehensive evaluation showed that the local context analysis ranking algorithm, when used in combination with one of the reweighting algorithms – Rocchio, the probabilistic model, and our variants – significantly outperformed other algorithm combinations by up to 12% (paired t-test; p < 0.05). In a pseudo-relevance feedback framework, effective query expansion would be achieved by the careful consideration of term ranking and re-weighting algorithm pairs, at least in the context of the OHSUMED corpus. Conclusions: Comparative experiments on term ranking algorithms were performed in the context of a subset of MEDLINE documents. With medical documents, local context analysis, which uses co-occurrence with all query terms, significantly outperformed various term ranking methods based on both frequency and distribution analyses. Furthermore, the results of the experiments demonstrated that the term rank-based re-weighting method contributed to a remarkable improvement in mean average precision.
참고문헌 (33)
- Hersh WR, "Information retrieval: a health and biomedical perspective. 2nd ed" (2003), Springer
- Nankivell C, "Networked information and clinical decision making: the experience of Birmingham Heartlands and Solihull National Health Service Trust (Teaching)", Med Educ, 35, pp.167~172 (2001)
- Haux R, "Knowledge retrieval as one type of knowledge-based decision support in medicine: results of an evaluation study", Int J Biomed Comput, 41, pp.69~85 (1996)
- Bernstam EV, "Using citation data to improve retrieval from MEDLINE", J Am Med Inform Assoc, 13, pp.96~105 (2006)
- Vechtomova O, "A study of the effect of term proximity on query expansion", J Inf Sci, 32, pp.324~333 (2006)
- Exploring query expansion strategies for MEDLINE, "Srinivasan P", Cornell University, ~, pp.1528 (1995)
- Srinivasan P, "Retrieval feedback in MEDLINE", J Am Med Inform Assoc, 3, pp.157~167 (1996)
- Hersh W, "Ohsumed: an interactive retrieval evaluation and new large test collection for research", Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.192~201 (1994), Dublin, IE: Springer-Verlag Inc
- Yu H, "Enabling Multi-Level Relevance Feedback on PubMed by Integrating Rank Learning and Processing", BMC BIOINFORMATICS, 11(Suppl 2), pp.S6~ (2010), BioMed Central
- States DJ, "MiSearch adaptive PubMed search tool", Bioinformatics, 25, pp.974~976 (2009)
- Yoo S, "On the query reformulation technique for effective MEDLINE document retrieval", J Biomed Inform, 43, pp.686~693 (2010)
- Aronson AR, "Exploiting a large thesaurus for information retrieval", Proceedings of Recherche d'Information Assistee par Ordinateur (RIAO), pp.197~216 (1994), New York: ACM
- Yang Y, "An application of least squares fit mapping to text information retrieval", Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.281~290 (1993), Pittsburgh, PA: ACM
- Yang Y, "Words or concepts: the features of indexing units and their optimal use in information retrieval", Proc Annu Symp Comput Appl Med Care, pp.685~689 (1993)
- Yang Y, "Expert network: effective and efficient learning from human decisions in text categorization and retrieval", Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.13~22 (1994), Dublin, IE: Springer-Verlag Inc
- Hersh W, "Assessing thesaurusbased query expansion using the UMLS metathesaurus", Proceeding of the AMIA Symposium, pp.344~348 (2000), Los Angeles, CA: Hanley & Belfus
- Chu WW, "A knowledge-based approach for retrieving scenario-specific medical text documents", Control Eng Pract, 13, pp.1105~1121 (2005)
- Efthimiadis EN, "Query expansion", Ann Rev Inf Sci Tech, 31, pp.121~187 (1996)
- Harman D, "Relevance feedback revisited", Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.1~10 (1992), Copenhagen, DK: ACM
- Efthimiadis EN, "UCLA-Okapi at TREC-2: query expansion experiments", Proceedings of 2nd Text Retrieval Conference, pp.200~215 (1994), Gaithersburg, MD: National Institute of Standards and Technology
- Carpineto C, "An information theoretic approach to automatic query expansion", ACM Trans Inf Syst, 19, pp.1~27 (2001)
- Fan W, "Tuning before feedback: combining ranking discovery and blind feedback for robust retrieval", Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.138~145 (2004), Sheffield, UK: 2004. New York, NY: ACM
- Xu J, "Improving the effectiveness of information retrieval with local context analysis", ACM Trans Inf Syst, 18, pp.79~112 (2000)
- Implementation of the SMART information retrieval system, "Buckley C", Cornell University, ~, pp.TR85-686 (1985)
- Lovins JB, "Development of a stemming algorithm", Mech Transl Comput Linguist, 11, pp.22~31 (1968)
- Robertson SE, "Okapi", Keenbow at TREC-8, 2000, 151~162[DOI]
- Baeza-Yates R, "Modern information retrieval" (1999), ACM Press
- Robertson SE, "On relevance weight estimation and query expansion", J Doc, 42, pp.182~188 (1986)
- Robertson SE, "On term selection for query expansion", J Doc, 46, pp.359~364 (1990)
- Rocchio JJ, "Relevance feedback in information retrieval, in The SMART retrieval system: Englewood Cliffs", pp.313~323 (1971), Prentice-Hall
- Ide E, "New experiments in relevance feedback, in The SMART retrieval system: Englewood Cliffs", pp.337~354 (1971), Prentice-Hall
- Carpineto C, "Improving retrieval feedback with multiple term-ranking function combination", ACM Trans Inf Syst, 20, pp.259~290 (2002)
- Hull D, "Using statistical testing in the evaluation of retrieval experiments", Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.329~338 (1993), Pittsburgh, PA: ACM