Improving the Performance of Text Categorization Models used for the Selection of High Quality Articles
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Objectives: Machine learning systems can considerably reduce the time and effort needed by experts to perform new systematic reviews (SRs). This study investigates categorization models, which are trained on a combination of included and commonly excluded articles, which can improve performance by identifying high quality articles for new procedures or drug SRs. Methods: Test collections were built using the annotated reference files from 19 procedure and 15 drug systematic reviews. The classification models, using a support vector machine, were trained by the combined even data of other topics, excepting the desired topic. This approach was compared to the combination of included and commonly excluded articles with the combination of included and excluded articles. Accuracy was used for the measure of comparison. Results: On average, the performance was improved by about 15% in the procedure topics and 11% in the drug topics when the classification models trained on the combination of articles included and commonly excluded, were used. The system using the combination of included and commonly excluded articles performed better than the combination of included and excluded articles in all of the procedure topics. Conclusions: Automatically rigorous article classification using machine learning can reduce the workload of experts when they perform systematic reviews when the topic-specific data are scarce. In particular, when the combination of included and commonly excluded articles is used, this system will be more effective.
Abstract
Objectives: Machine learning systems can considerably reduce the time and effort needed by experts to perform new systematic reviews (SRs). This study investigates categorization models, which are trained on a combination of included and commonly excluded articles, which can improve performance by identifying high quality articles for new procedures or drug SRs. Methods: Test collections were built using the annotated reference files from 19 procedure and 15 drug systematic reviews. The classification models, using a support vector machine, were trained by the combined even data of other topics, excepting the desired topic. This approach was compared to the combination of included and commonly excluded articles with the combination of included and excluded articles. Accuracy was used for the measure of comparison. Results: On average, the performance was improved by about 15% in the procedure topics and 11% in the drug topics when the classification models trained on the combination of articles included and commonly excluded, were used. The system using the combination of included and commonly excluded articles performed better than the combination of included and excluded articles in all of the procedure topics. Conclusions: Automatically rigorous article classification using machine learning can reduce the workload of experts when they perform systematic reviews when the topic-specific data are scarce. In particular, when the combination of included and commonly excluded articles is used, this system will be more effective.
참고문헌 (18)
- Sackett DL., "Evidence based medicine : what it is and what it isn't", BMJ, 312, pp.71~72 (1996)
- Aphinyanaphongs Y., "Text categorization models for highquality article retrieval in internal medicine", J Am Med Inform Assoc, 12, pp.207~216 (2005)
- Matwin S., "A new algorithm for reducing the workload of experts in performing systematic reviews", J Am Med Inform Assoc, 17, pp.446~453 (2010)
- "About Cochrane systematic reviews and protocols" (2012)[URL]
- Cohen AM., "Cross-topic learning for work prioritization in systematic review creation and update", J Am Med Inform Assoc, 16, pp.690~704 (2009)
- "Committee for New Health Technology Assessment" (2012)[URL]
- Koch G, "No improvement-still less than half of the Cochrane reviews are up to date", 14th Cochrane Colloquium (2006)
- Chalmers I., "Avoidable waste in the production and reporting of research evidence", Lancet, 374, pp.86~89 (2009)
- Cohen AM., "Reducing workload in systematic review preparation using automated citation classification", J Am Med Inform Assoc, 13, pp.206~219 (2006)
- "Systematic drug class review gold standard data" (2010)[URL]
- "API reference" (2000)[URL]
- Porter MF., "An algorithm for suffix stripping", Program, 14, pp.130~137 (1980)
- Cohen AM., "Optimizing feature representation for automated systematic review work prioritization", AMIA Annu Symp Proc, 2008, pp.121~125 (2008)
- Joachims T., "Text categorization with support vector machines: learning with many relevant features", Proceedings of the 10th European Conference on Machine Learning, pp.137~142 (1998)
- Kilicoglu H., "Towards automatic recognition of scientifically rigorous clinical research evidence", J Am Med Inform Assoc, 16, pp.25~31 (2009)
- "Support vector machine: SVMlight" (2008)[URL]
- Joachims T., "Making large-scale support vector machine learning practical, In Advances in kernel methods", pp.169~184 (1999), MIT Press
- Lee YH., "Overcoming small-size training set problem in content-based recommendation : a collaboration-based training set expansion approach", Proceedings of the 11th International Conference on Electronic Commerce, pp.99~106 (2009)