오픈 피어 리뷰 환경에서 학술 논문 심사 결과와 영향력 지표 간의 관련성에 관한 연구
A Study on the Relationship between the Review Results of Articles and Impact Metrics in an Open Peer Review Platform
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 연구는 대표적인 OPR(Open Peer Review) 플랫폼인 F1000Research에서 사회과학분야의 논문 585건을 대상으로 개방형 동료 심사 결과를 분석하고 피인용, 알트메트릭스와 어떠한 관련성을 보이는지 확인하였다. 더불어 논문의 활용이 피인용에 미치는 영향 관계 내에서 심사 점수가 조절효과(Moderating effect)를 나타내는지 검증함으로써, OPR 환경에서 고품질로 평가된 논문이 피인용을 촉진할 수 있는지 확인하였다. 분석 결과 첫 번째, 승인과 조건부 승인된 논문 그룹 간에 피인용 횟수에 유의미한 차이가 나타나지 않았지만, 환산된 심사 점수와 피인용 횟수는 유의한 정(+)의 상관성(r= 0.40 ~ 0.60)을 나타냈다. 두 번째, 심사 점수는 알트메트릭스와도 약한 상관성을 나타내 심사자의 품질 평가 결과는 피인용과 사회적 영향을 약하게 예측할 수 있는 것으로 분석되었다. 마지막으로 심사 점수는 논문의 활용을 피인용으로 이어지게 하는데 유의한 양의 방향의 조절효과(B=1.69, P < 0.01)를 수행하며, 조건부 효과 검사 결과 가장 고품질로 평가된 논문 집단에서 가장 큰 효과(B=11.32, 95% CI [10.57, 12.08])를 나타내는 것으로 검증되었다. 따라서 공개된 심사 결과는 연구자들의 우수 논문 선별을 도와 인용을 유도하는데 도움을 줄 수 있는 것으로 파악되었다.
Abstract
This study analyzed the open peer review results for 585 papers in the field of social sciences in F1000Research, a representative OPR(Open Peer Review) platform, and checked the relationship between the number of cited-by, altmetrics and review score. In addition, by verifying whether the review score shows a moderating effect between the relationship between the utilization of the paper and the cited-by, it was confirmed whether the paper evaluated as high quality in the open review platform can promote the number of cited-by. As a result of the analysis, first, there was no significant difference in the number of cited-by between the approved and conditionally approved paper groups, but the converted review score and the number of cited-by showed a weak positive correlation (r = 0.40 - 0.60). Second, the review score showed a weak correlation with the altmetrics, and it was analyzed that review result could weakly predict the number of cited-by and social impact. Finally, it was verified that the review score performed a significant positive moderating effect (B=1.69, P < 0.01) in making the use of the paper lead to citation. As a result of the conditional effect test, it was verified that it showed the greatest effect(B=11.32, 95% CI [10.57, 12.08]) in the group of papers rated as the highest quality. Therefore, it was analyzed that the open review scores can help researchers select high quality papers and induce citations.
참고문헌 (28)
- 김지영, "메가 OA 학술지 국내 저자의 오픈 피어 리뷰 인식에 관한 연구", 정보관리학회지, 37(4), pp.131~150 (2020), 한국정보관리학회[DOI]
- 정용일, "학술연구정보의 오픈 액세스 관행에 대한 인식 연구: 과학기술분야학회의 개방형 동료심사 관행을 중심으로", 정보통신학회 종합학술대회논문집, pp.8~21 (2022)
- 허원무, "매개효과 분석 방법의 최근 트랜드 : 부트스트래핑을 이용한 단순, 다중, 이중매개효과분석 방법", 한국비즈니스리뷰, 6(3), pp.43~59 (2013)
- "Transparency, recognition, and innovation in peer review in the life sciences:peer review survey results"[URL]
- "The 7 biggest problems facing science, according to 270 scientists" (2016)[URL]
- Lutz Bornmann, "The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000", Journal of Informetrics, 7(2), pp.286~291 (2013), Elsevier BV[DOI]
- Bornmann, L., "Closed versus open reviewing of journal manuscripts : how far do comments differ in language use", Scientometrics, 91, pp.843~856 (2012)[DOI]
- "European Commission awards contract for setting up an open access publishing platform" (2020)[URL]
- "F1000Research" (2023)[URL]
- Hayes, A. F, "Introduction to Mediation, Moderation, and Conditional Process Analysis:A Regression-Based Approach" (2017), Guilford Press
- Huggett, S., "F1000Research journal rankings: an alternative way to evaluate the scientific impact of scholarly communications", Research Trends, 1(26) (2012)
- Janowicz, K., "Open and transparent : the review process of the semantic web journal", Learned Publishing, 25(1), pp.48~55 (2012)[DOI]
- Khan, K., "Is open peer review the fairest system?", BMJ, 341, pp.c6425~ (2010)[DOI]
- Merton, R. K., "Matthew effect in science : reward and communication systems of science are considered", Science, 159(3810), pp.56~63 (1968)
- Mulligan, A., "Peer review in a changing world : an international study measuring the attitudes of researchers", Journal of American Society for Information Science and Technology, 64, pp.132~161 (2013)[DOI]
- Ross-Hellauer, T., "What is open peer review? a systematic review", F1000Research Research, 6, pp.1~37 (2017)[DOI]
- Smith, R., "Peer review : a flawed process at the heart of science and journals", Journal of the Royal Society of Medicine, 99, pp.178~182 (2006)[DOI]
- Thelwall, M., "Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? evidence from the F1000Research post-publication open peer review publishing model", Journal of Information Science, 47(6), pp.809~820 (2021)[DOI]
- Tsakonas, G., "Open science cannot succeed without open peer review", LIBER Quarterly : The Journal of the Association of European Research Libraries, 31(1), pp.1~6 (2021)[DOI]
- van Rooyen, S., "Effect of open peer review on quality of reviews and on reviewers' recommendations : a randomised trial", British Medical Journal, 318(7175), pp.23~ (1999)[DOI]
- "How rigorous is the post-publication review process at F1000 research?" (2013)[URL]
- Walsh, E., "Open peer review : a randomised controlled trial", The British Journal of Psychiatry, 176(1), pp.47~51 (2000)[DOI]
- Waltman, L., "F1000Research recommendations as a potential new data source for research evaluation : a comparison with citations", Journal of the Association for Information Science & Technology, 65(3), pp.433~445 (2014)
- Wang, P., "F1000Research Prime recommended articles and their citations : an exploratory study of four journals", Scientometrics, 122, pp.933~955 (2020)[DOI]
- Williams, A. E., "F1000 : an overview and evaluation", Information and Learning Sciences, 118(7-), pp.1108~ (2017)
- Dietmar Wolfram, "Open peer review: promoting transparency in open science", Scientometrics, 125(2), pp.1033~1051 (2020), Springer Science and Business Media LLC[DOI]
- Zong, Q., "The relationship of polarity of post-publication peer review to citation count : evidence from publons", Online Information Review, 44(3), pp.583~602 (2020)[DOI]
- Zong, Q., "Does open peer review improve citation count? evidence from a propensity score matching analysis of PeerJ", Scientometrics, 125, pp.607~623 (2020)[DOI]