본문 바로가기
KERD
논문KCI 등재

사람들은 언제 팩트체크를 믿는가?: 검증 주체와 확증 편향의 영향

When Do People Trust Fact-Check Messages?: Effects of Fact-Checking Agents and Confirmation Bias

김현석|이종민|이은주
언론정보연구소 (Institute of Communication Research)|2023.08|언론정보연구 (Journal of Communication Research)|제60권 제3호|pp. 47-88|DOI 10.22174/jcr.2023.60.3.47

0

조회

0

다운로드

3

인용

0

참고문헌

키워드

인공지능artificial intelligence (AI)전문가confirmation bias확증 편향expertcrowdsourcing크라우드소싱팩트체크인공지능 휴리스틱에 대한 믿음belief in AI heuristicfact-checking
주제분류신문방송학

초록

이 연구는 팩트체크 검증 주체와 확증 편향이 팩트체크 메시지의 효과성에 미치는 영향을 온라인 실험을 통해 검증했다(N = 504). 구체적으로, 동일한 팩트체크 메시지에 대해 팩트체크를 사람(전문가 혹은 크라우드소싱)이 수행했다고 안내받았을 때와 인공지능이 수행했다고 안내받았을 때 이용자들의 판정 결과 수용 정도, 메시지 공신력 평가 및 공유 의도가 차이를 보이는지, 그리고 자신의 정치적 성향과 일치하는 판정 결과를 제시하는 팩트체크 메시지에 대해 우호적으로 반응하는 확증편향이 언제 강하게 나타나는지 살펴봤다. 연구 결과, 첫째, 팩트체크 메시지의 효과성과 관련해 확증 편향이 견고하게 작동했다. 참가자들은 팩트체크 메시지의 판정 결과가 자신의 정치 성향과 일치할 때 그렇지 않을 때보다 판정 결과를 더 잘 받아들였고, 메시지의 공신력을 더 높게 평가했으며, 메시지를 다른 사람들과 더 적극적으로 공유하고자 했다. 둘째, 팩트체크 주체가 메시지 공신력 평가와 공유 의도에 미치는 영향이 판정 결과의 정치 성향 일치도에 따라 달라졌다. 판정 결과가 자신의 정치 성향과 불일치할 때는 전문가 팩트체크 메시지보다 인공지능 팩트체크 메시지에 더 긍정적인 반응을 보인 반면, 일치할 때는 팩트체크 주체에 따른 차이가 없었다. 셋째, 판정 결과의 정치 성향 일치도, 팩트체크 주체, 인공지능 휴리스틱에 대한 믿음 사이의 삼원상호작용이 판정 결과 수용 및 메시지 공신력 평가에 영향을 미쳤다. 판정 결과 수용에 있어서는 확증 편향이 팩트체크 주체가 인공지능 혹은 전문가일 때 크라우드소싱일 때보다 약화되는 경향이 인공지능 휴리스틱에 대한 믿음이 강할수록 두드러졌다. 반면, 메시지 공신력 평가와 관련해서는 확증 편향이 팩트체크 주체가 인공지능 또는 크라우드소싱일 때 전문가일 때보다 약화되는 양상이 인공지능 휴리스틱에 대한 믿음이 약할수록 더욱 뚜렷하게 나타났다.

Abstract

An online experiment (N = 504) investigated how fact-checking agents (artificial intelligence [AI] agent vs. experts vs. crowdsourced individuals) and confirmation bias affect the effectiveness of fact-check messages, in terms of acceptance of verdicts, perceived credibility of fact-check messages, and intentions to share fact-check messages. Results demonstrated the robust operation of confirmation bias, such that participants were more likely to accept the verdict, evaluated the message as more credible, and were more willing to share the fact-check message that was politically congruent (vs. incongruent) with their political ideology. Moreover, the effects of fact-checking agent on perceived message credibility and sharing intention varied depending on the message congruence. With incongruent verdicts, AI-generated fact-check messages elicited more favorable reactions than those by experts, whereas no agent effect was observed with congruent verdicts. Furthermore, a three-way interaction among message congruence, fact-checking agent, and belief in AI heuristic emerged for the acceptance of verdict and perceived message credibility. Specifically, AI fact-checker (vs. crowdsourced fact-checkers) attenuated the confirmation bias in the acceptance of verdict, but such a tendency was found only among those with stronger belief in AI heuristic. For perceived message credibility, however, the tendency for AI fact-checker (vs. experts) to mitigate the confirmation bias was more evident among those with weaker belief in AI heuristic.

참고문헌 (66)

  1. 문재완, "언론보도에 대한 징벌적 손해배상의 헌법적 한계", 외법논집, 45(3), pp.1~26 (2021), 법학연구소[DOI]
  2. 윤성옥, "언론의 징벌적 손해배상제 반대론에 관한 검토", 한국방송학보, 36(3), pp.1~31 (2022), 한국방송학회[DOI]
  3. 이완수, "가짜뉴스(fake news)란 무엇인가? - 가짜뉴스 개념과 범위에 대한 다차원적 논의 -", 미디어와 인격권, 4(2), pp.173~214 (2018), 언론중재위원회[DOI]
  4. 2022 언론수용자 조사, "한국언론진흥재단", 한국언론진흥재단, ~ (2022)
  5. 2022 10대 청소년 미디어 이용 조사, "한국언론진흥재단", 한국언론진흥재단, ~ (2022)
  6. 황현정, "오정보(misinformation), 왜 믿는가? 어떻게 정정할 것인가?: 오정보와 그 정정에 대한 이론적 검토", 언론정보연구, 57(4), pp.5~54 (2020), 언론정보연구소[DOI]
  7. Dolores Albarracin, "Misleading Claims About Tobacco Products in YouTube Videos: Experimental Effects of Misinformation on Unhealthy Attitudes", Journal of Medical Internet Research, 20(6), pp.e229~ (2018), JMIR Publications Inc.[DOI]
  8. Hunt Allcott, "Social Media and Fake News in the 2016 Election", Journal of Economic Perspectives, 31(2), pp.211~236 (2017), American Economic Association[DOI]
  9. Alyssa Appelman, "Measuring Message Credibility", Journalism & Mass Communication Quarterly, 93(1), pp.59~79 (2015), SAGE Publications[DOI]
  10. "Media literacy: A report of the national leadership conference on media literacy"[URL]
  11. John A Banas, "When Machine and Bandwagon Heuristics Compete: Understanding Users’ Response to Conflicting AI and Crowdsourced Fact-Checking", Human Communication Research, 48(3), pp.430~461 (2022), Oxford University Press (OUP)[DOI]
  12. Barbera, D. L, "A hybrid human-in-the-loop framework for fact checking", Proceedings of the Sixth Workshop on Natural Language for Artificial Intelligence (NL4AI 2022) co-located with 21th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2022), 3287, pp.1~11 (2022)
  13. W Lance Bennett, "The disinformation order: Disruptive communication and the decline of democratic institutions", European Journal of Communication, 33(2), pp.122~139 (2018), SAGE Publications[DOI]
  14. Brown, T., "Language models are few-shot learners", Advances in neural information processing systems, 33, pp.1877~1901 (2020)[DOI]
  15. Burgoon, J. K., "Expectancy violations theory", pp.1~9 (2015), Wiley-Blackwell
  16. Joseph N. Cappella, "Selection and Transmission Processes for Information in the Emerging Media Environment: Psychological Motives and Message Characteristics", Media Psychology, 18(3), pp.396~424 (2014), Informa UK Limited[DOI]
  17. Joseph N. Cappella, "Interventions to Correct Misinformation about Tobacco Products", Tobacco Regulatory Science, 1(2), pp.186~197 (2015), JCFCorp SG PTE LTD[DOI]
  18. Man-pui Sally Chan, "Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation", Psychological Science, 28(11), pp.1531~1546 (2017), SAGE Publications[DOI]
  19. Joshua Cloudy, "The Str(AI)ght Scoop: Artificial Intelligence Cues Reduce Perceptions of Hostile Media Bias", Digital Journalism, 11(9), pp.1577~1596 (2021), Informa UK Limited[DOI]
  20. Demartini, G, "Human-in-the-loop artificial intelligence for fighting online misinformation: Challenges and opportunities", Bulletin of the Technical Committee on Data Engineering, 43(3), pp.65~74 (2020)
  21. Eloundou, T., "Gpts are gpts : An early look at the labor market impact potential of large language models" (2023)[DOI]
  22. Festinger, L., "A theory of cognitive dissonance" (1957), Stanford University Press
  23. Andreas Graefe, "Automated Journalism: A Meta-Analysis of Readers’ Perceptions of Human-Written in Comparison to Automated News", Media and Communication, 8(3), pp.50~59 (2020), Cogitatio[DOI]
  24. Jimmeka J. Guillory, "Correcting erroneous inferences in memory: The role of source credibility", Journal of Applied Research in Memory and Cognition, 2(4), pp.201~209 (2013), American Psychological Association (APA)[DOI]
  25. Zhijiang Guo, "A Survey on Automated Fact-Checking", Transactions of the Association for Computational Linguistics, 10, pp.178~206 (2022), MIT Press[DOI]
  26. Hoes, E., "Using ChatGPT to fight misinformation:ChatGPT nails 72% of 12,000 verified claims", PsyArXiv (2023)[DOI]
  27. Robert Hornik, "Using Theory to Design Evaluations of Communication Campaigns: The Case of the National Youth Anti-Drug Media Campaign", Communication Theory, 13(2), pp.204~224 (2003), Oxford University Press (OUP)[DOI]
  28. Hox, J. J., "Multilevel analysis:Techniques and applications (3rd ed.)" (2018), Routledge
  29. Hyun Suk Kim, "Attracting Views and Going Viral: How Message Features and News-Sharing Channels Affect Health News Diffusion", Journal of Communication, 65(3), pp.512~534 (2015), Oxford University Press (OUP)[DOI]
  30. Arunima Krishna, "Misinformation About Health: A Review of Health Communication and Misinformation Scholarship", American Behavioral Scientist, 65(2), pp.316~332 (2019), SAGE Publications[DOI]
  31. Ziva Kunda, "The case for motivated reasoning.", Psychological Bulletin, 108(3), pp.480~498 (1990), American Psychological Association (APA)[DOI]
  32. Chei Sian Lee, "News sharing in social media: The effect of gratifications and prior experience", Computers in Human Behavior, 28(2), pp.331~339 (2012), Elsevier BV[DOI]
  33. Eun-Ju Lee, "Gender Stereotyping of Computers: Resource Depletion or Reduced Attention?", Journal of Communication, 58(2), pp.301~320 (2008), Oxford University Press (OUP)[DOI]
  34. Eun-Ju Lee, "What Triggers Social Responses to Flattering Computers? Experimental Tests of Anthropomorphism and Mindlessness Explanations", Communication Research, 37(2), pp.191~214 (2010), SAGE Publications[DOI]
  35. Eun-Ju Lee, "Authenticity Model of (Mass-Oriented) Computer-Mediated Communication: Conceptual Explorations and Testable Propositions", Journal of Computer-Mediated Communication, 25(1), pp.60~73 (2020), Oxford University Press (OUP)[DOI]
  36. Eun-Ju Lee, "How Political Identity and Misinformation Priming Affect Truth Judgments and Sharing Intention of Partisan News", Digital Journalism, 11(1), pp.226~245 (2023), Informa UK Limited[DOI]
  37. Eun-Ju Lee, "Mediated Misinformation: Questions Answered, More Questions to Ask", American Behavioral Scientist, 65(2), pp.259~276 (2019), SAGE Publications[DOI]
  38. Lewandowsky, S, "The debunking handbook 2020" (2020)[DOI]
  39. Bingjie Liu, "Reading Machine-Written News: Effect of Machine Heuristic and Novelty on Hostile Media Perception", Human-Computer Interaction. Theories, Methods, and Human Issues, pp.307~324 (2018), Springer International Publishing[DOI]
  40. Lodge, M., "The rationalizing voter" (2013), Cambridge University Press
  41. Corine S. Meppelink, "“I was Right about Vaccination”: Confirmation Bias and Health Literacy in Online Health Information Seeking", Journal of Health Communication, 24(2), pp.129~140 (2019), Informa UK Limited[DOI]
  42. Molden, D. C., "Motivated thinking", pp.390~410 (2012), Oxford University Press
  43. Won-Ki Moon, "How Can We Fight Partisan Biases in the COVID-19 Pandemic? AI Source Labels on Fact-checking Messages Reduce Motivated Reasoning", Mass Communication and Society, 26(4), pp.646~670 (2022), Informa UK Limited[DOI]
  44. Clifford Nass, "Machines and Mindlessness: Social Responses to Computers", Journal of Social Issues, 56(1), pp.81~103 (2000), Wiley[DOI]
  45. Raymond S. Nickerson, "Confirmation Bias: A Ubiquitous Phenomenon in Many Guises", Review of General Psychology, 2(2), pp.175~220 (1998), SAGE Publications[DOI]
  46. O’Connor, C., "The misinformation age : How false beliefs spread" (2019), Yale University Press
  47. O’Keefe, D. J., "Persuasion: Theory and research (3rd ed.)" (2016), Sage
  48. GPT-4 technical report, "OpenAI", ~ (2023)
  49. "Climate change remains top global threat across 19-country survey"[URL]
  50. Sara Pluviano, "Parents’ beliefs in misinformation about vaccines are strengthened by pro-vaccine campaigns", Cognitive Processing, 20(3), pp.325~331 (2019), Springer Science and Business Media LLC[DOI]
  51. Reeves, B., "The media equation : How people treat computers, television, and new media like real people and places" (1996), Cambridge University Press
  52. Jon Roozenbeek, "Susceptibility to misinformation about COVID-19 around the world", Royal Society Open Science, 7(10), pp.201199~ (2020), The Royal Society[DOI]
  53. Jieun Shin, "Partisan Selective Sharing: The Biased Diffusion of Fact-Checking Messages on Social Media", Journal of Communication, 67(2), pp.233~255 (2017), Oxford University Press (OUP)[DOI]
  54. Brian G. Southwell, "The Prevalence, Consequence, and Remedy of Misinformation in Mass Media Systems", Journal of Communication, 65(4), pp.589~595 (2015), Oxford University Press (OUP)[DOI]
  55. Natalie Jomini Stroud, "The Consequences of Forced Versus Selected Political Media Exposure", Human Communication Research, 45(1), pp.27~51 (2018), Oxford University Press (OUP)[DOI]
  56. S. Shyam Sundar, "Machine Heuristic", Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019), ACM[DOI]
  57. Tandoc, E. C. Jr., "Defining"fake news" : A typology of scholarly definitions", Digital Journalism, 6(2), pp.137~153 (2018)[DOI]
  58. Emily Thorson, "Belief Echoes: The Persistent Effects of Corrected Misinformation", Political Communication, 33(3), pp.460~480 (2015), Informa UK Limited[DOI]
  59. Sander van der Linden, "Misinformation: susceptibility, spread, and interventions to immunize the public", Nature Medicine, 28(3), pp.460~467 (2022), Springer Science and Business Media LLC[DOI]
  60. T. Franklin Waddell, "A Robot Wrote This?", Digital Journalism, 6(2), pp.236~255 (2017), Informa UK Limited[DOI]
  61. Nathan Walter, "Evaluating the Impact of Attempts to Correct Health Misinformation on Social Media: A Meta-Analysis", Health Communication, 36(13), pp.1776~1784 (2020), Informa UK Limited[DOI]
  62. Nathan Walter, "Fact-Checking: A Meta-Analysis of What Works and for Whom", Political Communication, 37(3), pp.350~375 (2019), Informa UK Limited[DOI]
  63. Nathan Walter, "How to unring the bell: A meta-analytic approach to correction of misinformation", Communication Monographs, 85(3), pp.423~441 (2018), Informa UK Limited[DOI]
  64. Nathan Walter, "A Meta-Analytic Examination of the Continued Influence of Misinformation in the Face of Correction: How Powerful Is It, Why Does It Happen, and How to Stop It?", Communication Research, 47(2), pp.155~177 (2019), SAGE Publications[DOI]
  65. Sai Wang, "Moderating Uncivil User Comments by Humans or Machines? The Effects of Moderation Agent on Perceptions of Bias and Credibility in News Content", Digital Journalism, 9(1), pp.64~83 (2020), Informa UK Limited[DOI]
  66. B. E. Weeks, "Electoral Consequences of Political Rumors: Motivated Reasoning, Candidate Rumors, and Vote Choice during the 2008 U.S. Presidential Election", International Journal of Public Opinion Research, 26(4), pp.401~422 (2014), Oxford University Press (OUP)[DOI]

관련 논문AI추천

뉴스신뢰도, 뉴스관여도와 확증편향이 소셜 커뮤니케이션 행위에 미치는 영향: 가짜뉴스와 팩트뉴스 수용자 비교김미경한국정치커뮤니케이션학회 · 정치커뮤니케이션연구2019 · 제52호19대 대선 기간 후보자 간 의혹제기에 대한 팩트체크 뉴스의 설득효과: 팩트체크 뉴스 판정결과와 지지후보를 중심으로김선호, 백영민언론정보연구소 · 언론정보연구2018 · 제55권 제1호코로나19 가짜 뉴스(fake news) 인식 및 태도에 미치는 팩트 체크 저널리즘의 효과: 팩트 체크 뉴스 유형과 댓글 유형을 중심으로김민경한국헬스커뮤니케이션학회 · 헬스커뮤니케이션연구2020 · 제19권 제2호AI는 과연 정파성을 완화할 수 있는가?: 원자력 홍보 메시지에서 AI 출처의 명시가 공중의 정파적 편향에 미치는 영향신별, 김가람, 조수영한국PR학회 · PR연구2025 · 제29권 제4호텔레비전 뉴스에서 팩트체크 코너가 미디어 신뢰에 미치는 영향: 정보 유용성의 매개 효과안지수언론정보연구소 · 언론정보연구2019 · 제56권 제4호이용자의 확증편향과 설득지식 수준이 가짜뉴스 허구성 판단에 미치는 영향유현중, 정현주한국광고PR실학회 · 광고PR실학연구2020 · 제13권 제4호

이 간행물의 다른 논문 (2건)

AI 챗봇 서비스 이용과 디지털 정보격차: ‘구삐’ 이용자(탈북민·일반인)의 심층인터뷰를 중심으로최종환, 장병희pp. 5-46KBS 수신료 인상 논의는 어떤 이슈와 결합하는가?: 시기별 공론 수준 비교를 중심으로이수범, 이준영pp. 89-134
원문 보기