Chat Generated Pre-Trained Transformer를 통해 얻은 알레르기 비염 정보를 믿을 수 있을까?
Can We Trust the Information on Allergic Rhinitis From Chat Generated Pre-Trained Transformer?
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Background and Objectives Chat Generated Pre-Trained Transformer (ChatGPT) is alarge language model, which allows consumers to get information with one simple questionfor free. There are few studies that reported the reliability and usefullness of ChatGPT in thefield of otorhinolarynoglogy, so we would like to investigate the reliability of informationabout allergic rhinitis generated by ChatGPT. Materials and Method We asked ChatGPT 35 questions related to allergic rhinitis based onthe textbook from Korean Society of Otorhinolaryngology-Head and Neck Surgery, the clini-cal guidelines of American Academy of Otolaryngology-Head and Neck Surgery, and theguidelines of Allergic Rhinitis and Its impact on Asthma, and recorded the generated answers. Five specialists from our department were shown the answers in a blind test to assess their re-liability. Each question was rated either ‘accurate’ or ‘inaccurate,’ and if the contents were notfound in the textbook or the guideline, it was rated as ‘unreliable.’Results Of the 35 questions, 26 (74%) were identified as ‘accurate,’ 9 (26%) were ‘inaccu-rate,’ and 0 (0%) were ‘unreliable.’ Questions about epidemiology, causes, diagnosis, andprognosis were found to be more accurate than the average, whereas definitions and treat-ments were less accurate than the average. Conclusion The information about allergic rhinitis generated by ChatGPT was quite reli-able, showing that ChatGPT can be helpful in understanding and treating the disease. It isnecessary to use the developing medical artificial intelligence wisely.
Abstract
Background and Objectives Chat Generated Pre-Trained Transformer (ChatGPT) is alarge language model, which allows consumers to get information with one simple questionfor free. There are few studies that reported the reliability and usefullness of ChatGPT in thefield of otorhinolarynoglogy, so we would like to investigate the reliability of informationabout allergic rhinitis generated by ChatGPT. Materials and Method We asked ChatGPT 35 questions related to allergic rhinitis based onthe textbook from Korean Society of Otorhinolaryngology-Head and Neck Surgery, the clini-cal guidelines of American Academy of Otolaryngology-Head and Neck Surgery, and theguidelines of Allergic Rhinitis and Its impact on Asthma, and recorded the generated answers. Five specialists from our department were shown the answers in a blind test to assess their re-liability. Each question was rated either ‘accurate’ or ‘inaccurate,’ and if the contents were notfound in the textbook or the guideline, it was rated as ‘unreliable.’Results Of the 35 questions, 26 (74%) were identified as ‘accurate,’ 9 (26%) were ‘inaccu-rate,’ and 0 (0%) were ‘unreliable.’ Questions about epidemiology, causes, diagnosis, andprognosis were found to be more accurate than the average, whereas definitions and treat-ments were less accurate than the average. Conclusion The information about allergic rhinitis generated by ChatGPT was quite reli-able, showing that ChatGPT can be helpful in understanding and treating the disease. It isnecessary to use the developing medical artificial intelligence wisely.
참고문헌 (18)
- "ChatGPT: Optimizing language models for dialogue"[URL]
- Else H, "Abstracts written by ChatGPT fool scientists", Nature, 613(7944), pp.423~ (2023)
- van Dis EAM, "ChatGPT : Five priorities for research", Nature, 614(7947), pp.224~226 (2023)
- Brożek JL, "Allergic rhinitis and its impact on asthma(ARIA)guidelines-2016 revision", J Allergy Clin Immunol, 140(4), pp.950~958 (2017)
- Kim YH, "Clinical diagnostic guidelines for allergic rhinitis : Diagnosis", J Korean Med Assoc, 60(1), pp.81~88 (2017)
- Bauchau V, "Prevalence and rate of diagnosis of allergic rhinitis in Europe", Eur Respir J, 24(5), pp.758~764 (2004)
- Yeo YH, "Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma", Clin Mol Hepatol, 29(3), pp.721~732 (2023)
- Sarraju A, "Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model", JAMA, 329(10), pp.842~844 (2023)
- Lee BJ, "Otorhinolaryngology-Head and Neck Surgery", pp.237~247 (2018), Koonja Publishing Inc
- Kim SH, "Otorhinolaryngology-Head and Neck Surgery", pp.249~269 (2018), Koonja Publishing Inc
- Seidman MD, "Clinical practice guideline : Allergic rhinitis", Otolaryngol Head Neck Surg, 152(1 Suppl), pp.S1~S43 (2015)
- Hamet P, "Artificial intelligence in medicine", Metabolism, 69S, pp.S36~S40 (2017)
- Han ER, "Medical education trends for future physicians in the era of advanced technology and artificial intelligence : An integrative review", BMC Med Educ, 19(1), pp.460~ (2019)
- Jayakumar P, "Comparison of an artificial intelligence–enabled patient decision aid vs educational material on decision quality, shared decision-making, patient experience, and functional outcomes in adults with knee osteoarthritis : A randomized clinical trial", JAMA Netw Open, 4(2), pp.e2037107~ (2021)
- Dimov V, "Utilizing social networks, blogging and YouTube in allergy and immunology practices", Expert Rev Clin Immunol, 11(10), pp.1065~1068 (2015)
- Remvig CL, "YouTube as a source of(mis)information on allergic rhinitis", Ann Allergy Asthma Immunol, 129(5), pp.612~617 (2022)
- Madathil KC, "Healthcare information on YouTube : A systematic review", Health Informatics J, 21(3), pp.173~194 (2014)