ChatGPT가 한국 신경과 의사들에게 실제 임상에서 유용할까?
Will ChatGPT be Useful for Korean Neurologists in Clinical Practice?
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Background: Since the emergence of artificial intelligence (AI), such as ChatGPT, there has been extensive research on its utility in the medical field. This study evaluates the potential of utilizing the ChatGPT AI platform to provide specialized advice to Korean neurologists. Methods: Through a multifaceted approach, it assesses both the knowledge and practical clinical utility using neurology training evaluation questions and clinical case reports. Results: The accuracy of ChatGPT in answering medical exam questions (45.1%) is comparable to or slightly lower than previous studies. The study also examines performance of ChatGPT in providing advice based on clinical case reports, finding that while it can offer suitable diagnostic suggestions and propose relevant tests in some cases (8/12), its overall diagnostic accuracy is limited (2/12). Limitations include the platform’s text-based nature, which hinders the interpretation of image-based questions, and the uneven distribution of case reports across subfields. Conclusions: Despite these limitations, ongoing advancements in AI technology offer prospects for improved performance and expanded applications in the medical field, warranting continued research and monitoring of AI’s role in medicine.
Abstract
Background: Since the emergence of artificial intelligence (AI), such as ChatGPT, there has been extensive research on its utility in the medical field. This study evaluates the potential of utilizing the ChatGPT AI platform to provide specialized advice to Korean neurologists. Methods: Through a multifaceted approach, it assesses both the knowledge and practical clinical utility using neurology training evaluation questions and clinical case reports. Results: The accuracy of ChatGPT in answering medical exam questions (45.1%) is comparable to or slightly lower than previous studies. The study also examines performance of ChatGPT in providing advice based on clinical case reports, finding that while it can offer suitable diagnostic suggestions and propose relevant tests in some cases (8/12), its overall diagnostic accuracy is limited (2/12). Limitations include the platform’s text-based nature, which hinders the interpretation of image-based questions, and the uneven distribution of case reports across subfields. Conclusions: Despite these limitations, ongoing advancements in AI technology offer prospects for improved performance and expanded applications in the medical field, warranting continued research and monitoring of AI’s role in medicine.
참고문헌 (37)
- "Introducing ChatGPT"[URL]
- Nature, "Tools such as ChatGPT threaten transparent science; here are our ground rules for their use", Nature, 613, pp.612~ (2023)
- Else H, "Abstracts written by ChatGPT fool scientists", Nature, 613, pp.423~ (2023)
- Gordijn B, "ChatGPT : evolution or revolution?", Med Health Care Philos, 26, pp.1~2 (2023)
- Gilson A, "How does ChatGPT perform on the United States medical licensing examination (USMLE)? The implications of large language models for medical education and knowledge assessment", JMIR Med Educ, 9, pp.e45312~ (2023)
- Chen TC, "Assessment of ChatGPT’s performance on neurology written board examination questions", BMJ Neurol Open, 5, pp.e000530~ (2023)
- Hirosawa T, "Diagnostic accuracy of differential-diagnosis lists generated by generative pretrained transformer 3 Chatbot for clinical vignettes with common chief complaints : a pilot study", Int J Environ Res Public Health, 20, pp.3378~ (2023)
- Potapenko I, "Artificial intelligence-based chatbot patient information on common retinal diseases using ChatGPT", Acta Ophthalmol, 101, pp.829~831 (2023)
- Johnson SB, "Using ChatGPT to evaluate cancer myths and misconceptions : artificial intelligence and cancer information", JNCI Cancer Spectr, 7, pp.pkad015~ (2023)
- Yeo YH, "Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma", Clin Mol Hepatol, 29, pp.721~732 (2023)
- Choi JH, "Neuromyelitis optica spectrum disorder after COVID-19 infection", J Korean Neurol Assoc, 42, pp.35~39 (2024)
- Choo YS, "Hyperammonemic encephalopathy caused by the c.386+5G>A mutation in OTC gene in a young adult woman", J Korean Neurol Assoc, 42, pp.62~65 (2024)
- Eun MY, "Coexistence of anti-Hu and anti-SOX1autoantibodies in atezolizumab-related encephalitis", J Korean Neurol Assoc, 42, pp.48~52 (2024)
- Je Y, "Atypical presentation of Elsberg syndrome caused by herpes simplex virus type 2 infection", J Korean Neurol Assoc, 42, pp.27~30 (2024)
- Jeong BK, "Reversible homonymous hemianopia associated with focal hyperperfusion in hyperglycemic state", J Korean Neurol Assoc, 42, pp.57~61 (2024)
- Kim HJ, "Non-cirrhotic hyperammonemic encephalopathy with portosystemic shunt", J Korean Neurol Assoc, 42, pp.53~56 (2024)
- Kim N, "Optic neuritis after infliximab treatment in a patient with ulcerative colitis", J Korean Neurol Assoc, 42, pp.31~34 (2024)
- Ko PW, "A case of spinal muscular atrophy type 3a showing improvement in neurofilament light chain of cerebrospinal fluid : real-world evidence", J Korean Neurol Assoc, 42, pp.40~43 (2024)
- Lee JS, "Maternally inherited diabetes and deafness presenting with memory impairment and gait disturbance", J Korean Neurol Assoc, 42, pp.71~75 (2024)
- Lee S, "Atypical paroxysmal kinesigenic dyskinesia with paroxysmal exerciseinduced dyskinesia", J Korean Neurol Assoc, 42, pp.66~70 (2024)
- Shin J, "Primary central nervous system lymphoma mimicking lacunar infarction", J Korean Neurol Assoc, 42, pp.23~26 (2024)
- Song Y, "Anti-NMDA receptor encephalitis presenting with language impairment and bradycardia", J Korean Neurol Assoc, 42, pp.44~47 (2024)
- Kung TH, "Performance of ChatGPT on USMLE : potential for AI-assisted medical education using large language models", PLOS Digit Health, 2, pp.e0000198~ (2023)
- Humar P, "ChatGPT is equivalent to first-year plastic surgery residents : evaluation of ChatGPT on the plastic surgery in-service examination", Aesthet Surg J, 43, pp.NP1085~NP1089 (2023)
- Birkett L, "Performance of ChatGPT on a primary FRCA multiple choice question bank", Br J Anaesth, 131, pp.e34~e35 (2023)
- Mihalache A, "Performance of an artificial intelligence Chatbot in ophthalmic knowledge assessment", JAMA Ophthalmol, 141, pp.589~597 (2023)
- Nakhleh A, "ChatGPT’s response to the diabetes knowledge questionnaire : implications for diabetes education", Diabetes Technol Ther, 25, pp.571~573 (2023)
- Hopkins BS, "ChatGPT versus the neurosurgical written boards : a comparative analysis of artificial intelligence", machine learning performance on neurosurgical board-style questions, 2023, pp.139, 904~911
- Taesotikul S, "ChatGPT vs pharmacy students in the pharmacotherapy time-limit test : a comparative study in Thailand", Curr Pharm Teach Learn, 16, pp.404~410 (2024)
- Li DJ, "Comparing the performance of ChatGPT GPT-4, Bard, and Llama-2 in the Taiwan psychiatric licensing examination and in differential diagnosis with multi-center psychiatrists", Psychiatry Clin Neurosci, 78, pp.347~352 (2024)
- Lin KC, "Integration and assessment of ChatGPT in medical case reporting : a multifaceted approach", Eur J Investig Health Psychol Educ, 14, pp.888~901 (2024)
- Lee PY, "Use of ChatGPT in medical research and scientific writing", Malays Fam Physician, 18, pp.58~ (2023)
- Shieh A, "Assessing ChatGPT 4.0’s test performance and clinical diagnostic accuracy on USMLE STEP 2 CK and clinical case reports", Sci Rep, 14, pp.9330~ (2024)
- Fang C, "How does ChatGPT-4preform on non-English national medical licensing examination? An evaluation in Chinese language", PLOS Digit Health, 2, pp.e0000397~ (2023)
- Lee H, "The rise of ChatGPT : exploring its potential in medical education", Anat Sci Educ, 17, pp.926~931 (2024)
- Kianian R, "Can ChatGPT aid clinicians in educating patients on the surgical management of glaucoma?", J Glaucoma, 33, pp.94~100 (2024)