Evaluation of the Applicability of ChatGPT in Dental Hygiene Education

Abstract

Background: This study compared the performance of different versions of generative artificial intelligence (AI) using items from the National Examination for Dental Hygienists and evaluated the potential and limitations of ChatGPT as a learning support tool in dental hygiene education. Methods: A total of 200 items from the 2021 National Examination for Dental Hygienists were analyzed. For ChatGPT-4 and ChatGPT-4o, both Korean and English versions of the items were entered to calculate accuracy. Problem-solving items were also converted into open-ended questions to assess response accuracy and classify error types. Results: ChatGPT-4o demonstrated higher overall accuracy than ChatGPT-4, with accuracy exceeding 75% for Korean items. Accuracy for English items was lower, with a notable decline in the Medical Law and Regulations domain. Although ChatGPT-4o showed improved performance on open-ended problem-solving items, several responses still contained logical and informational errors. Conclusion: Accuracy improved with newer ChatGPT model versions, particularly for recall-oriented items, although substantial performance differences remained between Korean and English inputs. Persistent logical errors in problem-solving items indicate limited suitability for tasks requiring clinical judgment. These findings highlight the need for students to critically evaluate AI-generated responses and for educators to remain attentive to the rapid evolution of generative AI to ensure its appropriate, continuously updated use in dental hygiene education.