문항반응이론을 적용한 1급 응급구조사 국가시험 대비 모의시험의 적정성 연구
A study of the adequate number of questions in a mock test for the paramedic national examination using item response theory
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Purpose: To adjust item numbers in a national test, this study used item response theory to examine changes in average scores, reliability, difficulty, and discrimination according to the adjustment of item numbers. Methods: We analyzed the dichotomous coding of correct and incorrect answers of 473 examinees in a mock test conducted in 2023. Additionally, as an explanatory pilot study, we used an online questionnaire to survey experts on their perceptions of the appropriate item numbers for each test subject from January 18, 2024, to February 15, 2024. Results: Regarding the item numbers on the national exam, experts preferred to reduce the number of management of emergency patients (33.14±6.09, p<.05) and advanced emergency medical care: subtopics (104.49±11.55, p<.05), and the total number of questions (217.82±20.95, p<.05). In a simulation set in which items with low item fit were removed after fitting a two-parameter item response theory model, reliability was maintained at .910 until the 5th test consisting of 185 questions with little loss of difficulty, discrimination, and average score, and there was no correlation between item numbers and average score. Conclusion: Experts responded that reducing the number of items on the national exam was appropriate. As a result of the item reduction simulation, there was no significant loss in the average score, difficulty, discrimination, or reliability. More reliable results could be obtained if the results were based on a validity analysis and analyzed using actual national exams.
Abstract
Purpose: To adjust item numbers in a national test, this study used item response theory to examine changes in average scores, reliability, difficulty, and discrimination according to the adjustment of item numbers. Methods: We analyzed the dichotomous coding of correct and incorrect answers of 473 examinees in a mock test conducted in 2023. Additionally, as an explanatory pilot study, we used an online questionnaire to survey experts on their perceptions of the appropriate item numbers for each test subject from January 18, 2024, to February 15, 2024. Results: Regarding the item numbers on the national exam, experts preferred to reduce the number of management of emergency patients (33.14±6.09, p<.05) and advanced emergency medical care: subtopics (104.49±11.55, p<.05), and the total number of questions (217.82±20.95, p<.05). In a simulation set in which items with low item fit were removed after fitting a two-parameter item response theory model, reliability was maintained at .910 until the 5th test consisting of 185 questions with little loss of difficulty, discrimination, and average score, and there was no correlation between item numbers and average score. Conclusion: Experts responded that reducing the number of items on the national exam was appropriate. As a result of the item reduction simulation, there was no significant loss in the average score, difficulty, discrimination, or reliability. More reliable results could be obtained if the results were based on a validity analysis and analyzed using actual national exams.
참고문헌 (26)
- "Emergency medical service act"[URL]
- "National exam analysis results"[URL]
- "Health professional categories"[URL]
- Lee SE, "Curriculum and educational evaluation", pp.290~341 (2022), KNOU press
- Yim CI, "Educational technology", pp.249~250 (2022), KNOU press
- Development of practice-based integrated question type for national nursing examination, "Kim MY", Korea Health Personnel Licensing Examination institute, ~ (2023)
- Sung TJ, "Changes in reliability coefficient and test information function according to changes in number of items, item difficulty, and item discrimination", J Curric Eval, 6(2), pp.123~154 (1993)
- Yim MK, "Testing unidimensionality and goodness-of-fitness for the application of item response theory to the Korean Medical Licensing Examination", Korean J Med Educ, 19(2), pp.163~169 (2007)[DOI]
- Jeong GH, "Applicability of item response theory to the Korean Nurses’ Licensing Examination", J Educ Eval Health Prof, 22(1), pp.23~29 (2005)[DOI]
- Kim AJ, "Review of the qualification criteria for the national examinations for emergency medical technicians", Korean J Emerg Med Ser, 25(2), pp.39~53 (2021)[DOI]
- Kim MY, "Correlations between the scores of computerized adaptive testing, paper and pencil tests, and the Korean Medical Licensing Examination", J Educ Eval Health Prof, 2(1), pp.113~118 (2005)[DOI]
- Yoou SK, "Proposal of a new EMT national registry exam using the Delphi method", Korean J Emerg Med Ser, 14(2), pp.57~69 (2010)
- Yoou SK, "Proposal of a new emergency medical technician national practical examination", Korean J Emerg Med Ser, 16(3), pp.45~62 (2012)[DOI]
- Analysis of the appropriate number of questions that satisfy the validity and confidence level of the test, "Park HG", Korea Health Personnel Licensing Examination Institute, ~ (2022)
- Kim YH, "Trend analysis of the results of the national examination for EMT-paramedics for the Past 9 Years", Journal of Health Informatics and Statistics, 47(2), pp.133~138 (2022)[DOI]
- Brogden HE, "Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation", Psychometrika, 11(4), pp.197~214 (1946)
- "Academic stress and psychological well-being among college students"[URL]
- Suruchi S, "Test item analysis and relationship between difficulty level and discrimination index of test items in an achievement test in biology", PIJR, 3(6), pp.56~58 (2014)
- Smoleń T, "The quadratic relationship between difficulty of intelligence test items and their correlations with working memory", Front Psychol, 6, pp.156499~ (2015)
- "Emotional intelligence, test anxiety and academic stress among university students" (2018)[URL]
- Kaur M, "Item analysis of in use multiple choice questions in pharmacology", Int J Appl Basic Med Res, 6(3), pp.170~173 (2016)[DOI]
- Development of practice-based integrated question type for national nursing examination, "Kim MY", Korea Health Personnel Licensing Examination Institute, ~ (2023)
- Seong TJ, "Educational evaluation", pp.259~264 (2019), Haksisa
- Seong TJ, "Understanding and applying item response theory", pp.30~57 (2019), Kyoyookbook
- "Item response theory"[URL]
- Lim EY, "Comparison of item analysis results of Korean Medical Licensing Examination according to classical test theory and item response theory", J Educ Eval Health Prof, 1(1), pp.67~76 (2004)[DOI]