고등학생들을 대상으로 한 개정 시스템 사고 검사도구(Re_STMI)의 타당도 검증
Validation of the Revised Systems ThinkingMeasuring Instrument(Re_STMI) for HighSchool Students
1
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
The purpose of this study is to verify the validity of the revised STMI, which is a revised and supplemented versionof the Systems Thinking Measuring Instrument (STMI) developed by Lee et al.(2013) for high school students. Forthis purpose, we used data from 475 high school students' responses to 20 Revised STMI questions and 20 STSscale questions. Validity verification includes item response analysis (Item Reliability, Item Map, Infit and OutfitMNSQ, and DIF of male and female groups), exploratory factor analysis (main axis factor analysis using Promax),and confirmatory factor analysis using structural equation model. And central/cross validity verification andcorrelation analysis between the two scales were used. The research results are as follows. First, as a result of theitem response analysis of the Revised STMI 20 questions, Item Reliability was .98 and Infit MNSQ was .84~1.39,and valid results were also analyzed in Item Map and DIF analysis. Second, in the exploratory factor analysis, allsub-questions of the five sub-factors showed factor loading values of .3 or higher, and the reliability of each factorwas .631~.800 and the overall reliability was .863. Third, in the confirmatory factor analysis, all model fit valueswere good (χ2/df: 2.406, CFI: .923, TLI: .908, SRMR: .046, RMSEA: .054), and the convergent validity and crossvalidity were also established. The results showed that the model was suitable. Lastly, in the correlation analysiswith the 20 items of the STS scale, the correlation between all items was .754, and the correlation with STS itemsfor each sub-factor was high, ranging from .426 to .658. Therefore, in the case of the Revised STMI, significantresults were obtained not only in construct validity but also in official validity, and the results of this study showedimproved validity results compared to research results using existing scales. Continuously, additional validationresearch should be conducted so that the Revised STMI can be applied to various groups, including not only highschool students, but also elementary and middle school students, and college students. In addition, ongoingresearch on the relationship between system thinking and other learning-related measures should be activelyconducted using validated instrument.
Abstract
The purpose of this study is to verify the validity of the revised STMI, which is a revised and supplemented versionof the Systems Thinking Measuring Instrument (STMI) developed by Lee et al.(2013) for high school students. Forthis purpose, we used data from 475 high school students' responses to 20 Revised STMI questions and 20 STSscale questions. Validity verification includes item response analysis (Item Reliability, Item Map, Infit and OutfitMNSQ, and DIF of male and female groups), exploratory factor analysis (main axis factor analysis using Promax),and confirmatory factor analysis using structural equation model. And central/cross validity verification andcorrelation analysis between the two scales were used. The research results are as follows. First, as a result of theitem response analysis of the Revised STMI 20 questions, Item Reliability was .98 and Infit MNSQ was .84~1.39,and valid results were also analyzed in Item Map and DIF analysis. Second, in the exploratory factor analysis, allsub-questions of the five sub-factors showed factor loading values of .3 or higher, and the reliability of each factorwas .631~.800 and the overall reliability was .863. Third, in the confirmatory factor analysis, all model fit valueswere good (χ2/df: 2.406, CFI: .923, TLI: .908, SRMR: .046, RMSEA: .054), and the convergent validity and crossvalidity were also established. The results showed that the model was suitable. Lastly, in the correlation analysiswith the 20 items of the STS scale, the correlation between all items was .754, and the correlation with STS itemsfor each sub-factor was high, ranging from .426 to .658. Therefore, in the case of the Revised STMI, significantresults were obtained not only in construct validity but also in official validity, and the results of this study showedimproved validity results compared to research results using existing scales. Continuously, additional validationresearch should be conducted so that the Revised STMI can be applied to various groups, including not only highschool students, but also elementary and middle school students, and college students. In addition, ongoingresearch on the relationship between system thinking and other learning-related measures should be activelyconducted using validated instrument.
참고문헌 (66)
- Aguilar-Cisneros, J. R., "Students' systems thinking competency level detection through software cost estimation concept modeling", Programming & Computer Software, 48, pp.499~512 (2022)
- Bae, B., "Amos 24 Advanced Structural Equation Modeling" (2018), Cheongram
- Bak, B., "Scale revalidation of epistemological beliefs and exploration of their domainality", The Korean Journal of Educational Psychology, 26, pp.739~767 (2012)
- Batt, A. M., "New ways of seeing : Supplementing existing competency framework development guidelines with systems thinking", Advances in Health Sciences Education, 26, pp.1355~1371 (2021)
- Ben-zvi-Assaraf, O., "Development of system thinking skills in the context of earth system education", Journal of Research in Science Teaching, 42, pp.518~560 (2005)
- Ben-zvi-Assaraf, O., "A study of junior high students’ perceptions of the water cycle", Journal of Geoscience Education, 53, pp.366~373 (2005)
- Ben-Zvi-Assaraf, O., "A design based research of an earth systems based environmental curriculum", Eurasia Journal of Mathematics, Science & Technology Education, 5, pp.47~62 (2009)
- Boone, W., "The role of Rasch analysis when conducting science education research utilizing multiple-choice tests", Science Education, 90, pp.253~269 (2006)
- Boone, W., "Rasch Analysis in the Human Sciences" (2014), Springer
- Boone, W., "Using Rasch theory to guide the practice of survey development and survey data analysis in science education and to inform science reform efforts : An exemplar utilizing STEBI self-efficacy data", Science Education, 95, pp.258~280 (2011)
- Cheung, G. W., "Evaluating goodness-of-fit indexes for testing measurement invariance", Structural Equation Modeling, 9, pp.233~255 (2002)
- Cho, O., "The effects of simulation-based education on nursing students’ presence in education, systems thinking and proactivity in problem solving", Journal of Korean Academic Society of Home Health Care Nursing, 23, pp.147~154 (2016)
- Chung, S., "The validity and norms of reading motivation scale for elementary school students", The Korean Journal Child Education, 27, pp.215~240 (2018)
- Clark, S., "Teaching systems thinking to 4th and 5th graders using environmental dashboard display technology", PLoS ONE, 12, pp.1~11 (2017)[DOI]
- Davis A., "The relationship between systems thinking and the new ecological paradigm", Systems Research and Behavioral Science, 33, pp.575~586 (2016)
- de Sousa, L. O., "Teachers' understanding of the interconnectedness of soil and climate change when developing a systems thinking concept map for teaching and learning", International Research in Geographical & Environmental Education, 28, pp.324~342 (2019)
- Dolansky, M. A., "Development and validation of the systems thinking scale", Journal of General Internal Medicine, 35, pp.2314~2320 (2020)
- Emihovich, B., "Game-based learning and systems thinking : An innovative instructional approach for the 21st century", TechTrends : Linking Research & Practice to Improve Learning, 68, pp.174~185 (2024)
- Ha, M., "Examining the validity of history-of-science-based evolution concept assessment and exploring conceptual progressions by contexts", Journal of the Korean Association for Science Education, 36, pp.509~517 (2016)
- Ha, M., "Developing and applying the questionnaire to measure science core competencies based on the 2015 revised national science curriculum", Journal of the Korean Association for Science Education, 38, pp.495~504 (2018)
- Han, D., "A study on the exploration of relationship between environmental literacy and systems thinking for sustainability education in social studies", Social Studies Education, 54, pp.65~83 (2015)
- Ignacio, J. T., "Development of an e-learning module integrating systems thinking for climate change", Ubiquitous Learning : An International Journal, 16, pp.1~16 (2023)[DOI]
- Im, Y., "Development and analysis of effects of writing educational program for improving system thinking ability", Journal of Learner-Centered Curriculum and Instruction, 14, pp.407~427 (2014)
- Jeon, J., "The development and application of STEAM education program based on systems thinking for high school students", Journal of the Korean Association for Science Education, 35, pp.1007~1018 (2015)
- Jeon, J., "Development and application of instrument for level scale of the systems thinking ability about carbon cycle", Journal of the Korean Association for Science Education, 42, pp.397~415 (2022)
- Jung, J., "Development and validation of the Kkondae tendency scale", Korean Journal of Coaching Psychology, 7, pp.153~196 (2023)
- Kali, Y., "Effect of knowledge integration activities on students’ perception of the earth’s crust as a cyclic system", Journal of Research in Science Teaching, 40, pp.545~565 (2003)
- Kang, C., "Analysis of conceptions related to Earth system and systemsthinking of high school student about water cycle", Journal of Science Education, 32, pp.61~72 (2008)
- Kim, H., "The effect of a system thinking-based steam-type ESD program on environmental literacy and system thinking ability of elementary students", Journal of Korean Journal of Environmental Education, 30, pp.85~102 (2017)
- Kim, H., "Development of a systems thinking-based steam education program for elementary school students and the effect of its application", School Science Journal, 11, pp.288~301 (2017)
- Kowch, E. G., "Learning, Design, and Technology", pp.1~14 (2019), Springer International Publishing
- Kwon, H., "Current status of the implementation of convergence education in primary and secondary schools", Journal of Science Education, 45, pp.336~348 (2021)
- Kwon, Y., "Analysis of biology teachers' systems thinking about ecosystem", Biology Education, 39, pp.529~543 (2011)
- Lee, H., "Validity verification of systems thinking measuring instrument of elementary school students’ and middle school students’ : Using item response analysis", Secondary Education Research, 67, pp.249~277 (2019)
- Lee, H., "An analysis of systems thinking levels of the water cycle in earth system of primary school pre-service teachers", Teacher Education Research, 59, pp.21~38 (2020)
- Lee, H., "Effects of systems thinking on high school students’ science self-efficacy", The Journal of The Korean Earth Science Society, 37, pp.133~145 (2016)
- Lee, H., "Analysis and effects of high school students’ systems thinking using iceberg(IB)model", Journal of the Korean Association for Science Education, 37, pp.611~624 (2017)
- Lee, H., "Development of framework and rubric for measuring students’ level of systems thinking", Journal of the Korean Association for Science Education, 38, pp.355~367 (2018)
- Lee, H., "Verification the systems thinking factor structure and comparison of systems thinking based on preferred subjects about elementary school students’", Journal of the Korean Association for Science Education, 39, pp.161~171 (2019)
- Lee, H., "An instrument development and validation for measuring high school students' systems thinking", Journal of the Korean Association for Science Education, 33, pp.995~1006 (2013)
- Lee, H., "Development and application of the educational program to increase high school students' systems thinking skills : Focus on global warming", Journal of the Korean Earth Science Society, 32, pp.784~797 (2011)
- Lee, H., "Development and application of the educational program to increase high school students' systems thinking skills : Focus on global warming", Journal of the Korean Earth Science Society, 32, pp.784~797 (2011)
- Lee, H., "The validation of the systems thinking assessment tool for measuring the higher-order thinking ability of vietnamese high school students", The Journal of The Korean Earth Science Society, 44, pp.318~330 (2023)
- Lim, A., "Development and validation of the acceptance for differences in couple relationships scale based on IBCT", Korean Journal of Psychology : General, 39, pp.599~632 (2020)
- Maani, K. E., "Links between systems thinking and complex decision making", System Dynamics Review, 20, pp.21~48 (2004)
- "Ministry of Education", 2015
- "2022 Revised Curriculum: Science", Ministry of Education, ~ (2022)
- Momsen, J., "Using systems and systems thinking to unify biology education", CBE—Life Sciences Education, 21, pp.es3~ (2022)
- Moon, B., "A study on the abilities and characteristics of the systems thinking for preservice elementary teachers", Korean System Dynamics Review, 8, pp.235~252 (2007)
- Mun, B., "Development and validation of a scale to measure of early childhood teachers climate literacy", Journal of Eco Early Childhood Education & Care, 23, pp.71~94 (2024)
- National Academies Press, , 2012
- Nguyen, H., "Impact of computer modeling on learning and teaching systems thinking", Journal of Research in Science Teaching, 58, pp.661~688 (2021)
- O'Connor, J., "The Art of Systems Thinking: Essential Skills for Creativity and Problem Solving" (1997), Thorsons Publishers
- Park, K., "Analysis of systems thinking level of pre-service teachers about carbon cycle in earth systems using rubrics of evaluating systems thinking", Journal of the Korean Association for Science Education, 39, pp.599~611 (2019)
- Park, M., "The effect of simulation integrated with problem based learning on system thinking, learning flow, proactivity in problem solving and performance ability for medication in nursing students", Journal of Digital Convergence, 16, pp.221~231 (2018)
- Park, S, "Item analysis and modification of a parental empathy inventory perceived by adolescent children scale under multiple item response theory", Korean Journal of Youth Studies, 24, pp.1~21 (2017)
- Ratinen, I., "Toward sustainable development : connecting systems thinking competency and carbon footprint knowledge", Sustainable Development, 31, pp.1593~1605 (2023)
- Roslan, S., "Big five personality traits as predictors of systems thinking ability of upper secondary school students", Pertanika Journal of Social Sciences &Humanities, 29, pp.251~269 (2021)
- Senge, P. M., "The Fifth Discipline: Fieldbook" (1996), Broadway Business
- Senge, P. M., "The Fifth Discipline: The Art & Practice of the Learning Organization" (2006), Crown Business
- Senge, P. M., "Schools that learn(Updated and Revised): A Fifth Discipline Fieldbook for Educators, Parents, and Everyone Who Cares About Education" (2012), Doubleday
- AMOS, "Statistical Analysis Methods Required for Writing SPSS", pp.2010, Song, J.
- Stavrianeas, S., "Empowering faculty to initiate STEM education transformation : Efficacy of a systems thinking approach", PLoS ONE, 17, pp.1~17 (2022)[DOI]
- Ulku, S. B., "Research trends on systems thinking approach in science education", International Journal of Science Education, 46, pp.485~502 (2024)
- Wright, B., "Reasonable mean-square-fit values", Rasch Measurement Transactions, 8, pp.370~ (1994)