본문 바로가기
KERD
논문KCI 등재

대화 분석을 통한 사회정서역량 측정 애플리케이션 개발 - ChatGPT API를 이용하여 -

Developing an Application Prototype for Social and Emotional Competence Measurement through Conversation Analysis - Using the ChatGPT API -

이상수
교육발전연구소 (Pusan National University Educational Development Institute)|2023.09|교육혁신연구 (Journal of Educational Innovation Research)|제33권 제3호|pp. 55-78|DOI 10.21024/pnuedi.33.3.202309.55

0

조회

0

다운로드

4

인용

0

참고문헌

키워드

애플리케이션ChatGPTapplicationconversation analysis사회정서역량대화 분석사회정서역량 측정ChatGPT APIsocial emotional competencysocial emotional competency measurement

초록

연구목적: 이 연구의 목적은 거대 언어 모델의 대표적 ChatGPT를 이용하여 대화 분석을 통한 사회정서역량 측정 애플리케이션의 가능성을 탐색하는 프로토타입을 개발하는 것이다. 연구 방법: 이 연구는 개발연구 방법론을 사용하였다. 애플리케이션은 ChatGPT API를 이용하여 개발하였으며, 애플리케이션 개발을 위해 클로바노트, Python, ChatGPT, Markdown, Streamlit, Plotly 등의 프로그램을 사용하였다. 어플리케이션의 타당화 방법으로 ChatGPT 분석 결과 간 그리고 전문가 분석 결과 간 일치도 분석을 위해 평정자 간 신뢰도 분석을 실시하였다. 연구결과: 애플리케이션은 클로바노트를 이용하여 자동적으로 대화를 녹취와 전사를 하여 분석을 위한 전처리를 실시 하였으며, 데이터를 업로드하면 참가자별 대화가 추출되고 문장별로 평정척도를 적용하여 분석이 이루어졌다. 그리고 결과제시는 참여자 별로 사회정서역량 점수와 분석 근거가 제시되고 오각형 점수를 통한 직관적 점수 이해가 가능하도록 하였다. 애플리케이션을 타당화하기 위하여 ChatGPT 결과간 그리고 전문가 분석 점수 간 내적 합치도를 분석하였고 높은 내적 합치도로 타당한 것으로 결과는 나타났다. 논의 및 결론: 이 연구는 ChatGPT API를 이용하여 대화 분석을 통한 사회정서역량 측정 가능성을 보여주었으며 또한 사회과학 분야에서 거대 언어 모델을 사용하여 사회과학적 문자 데이터을 분석하고 연구할 수 있는 가능성을 확인시켜 주었다는 차원에서 의미가 있다고 할 수 있다.

Abstract

Purpose: The purpose of this study is to develop a prototype that explores the possibility of measuring social emotional competency through conversation analysis using ChatGPT, a representative macro language model. Method: This study used the development research methodology. The application was developed using the ChatGPT API, and programs such as Clovanote, Python, ChatGPT, Markdown, Streamlit, and Plotly were used for application development. ICC(intraclass correlation coefficients) was conducted to analyze the agreement between ChatGPT analysis results and expert analysis results as a validation method of the application. Results: The application automatically recorded and transcribed conversations using Clovanote to perform preprocessing for analysis. When the data was uploaded, conversations by participant were extracted and analysis was performed by applying a rating scale for each sentence. In addition, in the presentation of the results, the social-emotional competency score and the basis for analysis were presented for each participant, and it was possible to understand the intuitive score through the pentagon score. In order to justify the application, the internal agreement between the ChatGPT results and the expert analysis scores was analyzed, and the result was found to be valid with a high internal agreement. Conclusion: This study is meaningful in that it showed the possibility of measuring social emotional competency through conversation analysis using the ChatGPT API, and also confirmed the possibility of analyzing and studying social scientific text data using a large language model in the field of social science.

참고문헌 (40)

  1. 이상수, "대화 분석을 통한 사회정서역량 평정 척도 개발", 교사교육연구, 62(2), pp.139~156 (2023), 과학교육연구소[DOI]
  2. Assessment Work Group, "Student social and emotional competence assessment: The current state of the field and a vision for its future" (2019), Collaborative for Academic, Social, and Emotional Learning
  3. Baidoo-Anu, D., "Education in the era of generative artificial intelligence (AI):Understanding the potential benefits of ChatGPT in promoting teaching and learning", SSRN 4337484, ~ (2023)
  4. Bang, Y., "A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity", ArXiv preprint arXiv:2302.04023, ~ (2023)
  5. "CASEL’s SEL FRAMEWORK: What are the core competence areas and where are they promoted?" (2020)[URL]
  6. Denham, S., "Computerized social-emotional assessment measures for early childhood settings", Early Childhood Research Quarterly, 51, pp.55~66 (2020)
  7. Denham, S. A., "Compendium of social–emotional learning and associated assessment measures" (2010), Collaborative for Academic, Social, and Emotional Learning
  8. DeRosier, M. E., "Establishing the criterion validity of Zoo U's game-based social emotional skills assessment for school-based outcomes", Journal of Applied Developmental Psychology, 55, pp.52~61 (2018)
  9. Durlak, J. A., "The impact of enhancing students' social and emotional learning : A meta-analysis of school-based universal interventions", Child Development, 82(1), pp.405~432 (2011)
  10. Elias, M. J., "Promoting social and emotional learning: Guidelines for educators" (1997), Association for Supervision and Curriculum Development
  11. Feng, Y., "Investigating Code Generation Performance of Chat-GPT with Crowdsourcing Social Data", Proceedings of the 47th IEEE Computer Software and Applications Conference, pp.1~10 (2023)
  12. Gilardi, F., "ChatGPT outperforms crowd-workers for text-annotation tasks", ArXiv arXiv:2303.15056, ~ (2023)
  13. Gilson, A., "How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment", JMIR Medical Education, 9(1), pp.e45312~ (2023)
  14. Greenberg, M. T., "Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning", American Psychologist, 58, pp.466~474 (2003)
  15. Guerra, N. G., "Linking the prevention of problem behaviors and positive youth development : Core competencies for positive youth development and risk prevention", New Directions for Child and Adolescent Development, 2008(122), pp.1~17 (2008)
  16. "A Teacher's Prompt Guide to ChatGPT aligned with 'What Works Best' Guide" (2023)[URL]
  17. Isen, A. M., "Positive affect facilitates creative problem solving", Journal of Personality and Social Psychology, 52, pp.1122~1131 (1987)
  18. Jansen, B. J., "Employing large language models in survey research", Natural Language Processing Journal, 4, pp.100020~ (2023)
  19. "Navigating SEL from the inside out" (2017)[URL]
  20. Kocoń, J., "ChatGPT : Jack of all trades, master of none", Information Fusion, 99, pp.1~37 (2023)
  21. Kosinski, M., "Theory of mind may have spontaneously emerged in large language models", arXiv:2302.02083, ~ (2023)
  22. Kring, A. M., "The regulation of emotion", pp.359~385 (2004), Erlbaum
  23. Lawson, G. M., "The Core omponents of Evidence-Based Social Emotional Learning Programs", Prevention Science, 20(4), pp.457~467 (2018)[DOI]
  24. McKown, C., "Social-emotional assessment, performance, and standards", The Future of Children, 27(1), pp.157~178 (2017)
  25. Merrell, K. W., "Development and validation of a teacher report measure for assessing social-emotional strengths of children and adolescents", School Psychology Review, 40, pp.226~241 (2011)
  26. Mizumoto, A., "Exploring the potential of using an AI language model for automated essay scoring", Research Methods in Applied Linguistics, 2(2), pp.100050~ (2023)
  27. "API REFERENCE" (2023)[URL]
  28. Pekrun, R., "Boredom in achievement settings : Exploring control–value antecedents and performance outcomes of a neglected emotion", Journal of Educational Psychology, 102, pp.531~549 (2010)
  29. Renk, K., "Cross informant ratings of social competence in children and adolescent", Clinical Psychology Review, 24, pp.239~254 (2004)
  30. Rimm-Kaufman, S. E., "Handbook of social and emotional learning: Research and practice", pp.27~51 (2015), Guilford Press
  31. Rosenberg, E. L., "Coherence between expressive and experiential systems in emotion", Cognition and Emotion, 8, pp.201~229 (1994)
  32. Sayer, A., "Realism and social science", Realism and Social Science, pp.1~224 (1999)
  33. "Test generation with tempatature ans top-p sampling in GPT models: an in-depth guide" (2023)[URL]
  34. Storbeck, J., "The affective regulation of cognitive priming", Emotion, 8, pp.208~215 (2008)
  35. Tabone, W., "Using ChatGPT for human–computer interaction research : a primer", Royal Society Open Science, 10, pp.231053~ (2023)[DOI]
  36. Törnberg, P., "ChatGPT-4 outperforms experts and crowd workers in annotating political Twitter messages with zero-shot learning", ArXiv arXiv:2304.06588, ~ (2023)
  37. Weissberg, R. P., "Handbook ofsocial and emotional learning: Research and practice", pp.3~19 (2015), Guilford
  38. Wu, P. Y., "Large language models can be used to estimate the ideologies of politicians in a zero-shot learning setting", arXiv preprintarXiv:2303.12057, ~ (2023)
  39. Yang, Z., "Counterfactual can be strong in medical question and answering", Information Processing & Management, 60(4), pp.103408~ (2023)
  40. Zhang, J., "Perspective-taking and depth of theory-of-mind reasoning in sequential-move games", Cognitive Science, 36(3), pp.560~573 (2012)

관련 논문AI추천

대화 분석을 통한 사회정서역량 평정 척도 개발이상수과학교육연구소 · 교사교육연구2023 · 제62권 제2호초등학생의 사회정서학습 역량 강화를 위한 생성형 AI 기반 챗봇 개발권희수, 심재권한국정보교육학회 · 정보교육학회논문지2025 · 제29권 제6호ChatGTP API 활용 인공지능 논･서술형 자동채점 프로그램 개발 실행연구임종한, 임종헌교육발전연구소 · 교육혁신연구2024 · 제34권 제1호인공지능 기반 유아 사회정서 발달평가 시스템 구축을 위한 사회정서 발달평가 루브릭 개발민선옥, 임미선한국생태유아교육학회 · 생태유아교육연구2025 · 제24권 제4호ChatGPT를 활용한 AI 글쓰기 의사소통 역량 평가도구 개발 과정에 대한 연구: 기술 전문가와의 상호소통을 중심으로박소영, 이병윤, 홍유정한국실천공학교육학회 · 실천공학교육논문지2024 · 제16권 제1호GPT-3.5를 활용한 진로성숙도 문장완성검사 응답의 자동 정량화 및 문항반응이론 적용민경석, 이서진한국직업능력연구원 · 직업능력개발연구2025 · 제28권 제2호

이 간행물의 다른 논문 (12건)

코로나19로 인한 교수법 변화: 테크노스트레스와 테크놀로지 수용 성향에 따른 교수자 유형 분석장윤경, 박연정, 이성아, 이수경pp. 1-29Ellen Key의 『어린이의 세기』에 나타난 아동관강기수, 박소영pp. 31-53메타분석의 내적타당성 확보를 위한 1차 연구의 질 평가에 대한 고찰심승엽, 김순근, 한기순, 윤소영pp. 79-101유아특수교사의 「2019 개정 누리과정」 실행 경험 연구에 대한 질적 메타분석: 행위성(Agency) 발견하기서보순pp. 103-137놀이 중심 유아교육과정의 교육내용 탐구 : 유아교사의 인식과 이해를 중심으로임부연, 최윤경pp. 139-167
원문 보기