수강소감 설문 문항반응 패턴과 강의 만족도
Course Survey Response Patterns and Course Satisfaction
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 연구는 온라인으로 실시된 두 학기 학기말 수강소감설문 문항반응을 분석하였다. 문항반응 패턴에 따라 응답자를 두 그룹으로, 즉 같은 문항번호를 연속적으로 선택한 그룹과 그렇지 않는 그룹으로 구분하고, 같은 문항 번호를 연속적으로 선택한 그룹은 1로 코딩하였고, 그렇지 않는 그룹은 0으로 코딩하였다. 학생문항반응이 강좌(교수)에게 내재되었다고 간주하고, 2수준 비선형모형을 만들었다. 학생특성(성별, 성적, 학년)과 강좌 혹은 교수특성(이․공계분야강좌, 교수신분)을 모형에 넣고, 동일문항번호를 연속적으로 선택할 가능성을 살펴보았다. 남학생, 그리고 이․공계강좌를 수강한 학생들이 여학생이나 비이․공계강좌 수강생보다 더 동일한 문항번호를 연속적으로 선택하는 것과 관련이 있었다. 모형에 들어간 다른 모든 변인을 통제한 후에도 수강과목 성적이 낮은 경우의 학생들이 높은 경우보다 더 동일 문항번호를 연속적으로 선택하는 것과 관련이 있었다. 학생들로부터 성실한 응답을 얻을 수 있는 방법에 대한 제안이 있었다.
Abstract
This study investigated student item responses reflected by the end-of-semester course evaluation surveys administered online at two different times. Two groups of respondents were identified: those who consistently chose the same option numbers were coded as 1, while those who did not choose the same option number consistently were coded as 0. By considering the finding that students’ response patterns were nested within a course (instructor), two-level, nonlinear, hierarchical models were constructed. The log-odds of a student choosing the same option number was obtained by incorporating both student characteristics (e.g., gender, course grade, year of enrollment) and course or instructor characteristics (e.g., science & engineering-related courses, instructor's tenure rank order). Male students and students who took natural science and engineering courses were more likely to choose the same option numbers compared to female students and non-natural science and engineering majors. Students with lower course grades were also more likely to choose the same option numbers than students with higher course grades, when controlling for the effects of other variables in the model. Methods to secure students' conscientious responses were also discussed.
참고문헌 (42)
- 김명화, "강의평가의 타당도와 신뢰도", 아시아교육연구, 6(3), pp.1~24 (2005), 교육연구소
- 김충행, "수업방법 개선을 위한 학생에 의한 대학강의 평가의 분석", 교육과정연구, (15) (1997)
- 김성열, "대학 학사과정 강의평가제 실태분석을 통한 교육업적 평가모형 개발연구", 교육인적 자원부 정책연구, ~ (2001)
- 민혜리, "대학교육의 질 향상을 위한 좋은 강의의 기준 탐구", 교육정보미디어연구, 11(3), pp.41~65 (2005), 한국교육정보미디어학회
- 박도순, "교육, 심리, 사회연구방법론 총서시리즈: 문항작성방법론", 교육과학사, ~ (2001)
- 한신일, "한국대학의 강의평가실태 분석", 교육행정학연구, 23(3), pp.379~404 (2005), 한국교육행정학회
- "日대학 97%, 학생에 의한 교수평가제 도입"[URL]
- d'Apollonia, S, "Navigating student ratings of instruction", American Psychologist, 52, pp.1198~1208 (1997)
- American Educational Research Association, American Psychological Association, "Standards for educational psychological testing" (1999), American Eduational Research Association
- Ballantyne, C, "Online evaluations of teaching: an examination of current practice and considerations for the future", New Directions for Teaching and learning, 43, pp.37~55 (2003)
- Centra, J. A, "Is there gender bias in student evaluations of teaching?", Journal of Higher Education, 71, pp.17~33 (2000)
- Couper, M. P, "Web Surveys: A Review of Issues and Approaches", Public Opinion Quarterly, 6, pp.464~494 (2000)
- Cranton, P., "Reconsidering the unit of analysis: A model of student ratings of instruction", Journal of Educational Psychology, 82, pp.207~212 (1990)
- Crocker, L, "Introduction to classical and modern test theory" (1986), Rinehart, & Winston
- Glassic, C. E, "Scholarship assessed: Evaluation of the professoriate" (1997), The Carnegie Foundation for Advancement of Teaching
- Greenwald, A. G, "Validity concerns and usefulness of student raitngs of instruction", American Psychologist, 52, pp.1182~1186 (1997)
- "Web-based surveys: changing the survey process"[URL]
- Hardy, N, "Perceptions of online evaluation: Fact and fiction" (2002), Presented at AERA, New Orleans
- Johnson, T. D, "Online student ratings: Will students respond?", New Directions for Teaching and learning, 43, pp.37~55 (2003)
- Krosnick, J. A., "Survey research", Annual Review of Psychology, 50, pp.537~567 (1999)
- Liu, Y., "A comparison study of online versus traditional student evaluation of instruction", International Journal of Teaching and Distance Learning, 13, pp.15~29 (2006)
- Marsh, H. W., "The influence student, course, and instructor characteristics on evaluation of university teaching", American Educational Research Journal, 17, pp.219~237 (1980)
- Marsh, H. W., "Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility", Journal of Educational Psychology, 76, pp.707~754 (1984)
- Marsh, H.W, "Students' evaluation of university teaching: Chinese version of the students' evaluation of educational quality instrument", Journal of Educational Psychology, 89, pp.586~572 (1997)
- Marsh, H. W, "The Use of students' evaluations and an individually structured intervention to enhance university teaching effectiveness", American Educational Research Journal, 30, pp.217~251 (1993)
- Marsh, H. W., "Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility", American Psychologist, 52, pp.1187~1197 (1997)
- Messick, S, "Test validity: A matter of consequence", Social Indicators Research, 45, pp.35~44 (1998)
- Moss, J, "Use of electronic surveys in course evaluation", British Journal of Eucational Technology, 33, pp.583~592 (2002)
- Ory, J. C, "Faculty Thoughts and Concerns About Student Ratings", New Directions for Teaching and Learning, 87, pp.3~15 (2001)
- Overall, J. U, "Students' evaluations of instruction: A longitudinal study of their stability.”", Journal of Educational Psychology, 72, pp.321~325 (1980)
- 박혜숙, "독서능력이 설문완성도에 미치는 효과", 교육평가연구, 18(1), pp.249~269 (2005), 한국교육평가학회
- Park, H. S, "Issues related to survey completion rate in web-based course evaluation", American Educational Research Association Annual conference (2005), Montreal
- Raudenbush, S. W, "Hierarchical linear models" (2002), Sage Publications
- Raudenbush, S. W, "HLM6: Hierarchical Linear and Nonlinear Modeling" (2005), SSI Scientific Software International
- "Web surveying: How to collect important assessment data without any paper"[URL]
- Snyder, C., R, "Effects of expected and obtained grades on teacher evaluation and attribution of performance", Journal of Educational Psychology, 68, pp.75~82 (1976)
- Sorenson, L, "Charting the unchartered seas of online student ratings of instruction", New Directions for Teaching & Learning, 96, pp.1~25 (2003)
- Thorpe, S. W., "Online student evaluation of instruction: An investigation of Non-Response Bias", AIR 2002 Forum Paper (2002)
- Ting, K-F, "A multilevel perspective on student ratings of instruction: Lessons from the Chinese experience", Research in Higher Education, 41, pp.637~661 (2000)
- Tomsic M, "A world wide web response to student satisfaction surveys: Comparisons using paper and internet formats", Paper presented at the 40th Annual Meeing of the Association of Institutional Research (2000)
- Underwood, D, "To mail or to web: Comparisons of survey response rates and respondent characteristics", Paper presented at the 40th Annual Meeting of the Association of Institutional Research (2000)
- "Web & E-mail surveys"[URL]