교육？훈련프로그램 반응평가를 위한 평가범주와 질문지 개발준거
Dimension Classification and Questionnaire Design Criteria for Reaction Evaluation of Educational/Training Programs
0
조회
0
다운로드
52
인용
0
참고문헌
키워드
초록
이 연구는 교육훈련프로그램 반응평가를 위한 평가범주와 질문지 개발에 요구되는 필수요소를 탐색하고, 이들 반응평가 평가범주와 질문지 개발준거를 사용하여 우리나라 교육훈련기관에서 사용하는 반응평가 질문지를 분석하고자 함이 그 목적이다. 이 연구는 반응평가를 위한 11영역의 평가범주와 질문지 개발에 필요한 5요소의 개발준거를 종합제시하였으며, 이들 평가범주와 질문지 개발준거를 기준으로 우리나라 교육훈련기관에서 사용하고 있는 반응평가 도구를 수집분석하고, 반응평가 평가범주별 표본평가문항을 발췌제시하였다. 반응평가에 사용되고 있는 질문지에 대한 분석결과는 대부분 교육훈련기관은 프로그램 평가의 목적과 범위에 맞추기 위하여 3～5영역의 평가범주를 선택적으로 사용하고 있었으며, 소수의 반응평가 질문지가 질문지 개발준거에 따라서 개발사용되고 있었다. 적절한 평가범주의 선정과 질문지 개발준거에 부합되는 반응평가 질문지 개발활용은 반응평가가 교육훈련프로그램 평가에서 ‘학습자 만족도’ 확인을 위한 보다 유용하고 가치 있는 도구로 사용될 수 있을 것이다.
Abstract
The purposes of the present study were to classify the dimensions of information sought using reaction evaluations and to establish design criteria for developing reaction questionnaires of educational/training program. There are major phases of the study: (1) classification of the dimensions and questionnaire design criteria for using reaction evaluations; (2) validation of the classified dimensions and design criteria of questionnaire by subject matter experts; and (3) assessment of a sample of educational/training reaction questionnaires currently used in educational/training centers of Korea. The findings of the study were as follows: eleven dimensions for reaction evaluation of educational/training programs were identified and classified by purposes. Five overall design criteria, each consisting of several sub-criteria, were judged important factors in designing questionnaires for reaction evaluation. The criteria include: introduction, directions, and closing statements; question format; question construction; questionnaire layout; and data analysis. It was concluded that a well-designed questionnaire for reaction evaluation of educational/training programs integrates the proper application of design criteria with appropriate reaction dimensions. Most questionnaires for reaction evaluation of educational/training programs used in educational/training centers of Korea consisted of questions representing only 3-5 dimensions. The questionnaires varied in form and length. Few of them poorly utilized the established questionnaire design criteria. With appropriate dimensions of reaction evaluation and design criteria for constructing questionnaire of educational/training program evaluation, program evaluation professionals can make reaction evaluations of program more useful and valuable in educational/training programs.
참고문헌 (51)
- 강이철, "기업교육 프로그램 평가. 나일주, 임철일, 이인숙 (편저) (2003)", pp.225~249 (2003), 학지사
- 정재삼, "교육프로그램평가" (2004), 교육과학사
- 황정규, "학교학습과 교육평가(개정판)" (1998), 교육과학사
- & Janak, "Kirkpatrick‘s levels of training criteria", pp.331~342 (1989)
- 7th ed, "The practice of social research (7th ed.)." (1995), Wadsworth
- Basarab,D.J, "Assessing the quality of training evaluation studies", 33(3), pp.19~22 (1994)
- & Root, "A practical approach to evaluating corporate training programs" (1992)
- Beer, "Levels of evaluation", pp.335~345 (1986)
- Boyle, "Handbook of adult education" (1970), Macmillan
- Brown, "Evaluating corporate training: Models and issues" (1998), Kluwer Academic
- Carnevale, "Training in America The organization and strategic role of training" (1990), Jossey-Bass
- Covert,R.W, "A checklist for developing questionnaires", 5(3), pp.74~78 (1984)
- Cronbach,L.J, "Course improvement through evaluation", 64(8), pp.673~683 (1963)
- Dixon,N.M, "New routes to evaluation", 50(5), pp.82~85 (1996)
- Dolezalek,H, "Industry report 2004:Training magazine's 23rd annual comprehensive analysis of employer-sponsored training in the United States", 41(10), pp.20~37 (2004)
- Dyer,S, "Kirkpatrick's mirror", 18(5), pp.31~32 (1994)
- Easterby-Smith,M, "Turning course evaluation from an end to a means", 17(4), pp.25~27 (1985)
- & Ban, "Issues in training evaluation", pp.299~314 (1993)
- Fisk,C.N, "ASTD trainer's toolkit:Evaluation instruments" (1991), Pfeiffer
- Forsyth, "Evaluating a course: Practical strategies for teacher, lecturers and trainers" (1999)
- Gordon,S.E, "Systematic training program design:Maximizing effectiveness and minimizing liability" (1994), PTR Prentice Hall
- Hamblin,A.C, "Evaluation and control of training" (1974), McGraw-Hill
- Hayes,B.E, "Measuring customer satisfaction:Development and use of questionnaire" (1992), ASQC Quality
- Holton,E.F.III, "The flawed four-level evaluation model", 7(1), pp.5~21 (1996)
- Jick,T.D, "Mixing qualitative and quantitative methods:Triangulation in action", 24(4), pp.603~611 (1979)
- Kaufman, "Evaluation beyond Kirkpatrick", 35(2), pp.8~12 (1995)
- Keller,J.M, "Evaluating diversity training:17 ready-to-use tools" (1996), Pfeiffer
- Kirkpatrick, D. L., "Evaluating training programs: The four levels (2nd ed.)." (1998), Berrett-Koehler
- Mattoon,J.S, "Evaluating training and educational programs:A review of the literature" (1992), Air Force Material Command
- McBean, "Questionnaire design for student measurement of teaching effectiveness", pp.273~288 (1982)
- McGehee, "Training in business and industry" (1961)
- Molenda, "A guide to human resource development", pp.266~293 (1996), McGraw-Hill
- Neuman, W. L., "Social research methods: Qualitative and quantitative approach" (1997), Allyn and Bacon
- Ostroff,C, "Training effectiveness measures and scoring schemes:A comparison", 44(2), pp.353~374 (1991)
- Payne, "The art of asking questions" (1951), Princeton Princeton University Press
- Phillips, J. J., "Handbook of training evaluation and measurement methods: Proven models and methods for evaluating any HRD program (3rd ed.)." (1997), Gulf
- Phillips,S, "Evaluation" (1993), Library Association
- Richey,R.C, "Trends in instructional design:Emerging theory-based models", 8(3), pp.96~110 (1995)
- Roberts,A, "Evaluating training programmes", 26(4), pp.18~23 (1990)
- Robinson, "Training for impact How to link training to business needs and measure the results" (1989), Jossey-Bass
- Rosenberg, "Evaluating training programs for decision making", pp.57~74 (1987)
- Schouborg,G, "Flex:A flexible tool for continuously improving your evaluation of training effectiveness" (1993), HRD
- In P, "Handbook of survey research"
- March 24-26, "Research & theory to practice Paper presented at the International Society for Performance Improvement" (1998)
- Sugrue, "American Society for Training & Development's 2004 state of the industry report American Society for Training & Development" (2004), American Society for Training & Development
- Swisher,R, "Criteria for the design of mail questionnaire", 21(2), pp.159~165 (1980)
- Thiagarajan,S, "Instructional product verification and revision:20 questions and 200 speculation", 26(2), pp.133~141 (1978)
- Warr, "Evaluation of management training" (1970)
- Cayer, "Handbook of training and development for the public sector" (1993), Jossey-Bass
- Weisberg, "An introduction to survey research, polling, and data analysis" (1996), Sage
- Worthen, "Program evaluation: Alternative approaches and practical guidelines (2nd ed.)" (1997), Longman