How Good is Good Enough?: A Comparison of Three Methods for Establishing Cut Scores on Placement Tests
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
The purpose of this study was to determine whether three standard-setting methods would produce sufficiently consistent results for placement decisions. The cut scores were derived from modified Angoff method, borderline group method, and cluster analysis. The results indicate that the cut scores derived from the three standard setting methods did not entirely agree with each other in assigning students into different levels, suggesting that the choice of standard setting method influenced the resulting cut scores. Specifically, the borderline-group method tended to produce lower cut scores than the other two methods. The cluster analysis yielded two cut scores, which were very similar to the cut scores derived from the modified Angoff method at the corresponding levels. The implications of the findings are discussed and avenues for further study are suggested.
Abstract
The purpose of this study was to determine whether three standard-setting methods would produce sufficiently consistent results for placement decisions. The cut scores were derived from modified Angoff method, borderline group method, and cluster analysis. The results indicate that the cut scores derived from the three standard setting methods did not entirely agree with each other in assigning students into different levels, suggesting that the choice of standard setting method influenced the resulting cut scores. Specifically, the borderline-group method tended to produce lower cut scores than the other two methods. The cluster analysis yielded two cut scores, which were very similar to the cut scores derived from the modified Angoff method at the corresponding levels. The implications of the findings are discussed and avenues for further study are suggested.
참고문헌 (40)
- Alderson, J. C., "Language test construction and evaluation" (1995), Cambridge University Press
- Angoff, W. H., "Educational measurement", pp.508~600 (1971), American Council on Education
- Bechger, T. M., "Standard setting in relation to the Common European Framework of Reference for Languages : The case of the state examination of Dutch as a second language", Language Assessment Quarterly, 6(2), pp.126~150 (2009)
- Berk, R. A., "A consumer’s guide to setting performance standards on criterionreferenced tests", Review of Educational Research, 56, pp.137~172 (1986)
- Brindley, G., "Outcomes-based assessment in practice : Some examples and emerging insights", Language Testing, 18(4), pp.393~407 (2001)
- Brown, J. D., "On TESOL ’80building bridges: Research and practice in teaching English as a second language" (1980), TESOL
- Brown, J. D., "Testing in language programs" (1996), Prentice Hall Regents
- Cizek, G. J., "Standard setting: A guide to establishing and evaluating performance standards on tests" (2007), Sage
- Cohen, J., "A coefficient of agreement for nominal scales", Educational and Psychological Measurement, 20, pp.37~46 (1960)
- Cross, L., "A comparison of three methods for establishing minimum standards on the National Teacher Examinations", Journal of Educational Measurement, 21, pp.113~130 (1984)
- Ebel, R. L., "Essentials of educational measurement" (1972), Prentice-Hall
- Educational Testing Service, "Report on a study of the use of the National Teachers’Examination by the state of South Carolina" (1976), Educational Testing Service
- Formann, A. K., "Die latent-class-Analyse : Einführung in die theorie und anwendung" (1984), Beltz
- Glass, G. V., "Standards and criteria", Journal of Educational Measurement, 15, pp.237~261 (1978)
- Griffin, P., "Experimenting with uncertainty: Essays in honor of Alan Davies", pp.97~107 (2001), Cambridge University Press
- Hambleton, R. K., "Using an extended Angoff procedure to set standards on complex performance assessments", Applied Measurement in Education, 8, pp.41~56 (1995)
- Hambleton, R. K, "Educational Measurement", pp.433~470 (2006), American Council on Education
- Hsieh, M., "Comparing Yes", No Angoff and Bookmark standard setting methods in the context of English assessment, 2013, pp.10(3), 331~350
- Jaeger, R. M., "An iterative structured judgment process for establishing standards on competency tests : Theory and application", Educational Evaluation and Policy Analysis, 4, pp.461~476 (1982)
- Jaeger, R. M., "Educational measurement", pp.485~514 (1989), Macmillan
- Jaeger, R. M., "The effects of a Delphi modification of the Angoff-Jaeger standard setting procedure on standards recommended for the National Teacher Examinations", Paper presented at the annual meeting of the American Educational Research Association (1984)
- Kane, M. T., "In Proceedings of joint conference on standard setting for large-scale assessments", pp.119~139 (1995), National Assessment Governing Board and National Center for Education Statistics
- Kozaki, Y., "Using GENOVA and FACETS to set multiple standards on performance assessment for certification in medical translation from Japanes into English", Language Testing, 24(1), pp.1~27 (2004)
- Livingston, A., "Passing scores: A manual for setting standards of performance on educational and occupational tests" (1982), Educational Testing Service
- Livingston, S. A., "A comparative study of standard setting methods", Applied Measurement in Education, 2, pp.121~141 (1989)
- Mehrens, W. A., "Methodological issues in standard setting for educational exams", Paper presented at the Joint Conference on Standard Setting for Large-Scale Assessments (1994)
- Messick, S., "Educational measurement", pp.13~103 (1989), Macmillan
- Nedelsky, L., "Absolute grading standards for objective tests", Educational and Psychological Measurement, 14, pp.3~19 (1954)
- O’Neill, T. R., "Recommending a nursing-specific passing standard for the IELTS examination", Language Assessment Quarterly, 4(4), pp.295~317 (2007)
- Pampel, F. C., "Logistic regression: A primer" (2000), Sage Publications
- Reid, J. B., "Training judges to generate standard setting data", Educational Measurement : Issues & Practice, 10, pp.11~14 (1991)
- Shepard, L., "Setting performance standards for student achievement" (1993), National Academy of Education
- Sireci, S. G., "Setting performance standards", pp.339~354 (2001), Lawrence Erlbaum
- Sireci, S. G., "Using cluster analysis to facilitate standard setting", Applied Measurement in Education, 12, pp.301~325 (1999)
- Stansfield, C. W., "Examining the predictive validity of a screening test for court interpreters", Language Testing, 22(4), pp.438~462 (2005)
- "Graduate Admissions Information" (2014)[URL]
- Xi, X., "Validating TOEFL iBT speaking and setting score requirements for ITA screening", Language Assessment Quarterly, 4(4), pp.318~351 (2007)
- Zieky, M. J., "A historical perspective on setting standards. In Proceedings of the joint conference on standard setting for large-scale assessments" (1995), National Assessment Governing Board and the National Center for Education Statistics
- Zieky, M. J., "Setting performance standards" (2001), Lawrence Erlbaum
- Zieky, M. J., "Manual for setting standards on the basic skills assessment tests" (1977), Educational Testing Service