"Types" of Classification Consistency from the Perspective of Replication
2
조회
0
다운로드
4
인용
0
참고문헌
키워드
초록
Classification consistency is often considered as reliability of a measurement procedure involving classifications of examinees into a set of ordered categories. In order to quantify classification consistency, an investigator should define the universe of intended replications, and determine data collection designs as well as methods for estimating classification consistency that would be the most suitable for the intended replications. This process is important because classification consistency estimated from different conceptualizations about replications may produce different types of classification consistency estimates. This paper differentiates three types of classification consistency from the perspective of replications (test-retest classification consistency; alternate-forms classification consistency; and internal classification consistency), and introduces three model-based approaches to estimating classification consistency based on each conceptualization, with focuses being given to the intended universe of replications. An empirical analysis will be provided at the end to illustrate differences in estimation according to different conceptualizations about the universe of replications.
Abstract
Classification consistency is often considered as reliability of a measurement procedure involving classifications of examinees into a set of ordered categories. In order to quantify classification consistency, an investigator should define the universe of intended replications, and determine data collection designs as well as methods for estimating classification consistency that would be the most suitable for the intended replications. This process is important because classification consistency estimated from different conceptualizations about replications may produce different types of classification consistency estimates. This paper differentiates three types of classification consistency from the perspective of replications (test-retest classification consistency; alternate-forms classification consistency; and internal classification consistency), and introduces three model-based approaches to estimating classification consistency based on each conceptualization, with focuses being given to the intended universe of replications. An empirical analysis will be provided at the end to illustrate differences in estimation according to different conceptualizations about the universe of replications.
참고문헌 (21)
- "A mathematical analysis of some ETS equating procedures", pp.9~49 (1982), Academic Press
- "An essay on the history and future of reliability from the perspective of replications", 38, pp.295~317 (2001)
- "Manual for BB-CLASS A computer program that uses the beta-binomial model for classification consistency and accuracy", (9) (2004), Iowa City, IA: Center for Advanced Studies in Measurement and Assessment, The University of Iowa
- "Pass-Fail Reliability for Tests with Cut Scores" (1994), Princeton
- "Special problems in measuring change with psychometric devices American Institutes for Research" (1970)
- "American Council on Education and McMillan", pp.105~146 (1989)
- "Method of moments estimates for the four-parameter beta compound binomial model and the calculation of classification consistency indexes" (1991), Inc
- "An investigation of classification consistency indexes estimated under alternative strong true score models Journal of Educational Measurement", pp.345~359 (1990)
- "On the reliability of decisions in domain-referenced testing Journal of Educational Measurement", pp.253~264 (1976)
- "Test equating, scaling, and linking: Methods and practices (2nd ed.)" (2004), New York: Springer-Verlag
- "Estimating consistency and accuracy indexes for multiple classifications", 26(4), pp.412~432 (2002)
- "Estimating the consistency and accuracy of classifications based on test scores", 9, pp.13~26 (1995)
- "Statistical theories of mental test scores" (1968), Addison-Wesley Publishing Company
- "Comparison of IRT true-score and equipercentile observed-score ‘equatings’", pp.453~461 (1984)
- "Item and test scoring with binary logistic models", pp.~3 (1990)
- "Estimating reliability from a single administration of a criterion-references test", 13, pp.265~276 (1976)
- "A reliability of criterion-referenced tests Journal of Educational Measurement", pp.263~267 (1975)
- "Psychometric properties of scale scores and performance levels for performance assessments using polytomous IRT", 37, pp.141~162 (2000)
- "Methods for estimating classification consistency indices for two equated forms" (2004), San Diego, CA
- "A Method for estimating classification consistency indices for two equated forms", 31(4), pp.275~291 (2007)
- "Multi-group IRT analysis and test maintenance for binary items" (1996), Chicago: IL: Scientific Software International