수사구조이론(RST) 기반의 내용 조직 능력 진단 도구의 채점 방안 탐색

Abstract

The current study reviewed scoring methods for the assessment tool on learners’ ability to organize text construction using the rhetorical structure theory (RST) vis-à-vis actual scoring results. Our findings are summarized as follows. First, in relation to overlapping assessment items, the additional points method and the appraisal method were both useful in defining the levels of the ability but demonstrated different advantages and disadvantages. The former method demonstrated strengths in identifying minute differences in individual learners’ abilities whereas the latter method outperformed the former in preventing distortion between assessment results and the actual gap in learners’ abilities. Second, we reviewed the issue of applying weighted scoring as per the importance of each assessment item. It was found that weighted scoring as per the number of levels in a node may prevent accurate assessment due to the influence of the type of semantic relations and innate difficulties in the text structure. Third, we reviewed the two scoring options of accommodating the hierarchy shown in the RST schema and scoring all sentence arrangements in a text. We confirmed that scoring the connection of sentences based on the RST schema, even when they are not directly connected, may throw into dispute the accuracy of assessment on the corresponding ability among learners. The assessment tool on the text organizing ability proposed in the current study provides the advantage of automating the scoring process but entails the limitation of assessing only some aspects of learners’ ability to build semantic structures using RST. This is also the limitation of our study as we assumed that learners were being assessed on the basis of a single text that took the form of sentence arrangement. Nonetheless, it is the significance of our study that we reviewed practical scoring methods that leverage the strengths of an assessment tool that takes the form of sentence arrangement and, thereby, demonstrated the potential for valid differentiation of learners’ abilities.