Ko, Lee, Shin, & Lee (2025). 교양 글쓰기 교육 강화를 위한 자동평가 적용 사례

[Dongkwang Shin]

Ko, K., Lee, Y., Shin, D., & Lee, H. (2025). Application of an automated scoring system to enhance college writing education: A case study of K university. Journal of Education & Culture, 31(4), 605~619. https://doi.org/10.24159/joec.2025.31.4.605

This study investigates the effectiveness and reliability of an AI-based automated scoring system in K University’s college writing eval‎uation. The assessment included two tasks—argumentative writing and summarization—and students’ responses were scored across three domains: task completion, content organization, and language use. An automated scoring program, PASTA, was utilized to assess student writing. The performance of the automated scoring model was validated by comparing its results with those of human raters. The findings showed that the agreement and near-agreement rates were comparable to, or even higher than, those observed in human-to-human scoring in similar assessments, demonstrating the reliability of the automated scoring model. Additionally, PASTA was found to be a useful tool in writing classes, as it provided various eval‎uation results and personalized feedback to students. The use of the automated scoring system was shown to reduce instructors' grading workload while increasing opportunities for students to engage in written eval‎uations. Furthermore, analysis revealed that students with prior writing experience outperformed those without such experience, and that students with more than one week of writing experience scored significantly higher than those with less experience. These results offer empirical evidence that sustained writing education contributes to improved student writing performance.

[Keywords] college writing, automated scoring, PASTA, feedback

(25. 이효신, 이용상 외)_논문최종.pdf

2.05MB