A novel automated essay scoring approach for reliable higher educational assessments

被引:31
作者
Beseiso, Majdi [1 ]
Alzubi, Omar A. [1 ]
Rashaideh, Hasan [1 ]
机构
[1] Al Balqa Appl Univ, Comp Sci Dept, Salt, Jordan
关键词
Automated essay scoring (AES); Deep learning; Essay scoring; Long short-term memory; Neural network; Transfer learning; ENGLISH;
D O I
10.1007/s12528-021-09283-1
中图分类号
G40 [教育学];
学科分类号
040101 ; 120403 ;
摘要
E-learning is gradually gaining prominence in higher education, with universities enlarging provision and more students getting enrolled. The effectiveness of automated essay scoring (AES) is thus holding a strong appeal to universities for managing an increasing learning interest and reducing costs associated with human raters. The growth in e-learning systems in the higher education system and the demand for consistent writing assessments has spurred research interest in improving the accuracy of AES systems. This paper presents a transformer-based neural network model for improved AES performance using Bi-LSTM and RoBERTa language model based on Kaggle's ASAP dataset. The proposed model uses Bi-LSTM model over pre-trained RoBERTa language model to address the coherency issue in essays that is ignored by traditional essay scoring methods, including traditional NLP pipelines, deep learning-based methods, a mixture of both. The comparison of the experimental results on essay scoring with human raters concludes that the proposed model outperforms the existing methods in essay scoring in terms of QWK score. The comparative analysis of results demonstrates the applicability of the proposed model in automated essay scoring at higher education level.
引用
收藏
页码:727 / 746
页数:20
相关论文
共 65 条
[1]  
Alikaniotis D, 2016, PROCEEDINGS OF THE 54TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1, P715
[2]  
Alzahrani S., 2020, INT J ADV COMPUT SC, DOI [10.14569/IJACSA.2020.0111027, DOI 10.14569/IJACSA.2020.0111027]
[3]  
Attali Y., 2006, J TECHNOLOGY LEARNIN, V4
[4]  
Bahdanau D, 2016, Arxiv, DOI arXiv:1409.0473
[5]  
Bennett R.E., 1998, Educational Measurement: Issues and Practice, V17, P9, DOI [10.1111/j.1745-3992.1998.tb00631.x, DOI 10.1111/J.1745-3992.1998.TB00631.X]
[6]   Validating automated speaking tests [J].
Bernstein, Jared ;
Van Moere, Alistair ;
Cheng, Jian .
LANGUAGE TESTING, 2010, 27 (03) :355-377
[7]   Seeing the Fisher Z-transformation [J].
Bond, CF ;
Richardson, K .
PSYCHOMETRIKA, 2004, 69 (02) :291-303
[8]  
Cer D, 2018, CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2018): PROCEEDINGS OF SYSTEM DEMONSTRATIONS, P169
[9]   Automatic Essay Scoring in E-learning System Using LSA Method with N-Gram Feature for Bahasa Indonesia [J].
Citawan, Rico Setiadi ;
Mawardi, Viny Christanti ;
Mulyawan, Bagus .
3RD INTERNATIONAL CONFERENCE ON ELECTRICAL SYSTEMS, TECHNOLOGY AND INFORMATION (ICESTI 2017), 2018, 164
[10]  
Dapeng Li, 2020, Innovative Mobile and Internet Services in Ubiquitous Computing. Proceedings of the 14th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS-2020). Advances in Intelligent Systems and Computing (AISC 1195), P264, DOI 10.1007/978-3-030-50399-4_26