Medical Science [@choi2016learning] and many more.

Pre-training techniques have been also applied to educational tasks with substantial performance improvements. For example, [@hunt2017transfer] predict whether a student will graduate or not based on students’ general academic information such as SAT/ACT scores or courses taken during college. They predict graduation of 465 engineering students by first pre-training the data of 6834 students in other departments with TrAdaBoost algorithm [@dai2007boosting]. [@ding2019transfer] suggests two transfer learning methods, each named Passive-AE transfer and Active-AE transfer, to predict student dropout in Massive Open Online Course (MOOC). Their experimental results show that both methods increase the prediction accuracy, with Passive-AE transfer more effective for transfer learning across the same subject and Active-AE transfer more effective for transfer learning across different subjects.

Most of the pre-training methods used in interactive educational system are NLP tasks on the content of learning materials. For example, the short answer grading model suggested in [@sung2019improving] use pre-trained BERT to ameliorate the limited amount of student-answer pair data. They used pre-trained, uncased BERT-base model and fine-tuned it with ScientsBank dataset and two psychology domain datasets. The resulting model outperformed existing grading models. Test-aware Attention-based Convolutional Neural Network (TACNN) [@huang2017question] is a model that 