slightly and trained in main tasks with possibly scarce data. This approach was successful in other AI fields including Natural Language Processing (NLP), computer vision, motion planning and so on [@devlin2018bert; @studer2019comprehensive; @schneider2019wav2vec]. In this line of thought, content-based pre-training methods have been studied [@huang2017question; @sung2019improving; @yin2019quesnet]. However, the pre-trained models fail to capture student learning behavior as student interactions are not considered.

In this paper, we propose assessment modeling, fundamental pre-training tasks for general IESs. Here, an assessment is any feature from student-system interactions which can act as pedagogical evaluation criterion. Examples of assessments are the time a student has spent on each exercise and the correctness of student response to a given exercise. While there is a wide range of interactive features available, narrowing down the prediction targets to assessments holds relevance to the label-scarce educational problems while reducing irrelevant noises. Also, most of assessments are available in large amount as they are automatically collected by IESs.

Inspired by the recent success of bidirectional representations in NLP domain [@devlin-etal-2019-bert], we develop an assessment model with a deep bidirectional Transformer encoder. In the pre-training phase, we randomly select a portion of entries in a sequence of interactions and mask 