bidirectional representation in NLP domain [@devlin-etal-2019-bert], we develop an assessment model with a deep bidirectional Transformer encoder.

-   We empirically verify that assessment modeling as pre-training tasks achieves an improvement of $14.16$ mean absolute error and $0.014$ accuracy from the previous state-of-the-art model for exam score and review correctness prediction respectively.

Related Works
=============

### Artificial Intelligence in Education

AIEd supports education through different AI technologies, including machine learning and deep learning, to “promote the development of adaptive learning environments and other AIEd tools that are flexible, inclusive, personalised, engaging, and effective" [@luckin2016intelligence]. A large body of work have committed to the development of AI models for AIEd tasks, including knowledge tracing [@corbett1994knowledge; @piech2015deep], question analysis [@yin2019quesnet; @liu2018finding], student score/grade prediction [@iqbal2017machine; @morsy2017cumulative; @almasri2019emt; @patil2017effective], educational content recommendation [@liu2019exploiting] and many more. With these models, an IES constantly updates the state of each student through constant feedback and provide appropriate learning experience to learners.

### Pre-training Methods in Education

Pre-training is the act of training a model to an unsupervised auxiliary task before the supervised main task [@erhan2010does]. Models using pre-training have been shown to outperform existing models in various fields including NLP [@devlin2018bert], Computer Vision [@studer2019comprehensive], Speech Recognition [@schneider2019wav2vec], 