knowledge tracing task. In particular, we embed the following interactive features:

-   `start_time`: The month, day and hour of the absolute time when a student started to solve each exercise is recorded. We assign a latent vector to every possible combination of month, day and hour.

-   `response_correctness`: The value is 1 if a student response is correct and 0 otherwise. We assign a latent vector corresponding to each possible value 0 and 1.

-   `timeliness`: The value is 1 if a student responds within a specified time limit and 0 otherwise. We assign a latent vector corresponding to each possible value 0 and 1.

-   `longest_answer`: The value is 1 if a student selects an answer choice with the longest description. We assign a latent vector corresponding to each possible value 0 and 1.

Among the interactive features above, only `response_correctness` and `timeliness` serve as assessments. However, we study various combinations of the above interactive features for pre-training in Section \[sec:experiments\].

Let $e_t$ be the sum of the embedding vectors of `exercise_id`, `exercise_category` and `position`. Likewise, let $c_t$ and $t_t$ be the embedding vectors of `response_correctness` and `timeliness` respectively. Then, the representation of interaction $I_t$ is $e_t + c_t 