- The technical novelty and contributions are a bit limited. The overall idea of using a transformer to process time series data is not new, as also acknowledged by the authors. The masked prediction was also used in prior works e.g. MAE (He et al., 2022). The main contribution, in this case, is the data pre-processing approach that was based on the bins. The continuous value embedding (CVE) was also from a prior work (Tipirneni & Reddy 2022), and also the early fusion instead of late fusion (Tipirneni & Reddy, 2022; Zhang et al., 2022). It would be better to clearly clarify the key novelty compared to previous works, especially the contribution (or performance gain) from the data pre-processing scheme.
- It is unclear if there are masks applied to all the bins, or only to one bin as shown in Fig. 1.
- It is unclear how the static data (age, gender etc.) were encoded to input to the MLP. The time-series data was also not clearly presented.
- It is unclear what is the "learned [MASK] embedding" mean in the SSL pre-training stage of the proposed method.
- The proposed "masked event dropout scheme" was not clearly presented. Was this dropout applied to the ground truth or the prediction? If it was applied to the prediction or the training input data, will this be considered for the loss function?
- The proposed method was only evaluated on EHR data but claimed to be a method designed for "time series data" as in both the title and throughout the paper. Suggest either tone-down the claim or providing justification on more other time series data.
- The experimental comparison with other methods seems to be a bit unfair. As the proposed method was pre-trained before the fine-tuning stage, it is unclear if the compared methods were also initialised with the same (or similar scale) pre-trained model. If not, as shown in Table 1, the proposed method without SSL performs inferior to most of the compared methods.
- Missing reference to the two used EHR datasets at the beginning of Sec. 4.
- The experimental comparison with other methods seems to be a bit unfair. As the proposed method was pre-trained before the fine-tuning stage, it is unclear if the compared methods were also initialised with the same (or similar scale) pre-trained model. If not, as shown in Table 1, the proposed method without SSL performs inferior to most of the compared methods.