• 2D and 3D information encoding are from previous work. This work just simply combines them, and the model architecture is the same as Graphormer[1]. The novelty is not enough.
• Supervised pretraining based on the prediction of homo-lumo gap may lead to negative transfer. For example, on QM9 in downstream experiments, Transformer-M performs poorly on most tasks other than homo, lumo, and gap. This may be contradictory to the description "general-purpose neural network model" claimed in this paper.
• Lack of description of PDBBind data processing and splitting in downstream tasks.
• Absence of some ablation experiments: ①(p2D, p3D, p2D&3D) = (1:0:0) / (0:1:0) / (0:0:1); ②Only using the 3D position denoising task while pretraining.
Other questions：
• Do the authors consider encoding 1D molecular data mode, e.g., SMILES, simultaneously?
• What do the authors think about the possibility of negative transfer on downstream tasks due to the supervised signal introduced during pretraining?
• Whether there is data leakage during finetuning on PDBBind, because we know that the general, refined, and core sets have overlapping parts.
• Supervised pretraining based on the prediction of homo-lumo gap may lead to negative transfer. For example, on QM9 in downstream experiments, Transformer-M performs poorly on most tasks other than homo, lumo, and gap. This may be contradictory to the description "general-purpose neural network model" claimed in this paper.