1. The task setup is not described clearly. For example, which notes in the EHR (only the current admission or all previous admissions) do you use as input and how far away are the outcomes from the last note date?
2. There isn't one clear aggregation strategy that gives consistent performance gains across all tasks. So it is hard for someone to implement this approach in practice.
1. Experimental setup details: Can you explain how you pick which notes from the patient's EHR do you use an input and how far away are the outcomes from the last note date? Also, how do you select the patient population for the experiments? Do you use all patients and their admissions for prediction? Is the test set temporally split or split according to different patients?
2. Is precision more important or recall? You seem to consider precision more important in order to not raise false alarms. But isn't recall also important since you would otherwise miss out on reporting at-risk patients?
3. You cannot refer to appendix figures in the main paper (line 497). You should either move the whole analysis to appendix or move up the figures.
4. How do you think your approach would compare to/work in line with other inputs such as structured information? AUCROC seems pretty high in other models in literature.
5. Consider explaining the tasks and performance metrics when you call them out in the abstract in a little more detail. It's a little confusing now since you mention mortality prediction and say precision@topK, which isn't a regular binary classification metric.
1. The task setup is not described clearly. For example, which notes in the EHR (only the current admission or all previous admissions) do you use as input and how far away are the outcomes from the last note date?