1.	This paper shows little novelty, as it is only a marginal improvement of the conventional attention mechanism based on some span-level inductive bias.
2.	The performance improvement over simple baselines like max-pooling is also marginal for probing tasks.
3.	Although 4 patterns are proposed in this paper, it seems hard to find a unified solution or guideline for how to combine them in different tasks. If we have to try all possible combinations every time, the practicality of this method would be significantly reduced.
4. The authors validate the effectiveness of the proposed span-level attention only on the backbone of BERT. The paper lacks experiments on other encoder backbones to further demonstrate the generality of the proposed method.
5. The authors do not compare their methods with other state-of-the-art methods for span-related tasks, such as SpanBERT, thus lacking some credibility.
6. The writing can be improved. There are some typos and unclear descriptions. Please refer to comments for detail.
5. The authors do not compare their methods with other state-of-the-art methods for span-related tasks, such as SpanBERT, thus lacking some credibility.