QUESTIONS FOR THE AUTHORS - recently, [1] criticized the usage of standard similarity measures such as cosine and Euclidean distance with contextualized embeddings, given that such metrics might be dominated by a small number of outlier dimensions. Do you think this could be affecting your results? Have you thought about using standardized metrics or a rank-based metric (e.g. Spearman correlation between vectors)?
- Section 5.1: "... in cases where a verb is split into multiple subwords, we take the embedding of the first subword token as the verb embedding". I am not sure I understand the motivation of this step. Could you please elaborate more? Why not taking e.g. the average of the subword embeddings?
TYPOS AND MINOR ISSUES: l. 264 LSTMS --> LSTMs l. 301 is most similar --> is the most similar EXTRA REFERENCES [1] Timkey and Van Schjindel, 2021. All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality. Proceedings of EMNLP.
301 is most similar --> is the most similar EXTRA REFERENCES [1] Timkey and Van Schjindel, 2021. All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality. Proceedings of EMNLP.