DatologyAI; NYU; Kempner Institute, Harvard; NYU, Genentech, CIFAR LMB
Angelica Chen, Ravid Shwartz-Ziv, Kyunghyun Cho, Matthew Leavitt, Naomi Saphra
interpretability;BERT;syntax;phase changes;simplicity bias;training dynamics
Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs