AI21 Labs, Tel Aviv, Israel
Yoav Levine, Barak Lenz, Opher Lieber, Omri Abend, Kevin Leyton-Brown, Moshe Tennenholtz, Yoav Shoham
Language modeling;BERT;pointwise mutual information
PMI-Masking: Principled masking of correlated spans