Center for Data Science, Peking University; University of Oxford; Bytedance Inc
Keyu Tian, Yi Jiang, qishuai diao, Chen Lin, Liwei Wang, Zehuan Yuan
Self-Supervised Learning;Masked Autoencoding;Masked Pre-training;Masked Modeling;Convolutional Neural Networks
Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling