Carnegie Mellon University & Google; Google
natural language processing;large-scale retrieval;unsupervised representation learning;paragraph-level pre-training;two-tower Transformer models
Pre-training Tasks for Embedding-based Large-scale Retrieval