Title: Self-Supervised Extreme Compression of Gigapixel Images. Abstract: Whole slide images (WSI) are microscopy images of stained tissue slides routinely prepared for diagnosis and treatment selection in clinical practice. WSI are very large (gigapixel size) and complex (made of up to millions of cells). The current state-of-the-art (SoTA) approach to classify WSI subdivides them into tiles, encodes them by pre-trained networks, and applies Multiple Instance Learning (MIL) to train for specific downstream tasks. However, annotated datasets are often small, typically a few hundred to a few thousand WSI, which may cause overfitting and underperforming models. On the other hand, the number of unannotated WSI is ever increasing, with datasets of tens of thousands (soon to be millions) of images available. Nevertheless, using unannotated WSI is limited due to the challenges of extending self-supervised learning from natural images to WSI. We propose a strategy of slide-level self-supervised learning (SSL) to leverage the large number of images without annotations to infer powerful slide representations. The resulting embeddings allow compression of the whole public WSI dataset available at the Cancer-Genome Atlas (TCGA), one of the most widely used data resources in cancer research, from 16 TB to 23 MB, thus dramatically simplifying future studies in the field of computational pathology in terms of data storage and processing. We show that a linear classifier trained on top of these embeddings maintains or improves previous SoTA performances on various benchmark WSI classification tasks. Finally, we observe that training a classifier on these representations with tiny datasets (e.g. 50 slides) improved performances over SoTA by an average of +6.3 AUC points over all downstream tasks. We further analyze the conditions necessary for such a training framework to succeed, bringing insights into WSI processing.