Title: Exploring Generalization of Non-Contrastive self-supervised Learning. Abstract: Contrastive learning have recently produced results comparable to the state-of-the-art supervised models. Non-contrastive methods do not use negative samples, but separate samples of different classes by explicitly or implicitly optimizing the representation space. Although we have some understanding of the core of the
non-contrastive learning method, theoretical analysis of its generalization performance is still missing. Thus we present a theoretical analysis of generalizability of non-contrastive models. We focus on the inter-class distance, show how non-contrastive methods increase the inter-class distance, and how the distance affects the
generalization performance of the model. We find that the generalization of non-contrastive methods is affected by the output dimension and the number of latent classes. Models with much fewer dimensions than the number of latent classes are not sufficient to generalize well. We demonstrate our findings through experiments on the CIFAR dataset.