Title: Representation Learning via Consistent Assignment of Views over Random Partitions. Abstract: We introduce Consistent Assignment of views over Random Partitions (CARP), a self-supervised clustering method for representation learning of visual features.
CARP learns prototypes in an end-to-end online fashion using gradient descent without additional non-differentiable modules to solve the cluster assignment problem.
We present a new pretext task based on random partitions of prototypes by enforcing consistency between views' assignments over these random subsets.
We use a fast (student) and a slow (teacher) learners to provide stable targets for the assignment task.
We present an extensive ablation study and show that our proposed random partition pretext task (1) improves the quality of the learned representations by devising multiple random classification tasks and (2) improves training stability and prevents collapsed solutions in joint-embedding training.
CARP achieves top-1 linear accuracy of 71.7% and k-NN performance of 64.8% on the ImageNet-1M, surpassing contemporary work under limited training conditions.
When trained for longer epochs, CARP outperforms state-of-the-art methods in the k-NN evaluation and performs comparably in other benchmarks.