Title: A Probabilistic Approach to Self-Supervised Learning using Cyclical Stochastic Gradient MCMC . Abstract: In this paper we present a practical Bayesian formulation for self-supervised learning method with Cyclical Stochastic Gradient Hamiltonian Monte Carlo (cSGHMC). Within this framework, we place a prior over the parameters of a self-supervised learning model and use cSGHMC to approximate the high dimensional and multimodal posterior distribution over the embeddings. By exploring an expressive posterior over the embeddings, the Bayesian self-supervised learning produces interpretable and diverse representations. Marginalising over these representations results improvement in semi-supervised learning and out-of-distribution detection tasks. We provide experimental results on multiple classification tasks in semi-supervised learning including Cifar10 and Cifar100. Moreover we demonstrate the effectiveness of the proposed method in out-of distribution detection task using SVHN dataset.