From Puri Swami moved to hill caves of Khandagiri, Dhauligiri, Nilagiri etc. near Bhubaneswar (capital of Orissa state). During that period Swami lived on water alone for twelve years and people used to call him "Nirahari Gosain". In 1838, Swami moved to Kapilash hill of Dhenkanal district of Orissa and spent twenty-one days in Atma Yoga Samadhi (unification of human mind) with "balkal" of the "kumbhi" tree (bark of careya arborea), leaving his clothes on a huge round stone. A great seven-hooded snake spread its hood covering his head. The nearby area of the forest was illuminated by the luster of the body of Swami. On the 21st day, a tribal, Sadananda of nearby Deogram village, witnessed the magnificent Atma Yoga Samadhi of Swami and served him fruits for twelve years during the stay of Swami at Kapilash hill. For this, Swami came to be known as "Phalahari Gosain". The next twelve years Swami survived on cow's milk alone, rendered as service by the ruler of Dhenkanal, King Bhagirathi Bhramarbar Bahadur. Both the king and the queen had the opportunity to witness Swami on top of Kapilash hill and would serve him milk in new earthen pots.