Yogaswami later spent years of intense tapas under the olive tree at Colombuthurai Road on the outskirts of Jaffna. His practice was to meditate for three days and nights in the open without moving about or taking shelter from the weather. On the fourth day he would walk long distances, returning to the olive tree to repeat the cycle. In his outward behaviour, Yogaswami followed the example of his guru, for he would drive away those who tried to approach him. After some years, he allowed a few sincere seekers to approach. As more and more devotees gathered around him, his austere demeanor relaxed. He was eventually persuaded to occupy a small hut in the garden of a house near his olive tree. This remained his 'base' for the rest of his life. Even here he initially forbade devotees to revere or care for him. Devotees would come to him for help in all their problems, usually in the early mornings and in the evenings. Day and night Yogaswami was absorbed in his inner worship. On one occasion, Yogaswami was seated in perfect stillness, like a stone. A crow flew down and rested several minutes on his head, apparently thinking this was a statue.