Sultan was born in Machimdia village, in what was then Jessore District, British India (now Narail District, Bangladesh) on 10 August 1923. After five years of primary education at Victoria Collegiate School in Narail, he went to work for his father, a mason. Even as a child he felt a strong artistic urge. He seized every opportunity to draw with charcoal, and developed his talent depicting the buildings his father worked on. Sultan wanted to study art in Calcutta (Kolkata), but his family did not have the means to send him. Eventually, he secured financial support from the local zamindar and went to Calcutta in 1938.