During the period of the sultanate of Bengal, a Sufi saint, "Hazrat Shahjalal Dakhini" of Dhaka, and his followers were killed by agents of the sultan and buried on the site of Bangabhaban. The site soon became famous as a "mazhar" ("mausoleum") for the devotees of the saint. It is conjectured that it belonged to a zamindar during the period of British rule. Nawab Khwaja Abdul Ghani of Dhaka bought the site and built a bungalow there, which he named as "Dilkusha Garden".