Gilani died in the evening of Tuesday, February 21, 1166 (11th Rabi' al-thani 561 AH) at the age of ninety one years according to the Islamic calendar. His body was entombed in a shrine within his madrasa in Babul-Sheikh, Rusafa on the east bank of the Tigris in Baghdad, Iraq. During the reign of the Safavid Shah Ismail I, Gilani's shrine was destroyed. However, in 1535, the Ottoman Sultan Suleiman the Magnificent had a "turba" (dome) built over the shrine, which exists to this day.