In the 1850s, additional building was constructed next to Old Konak, to the north, and colloquially called "Mali dvor" or "Mali konak" ("Little Palace" or "Little Konak"). When the Obrenović dynasty was restored in 1858, they moved in the residences. As an heir apparent, Prince Michael used the Little Palace. The palaces were surrounded by the auxiliary buildings, servants' quarters, horse stables, etc. Ruling prince Miloš Obrenović, Michael's father, decided in 1858 to build a new palace for his son, which was built next to the palace's garden, to the south on the location of modern Novi Dvor, which became known as the "Dvor sa kulama" ("Palace with towers"). Upon ascending to the throne in 1860, Michael decided to use the Old Konak, while part of the state administration was located in the Palace with towers (ministries of foreign and internal affairs).