When King Dragutin of house Nemanjić abdicated in favor of his brother Milutin, he retained control of Užice region and was given the Mačva region by the Hungarian king, from which he formed the Kingdom of Srem. After King Dragutin died, his lands were annexed to Serbia. The old Fortress on the hill was founded in the mid 14th century. After the death of Emperor Dušan the Mighty, in the period known as the 'fall of the Serbian Empire', Užice came under the control of Vojislav Vojinović, a nobleman in the service of Emperor Uroš the Weak. When Vojislav died, his nephew Nikola Altomanović controlled the region. When Uroš died childless, the former Imperial provincial lords begin fighting each other. "Serbian Autokrator" Lazar Hrebeljanović and Tvrtko I of Bosnia defeated Nikola Altomanović, and divided his lands between themselves. Nikola was blinded in the fortress on the orders of Stefan Musić. Užice came under the control of Lazar, then the Serbian Despotate under his son Stefan Lazarević.