Ernest was born in Bruck an der Mur in Styria, the third son of Duke Leopold III of Austria (1351–1386) and his consort Viridis Visconti (d. 1414), a daughter of Bernabò Visconti, Lord of Milan. Shortly after his birth, his father and his uncle Albert III divided the Habsburg lands by the 1379 Treaty of Neuberg: while Albert and his Albertinian descendants would rule over the Duchy of Austria proper, the Leopoldian line received the Inner Austrian states of Styria, Carinthia and Carniola with the remaining March of Istria, as well as Tyrol and the Further Austrian possessions. After Leopold's death in the 1386 Battle of Sempach, young Ernest and his brothers William, Leopold IV and Frederick IV remained under the guardianship of their uncle Albert III.