On his wife Mary of Burgundy's demise in 1482, Maximilian of Austria gave to his son Philip the Handsome, still a child, his mother's inheritance, thus Hainaut passed to Austria. As to Le Quesnoy, it no longer served as favorite home to the new princes: the remoteness of the court was detrimental to the town. In 1492, the population which had previously more than 800 heads of families was reduced to a quarter of its former size. The Court had moved to Mons with many middle-class families and many trades following. Le Quesnoy from this time was seen as a stronghold under the authority of lords and captains-governors: in 1478, The Lord de Maingoval, Count of Chimay took this role. In 1493, we find Robert de Melun, in 1499, John of Luxembourg, in 1511, Philippe de Belleforière, Lord of Romeries and of Caudry. Maximilian of Austria kept a considerable bastion in Le Quesnoy to withstand the incessant incursions of the French settled in Cambrai. At that time, the religious communities in the town, already very numerous, developed in peace yet appointments of the heads of these establishments had to be made with the consent of the central government. Philip the Handsome (), the ruler upon attaining the age of majority, married Joanna of Castile and Aragon. In 1500 Philip gave the castle of Le Quesnoy to his sister Margaret of Austria (later regent of the Netherlands under her godson Charles V). Philip, the local prince born, raised and loved by people in the Netherlands died in 1506, leaving two infant sons: Charles of Luxembourg, later Charles V of Spain and Ferdinand I, Holy Roman Emperor, the succession of the nations.