The prince earned himself a reputation as a patron, but also as a spendthrift. He spent most time at his castle in Vienna, but when his relatives the Dukes of Saxe-Hildburghausen were heavily indebted, he became by order of the Holy Roman Emperor Joseph II in 1769 manager of the Duchy, in order to avoid the bankruptcy of Saxe-Hildburghausen. His grandnephew, Duke Ernst Frederick III, was incapacitated to rule. When he died (1780) he left a young heir, the seventeen years old prince Frederick, over whom Joseph took on the role of prince-regent, which he retained until his own death, aged eighty-four.