Though his family was once fell on bad time as his father lost his position while involved in intrigue during the incident of Hidetsugu TOYOTOMI, Yasushige passed genpuku (ceremony of attaining manhood) in 1603 after the death of Hideyoshi TOYOTOMI, and was designated as Shorokuinojo Ukone no shogen and Kurodo (Senior Sixth Rank, Upper Grade, Lieutenant of the Right Division of Inner Palace Guards and Chamberlain), and promoted to Nakatsukasa Taijo in 1612.
父は豊臣秀次の事件に連座して失脚したため、一時衰微していたが、豊臣秀吉死後の慶長8年（1603年）に元服して正六位上右近将監兼蔵人に任じられ、同17年（1612年）には中務大丞に転じる。