In 1590, Nobukatsu was deprived of his post by Hideyoshi TOYOTOMI, so she transferred to Koori in Owari Province dominated by the Ikoma clan, and this was evidently instructed by Hideyushi judging from the Shuinjo (a shogunal license for foreign trade) of Hideyoshi existing in "Haibara-ke Monjo" (Haibara Family's Written Material) as well as the fact that soon she lived back in Kyoto, and it seems that Gotoku's treatment was under the control of Hideyoshi.
しかし、天正18年（1590年）には信雄が豊臣秀吉によって改易されたため、生駒氏の尾張国の小折に移り住んだが、これは「埴原家文書」に残された秀吉の朱印状から秀吉による処置だったことが明らかで、その後すぐにまた京都に居住するなど、五徳の処遇は秀吉の支配下にあったことが推測できる。