However, in 1599, he incurred chokkan (the emperor's censure) for a scandal that included his father Atsumichi and son Michiyo and a court lady, then he and his father were expelled from Kyoto.
しかし慶長4年（1599年）、敦通・通世父子と宮中の女官との醜聞のことで天皇の勅勘を被り、父とともに京都を追放された。