However, he was envied by the side that demanded expulsion of foreigners, demoted after impeachment that occurred soon after, and underwent house arrest and entered the priesthood.
しかし攘夷派に恨まれ、間もなく弾劾を受けて失脚、蟄居落飾処分となった。