He was exiled to Noto Province (the eastern part of Ishikawa Prefecture) in 1595 by being implicated in the downfall of Hidetsugu TOYOTOMI, the chief adviser to the Emperor, but soon returned thanks to the arrangement made by Toshiie MAEDA and Ieyasu TOKUGAWA.
文禄4年（1595年）、関白・豊臣秀次の失脚に連座し能登国（石川県東部）に配流されたが、前田利家・徳川家康のとりなしもあってまもなく復帰した。