He was transferred to Kitsuki-jo Castle in 1597 because of his fault during the dispatch of troops to Korea, however, four members of Gotairo (Council of Five Elders) after Hideyoshi's death admitted that he had no fault with his performance as the reinforcement at the time of siege in Urusan-jo Castle, so that he regained his position of the castellan of Funai-jo Castle in March 19, 1599.
朝鮮出兵時に落ち度があったとして、慶長2年（1597年）に杵築城に移されるが、秀吉亡き後の四人の五大老により、蔚山籠城時の援軍としての働きに落ち度はなかったとして、慶長4年（1599年）閏3月19日、に再び府内城主となる。