Thereafter, he was banished to Akita in Dewa Province and then to Iyo Province, but, shortly thereafter--on the occasion of the Bunroku-no-eki War in 1592 (Japan's first invasion of Korea)--he was pardoned by Hideyoshi TOYOTOMI through the mediation of Ieyasu, and returned to service as samurai and joined Shobanshu (officials who accompanied the Shogun).
その後、出羽国秋田、さらに伊予国に流されたが、間もなく家康の斡旋により、文禄元年（1592年）の文禄の役に際し、豊臣秀吉によって赦免されて再出仕し、相伴衆に加えられた。