However, in 1599 following questioning by Ieyasu, he was banished to Shimotsuke Province (Tochigi Prefecture), under suspicion of being a ringleader of a failed plot to assassinate Ieyasu that had been hatched by servants of Tokugawa Ieyasu and Masanobu HONDA.
しかし慶長4年（1599年）、徳川家康と本多正信の主従が策した家康暗殺疑惑事件の首謀者の一人として家康にその罪を問われ、下野国（栃木県）に流罪とされてしまう。