However, since he got onto bad terms with his new colleague, Kataharu YANAGIMOTO (a younger brother of Tanemichi HATANO and an older brother of Motomori KOZAI), in 1529 he withdrew to Awa Province to live there.
しかし、享禄2年（1529年）には新たに同僚となった柳本賢治（波多野稙通の弟、香西元盛の兄）らと折り合いを悪くした為、阿波に逼塞する。