He returned to Kawachi Province, where he formed an alliance with Yoshikata ROKKAKU and continued his feud with the Miyoshi clan however, on Nov 30, 1558 he was banished from the castle of Takaya KAWACHI and escaped to Sakai City following a falling-out with Shugodai (deputy military governor) Munefusa YASUMI.
その後は河内に戻り六角義賢と手を結んだうえで三好氏と抗争を続けていたが、永禄元年（1558年）11月30日、守護代の安見宗房の離反によって居城の河内高屋城を追放されて堺市に逃れる。