In 1528, Yoshiharu fled to Omi Province (present-day Shiga Prefecture) to seek shelter under Tanetsuna KUTSUKI, a general of Sengoku period, and stood against Yoshitsuna ASHIKAGA, who was then called Sakai Kubo, who was the head of the Sakai-based municipal government.
享禄元年（1528年）には朽木稙綱(戦国武将)を頼って近江国（現滋賀県）に落ち延び、元長らが擁立した堺公方・足利義維と対立した。