Around 1183, he started the anti-Taira clan campaign in Hoki Province, and won the force of samurai group led by KI no Narimori over to his side, placing the half of Hoki Province and a part of Mimasaka Province under his control.
寿永2年頃、同国において反平氏行動を開始、紀成盛を中心とする武士団を味方に伯耆の半分と美作国の一部を支配下に入れた。