Yoshinaka, Yoshitaka's father, raised his army in 1183, taking Hokuriku no miya who was a bereaved child of Prince Mochihito as the master, and expanded his influence mainly in Shinano Province, which posed a conflict against MINAMOTO no Yoritomo who also belonged to the Minamoto clan.
寿永2年（1183年）、挙兵した父義仲は以仁王の遺児・北陸宮を奉じて信濃国を中心に勢力を広げ、同じ源氏の源頼朝とは独立した勢いを見せた。